Skip to content

dd: ignore a failed cache drop while copying - #14748

Open
ChihweiLHBird wants to merge 1 commit into
uutils:mainfrom
ChihweiLHBird:zhiwei/fix-dd-nocache-on-a-pipe
Open

ChihweiLHBird wants to merge 1 commit into
uutils:mainfrom
ChihweiLHBird:zhiwei/fix-dd-nocache-on-a-pipe

Conversation

@ChihweiLHBird

@ChihweiLHBird ChihweiLHBird commented Sep 20, 2026 •

Copy link
Copy Markdown
Contributor

Summary

iflag=nocache and oflag=nocache diagnosed every failed cache drop, so copying from or to a pipe printed failed to discard cache for: 'standard input': Illegal seek once per read and exited 1 even though the copy succeeded. GNU ignores those failures while copying and diagnoses them only with count=0.

Source::discard_cache and Dest::discard_cache now return the error, Input and Output pass it through, and the callers decide: dd_copy ignores it, the count=0 path reports it and sets the exit status.

nocache is accepted only on Linux and Android, so other targets are unaffected; the platform stub moved from Input/Output into Source/Dest, which lets the wrappers be plain pass-throughs.

Part of #14705 (tests/dd/nocache.sh).

Example

dd if=/dev/zero bs=1M count=1 status=none | dd iflag=nocache bs=1M of=/dev/null

Before: one failed to discard cache line per read, exit 1. After: silent, exit 0, as GNU 9.12.

count=0 still diagnoses the failure:

$ dd count=0 status=none | dd iflag=nocache count=0 status=none; echo $?
dd: failed to discard cache for: 'standard input': Illegal seek
1

@ChihweiLHBird
ChihweiLHBird marked this pull request as ready for review September 20, 2026 08:44
@codspeed

codspeed Bot commented Sep 20, 2026 •

Copy link
Copy Markdown

Merging this PR will not alter performance

⚠️ 9 benchmarks spent significant time in system calls

System calls cannot be consistently instrumented, so they are not included in the measure, which understates the real cost. Please switch to the Walltime instrument to accurately measure system calls.

Measurement and system calls

✅ 18 untouched benchmarks
⏩ 454 skipped benchmarks1


Comparing ChihweiLHBird:zhiwei/fix-dd-nocache-on-a-pipe (cfd839d) with main (774cca5)2

Open in CodSpeed

Footnotes

  1. 454 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

  2. No successful run was found on main (fb68e6c) during the generation of this report, so 774cca5 was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

@github-actions

github-actions Bot commented Sep 20, 2026 •

Copy link
Copy Markdown

GNU testsuite comparison:

Congrats! The gnu test tests/dd/nocache is no longer failing!

@ChihweiLHBird
ChihweiLHBird force-pushed the zhiwei/fix-dd-nocache-on-a-pipe branch from 5b69be5 to ae1248c Compare September 20, 2026 16:54
@ChihweiLHBird

Copy link
Copy Markdown
Contributor Author

I looked into the CodSpeed report. The regression is real but small, and it comes from compiler register allocation rather than from any added work.

benchmark executed instructions vs main
dd_copy_default (512 B blocks) +0.68%
dd_copy_8k_blocks +0.04%
dd_copy_with_skip +0.11%
dd_copy_with_seek +0.11%

Cause. None of these benchmarks pass nocache to dd, so the changed code never runs in them. As a diagnostic I built the PR with the three nocache blocks deleted from the copy loop; that build matches main's instruction count exactly, so all of the extra instructions come from how the compiler lays out the loop around those blocks. The PR changes what the call inside them does (return the error instead of printing it), which changes the compiler's register plan for the whole loop: the same work is done with about ten more register-to-stack moves per block. The default benchmark copies 65,536 blocks, which is where 0.68% comes from.

I couldn't find a variant of this fix reaches zero instructions increase. Another variant, adding a bunch of quietly_discard_cache functions, would lower +0.68% to +0.52%, but the code structure can look, in my opinion, less readable, that way. Let me know if the performance if prioritized though, and I would be happy to adopt that design.

@ChihweiLHBird
ChihweiLHBird force-pushed the zhiwei/fix-dd-nocache-on-a-pipe branch 6 times, most recently from c77a8fa to e99a685 Compare September 27, 2026 19:31
Comment thread src/uu/dd/src/dd.rs
Comment thread src/uu/dd/src/dd.rs
// Using len=0 in posix_fadvise means "to end of file"
if i.settings.iflags.nocache {
i.discard_cache(0, 0);
let file = i

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

almost the same block twice for input and output, could be a small helper taking the name and the result, no?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think building a helper for them is a good idea. i and o are different type, and the error messages in translate! macro are also different. Building a helper might make it looks much messy than the current shape

Comment thread tests/by-util/test_dd.rs Outdated
@ChihweiLHBird
ChihweiLHBird force-pushed the zhiwei/fix-dd-nocache-on-a-pipe branch 2 times, most recently from f200407 to 74118f5 Compare September 28, 2026 19:17
@ChihweiLHBird
ChihweiLHBird force-pushed the zhiwei/fix-dd-nocache-on-a-pipe branch from 74118f5 to 5650bed Compare September 28, 2026 19:27
Comment thread src/uu/dd/src/dd.rs Outdated

// TODO: Is there a way to discard filesystem cache on other targets?
#[cfg(not(any(target_os = "linux", target_os = "android", target_os = "freebsd")))]
#[allow(

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

these wrappers are just pass-throughs now, could the non-linux stub live in Source/Dest::discard_cache instead? that would drop both #[allow]

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved. But it adds an extra stub inside Source/Dest

@ChihweiLHBird
ChihweiLHBird force-pushed the zhiwei/fix-dd-nocache-on-a-pipe branch from 5650bed to 8279227 Compare October 9, 2026 00:19
Copilot AI balanced review requested due to automatic review settings October 9, 2026 00:19

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

GNU-source provenance must be clarified and the changed output-side count=0 path needs regression coverage.

2 open findings
What changed in this PR

Updates dd cache-discard handling so advisory failures during copying are ignored while count=0 failures remain fatal.

Changes:

  • Return cache-discard errors to callers.
  • Preserve diagnostics for count=0.
  • Add pipe-copy regression coverage.
File Description
src/​uu/​dd/​src/​dd.rs Moves cache-error policy to callers.
src/​uu/​dd/​src/​bufferedoutput.rs Propagates cache-discard results.
tests/​by-util/​test_dd.rs Tests silent pipe copying with nocache.

🧠 Review effort: Balanced


💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread src/uu/dd/src/dd.rs
Comment thread tests/by-util/test_dd.rs
@ChihweiLHBird
ChihweiLHBird force-pushed the zhiwei/fix-dd-nocache-on-a-pipe branch from 8279227 to 40f78b8 Compare October 9, 2026 07:13
Copilot AI balanced review requested due to automatic review settings October 9, 2026 07:13

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The unsupported-target count=0 failure contradicts the stated unchanged behavior and lacks coverage.

3 open findings

🧠 Review effort: Balanced

Comment thread src/uu/dd/src/dd.rs Outdated
@ChihweiLHBird
ChihweiLHBird force-pushed the zhiwei/fix-dd-nocache-on-a-pipe branch from 40f78b8 to f457f28 Compare October 9, 2026 07:48
Copilot AI balanced review requested due to automatic review settings October 9, 2026 07:48

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Redirected regular-file stdin is incorrectly classified as non-seekable on Unix targets without fadvise.

1 open finding
3 resolved since last review

🧠 Review effort: Balanced

Comment thread src/uu/dd/src/dd.rs Outdated
@ChihweiLHBird
ChihweiLHBird marked this pull request as draft October 9, 2026 07:57
@ChihweiLHBird
ChihweiLHBird force-pushed the zhiwei/fix-dd-nocache-on-a-pipe branch from f457f28 to fe75807 Compare October 9, 2026 08:05
@ChihweiLHBird
ChihweiLHBird requested a balanced review from Copilot October 9, 2026 08:09
@ChihweiLHBird
ChihweiLHBird marked this pull request as ready for review October 9, 2026 08:10

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The implementation matches the stated behavior and includes focused regression coverage.

0 open findings

1 resolved since last review

🧠 Review effort: Balanced

Comment thread src/uu/dd/src/dd.rs Outdated
Comment thread src/uu/dd/src/dd.rs
@ChihweiLHBird
ChihweiLHBird force-pushed the zhiwei/fix-dd-nocache-on-a-pipe branch from fe75807 to ee528b4 Compare October 10, 2026 02:05
Copilot AI balanced review requested due to automatic review settings October 10, 2026 02:05

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The focused behavior change is consistent across call paths and has suitable regression coverage.

0 open findings

🧠 Review effort: Balanced

Signed-off-by: Zhiwei Liang <zhiwei.liang@zliang.me>
Copilot AI balanced review requested due to automatic review settings October 13, 2026 00:20
@ChihweiLHBird
ChihweiLHBird force-pushed the zhiwei/fix-dd-nocache-on-a-pipe branch from ee528b4 to cfd839d Compare October 13, 2026 00:20

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The focused behavior change is correctly propagated and covered across copying and count=0 paths.

0 open findings

🧠 Review effort: Balanced

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants