Skip to content

expand, unexpand: flush output after each line - #15293

Open
r3wretrhy wants to merge 3 commits into
uutils:mainfrom
r3wretrhy:expand-unexpand-stdin-flush
Open

r3wretrhy wants to merge 3 commits into
uutils:mainfrom
r3wretrhy:expand-unexpand-stdin-flush

Conversation

@r3wretrhy

Copy link
Copy Markdown
Contributor

When stdin is read line-at-a-time (interactive terminal, or any producer that pauses between lines), expand / unexpand held converted output in a BufWriter and only flushed on exit. An interactive session therefore showed no converted line until the user quit, while GNU shows each line immediately.

Like cut, skip BufWriter when stdout is a terminal. On the buffered path, flush once per input chunk that completed a line (before the next read), so line-at-a-time pipes also see output promptly without a write per line on large inputs.

$ expand -t3
        a          # typed tab+a, Enter
   a               # GNU shows this right away; we used to wait until quit

Added test_stdin_tty_flushes_each_line to test_expand.rs and test_unexpand.rs.

Closes #11091

When reading line-at-a-time from stdin, BufWriter only flushed on exit, so
an interactive session showed no converted output until the user quit. Match
cut by skipping BufWriter when stdout is a terminal, and on the buffered path
flush once per input chunk that completed a line, so piped line-at-a-time
producers also see prompt output without a write per line on large inputs.

Fixes uutils#11091
@codspeed

codspeed Bot commented Oct 13, 2026 •

Copy link
Copy Markdown

Merging this PR will regress 1 benchmark

⚠️ 48 benchmarks spent significant time in system calls

System calls cannot be consistently instrumented, so they are not included in the measure, which understates the real cost. Please switch to the Walltime instrument to accurately measure system calls.

Measurement and system calls

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

⚡ 3 improved benchmarks
❌ 1 regressed benchmark
✅ 76 untouched benchmarks
⏩ 391 skipped benchmarks1

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Benchmark BASE HEAD Efficiency
❌ factor_three_39_bit_primes 715.7 ms 795.8 ms -10.06%
⚡ factor_five_38_bit_primes 2 s 1.8 s +13.46%
⚡ expand_many_lines[100000] 115.9 ms 106 ms +9.31%
⚡ expand_custom_tabstops[50000] 31.6 ms 29.7 ms +6.51%

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.


Comparing r3wretrhy:expand-unexpand-stdin-flush (a56ecf7) with main (98194d4)2

Open in CodSpeed

Footnotes

  1. 391 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

  2. No successful run was found on main (9f52800) during the generation of this report, so 98194d4 was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

@oech3

oech3 commented Oct 13, 2026

Copy link
Copy Markdown
Contributor

Can we avoid large performance drop?

Keep the plain BufWriter<Stdout> path and flush before the next read only
when the input is stdin and stdin is not a regular file. File arguments and
redirected regular files are fully buffered again, avoiding the slowdown.
@github-actions

Copy link
Copy Markdown

GNU testsuite comparison:

Congrats! The gnu test tests/touch/now-owned-by-other is no longer failing!

Comment thread src/uu/expand/src/expand.rs Outdated
Comment thread src/uu/expand/src/expand.rs Outdated
Comment thread tests/by-util/test_expand.rs Outdated
@r3wretrhy

Copy link
Copy Markdown
Contributor Author

@oech3 The perf drop is fixed in 0a02252, it only flushes when stdin is a pipe/tty now. CodSpeed shows expand_many_lines +9% and expand_custom_tabstops +6.5% faster. The factor regression looks unrelated since this PR doesn't touch factor. Also enabled rustix for all unix targets so it builds on macOS/BSD.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

expand/unexpand: no output if input is read from stdin

2 participants