Skip to content

Measure this type against a bare double, and chart it per release - #91

Merged
matt-edmondson merged 2 commits into
mainfrom
claude/magical-knuth-idzj5r
Sep 16, 2026
Merged

matt-edmondson merged 2 commits into
mainfrom
claude/magical-knuth-idzj5r

Conversation

@matt-edmondson

Copy link
Copy Markdown
Contributor

Adds AbstractionCostBenchmarks and a third chart section carrying what it measures. The same class, with the same loops and the same methodology, goes into ktsu.PreciseNumber (#85) and ktsu.Semantics, so the three libraries answer one question the same way and their answers are comparable with each other as well as with double.

the chart as it stands

What it says

Every other benchmark here answers "how long does this take", which is only readable beside something. This supplies the something — the primitive a caller would otherwise have used.

release Add Multiply
1.3.0 1377.7× 5809.6×
1.4.0 1393.4× 5453.4×
1.4.20 1346.1× 5939.4×
1.4.40 1352.4× 5944.6×
2.0.0 89.7× 550.5×
2.0.1 89.9× 569.3×

Becoming a value type in 2.0 took roughly 15× off add and 10× off multiply.

And here is the thing neither chart could say alone: ktsu.PreciseNumber underneath moved by about 15% across the same change (99.7× → 82.8× on add). So most of what 2.0 recovered here was this layer's own allocation, not the number beneath it. That is a conclusion drawn by reading the two charts together, which is the point of them carrying the same benchmark.

The ratio is not expected to be 1 and is not a defect for being large — every operation pays twice, once for the arbitrary-precision arithmetic and again for rounding back to the significance the operands justify, and both buy something a double cannot do at all. What the section is for is noticing the day it moves.

How the number should be read

Three things decide that, and all three are in the class's remarks:

  • These are loops, deliberately. A single operation over operands that do not change is loop-invariant and the JIT hoists it out — which would leave the double side indistinguishable from an empty method, and a ratio against an empty method means nothing. Each iteration feeds the next, so there is nothing to hoist and both sides are measurable.
  • The loop's own cost biases toward 1, being paid identically by both sides, so a ratio is a floor on the real cost rather than the whole of it.
  • Both loops accumulate rather than compound. The number underneath carries as many digits as the arithmetic produces, so a compounding chain would measure that growth instead of the operation. Operands are also chosen to be values a double can hold, so the two sides do the same arithmetic on the same numbers. How the cost grows with digits is a different question, answered by ArithmeticBenchmarks across its Digits axis.

The chart section

It derives its ratio from two benchmarks already stored rather than recording a new field. So there is no schema change, no history to rewrite, and an entry gathered before the pair existed simply draws as "not measured".

The fields carry = default! because this type was a class before 2.0, and the backfill measures those releases too.

Verification

  • dotnet build -c Release — clean, 0 warnings.
  • dotnet test -c Release — 95 passed, 0 failed.
  • The table above is a real local seeding run: all 6 releases measured in one pass against a single reference reading, so the ratios are comparable as they stand. Every entry was checked to carry both halves of the pair rather than assumed to.

🤖 Generated with Claude Code

https://claude.ai/code/session_017jrnV7N94UGL8fDRRE8Xt8


Generated by Claude Code

…tch]

Adds AbstractionCostBenchmarks and a third chart section carrying what it
measures. The same class, with the same loops and the same methodology, goes
into ktsu.PreciseNumber and ktsu.Semantics, so the three libraries answer one
question the same way and their answers are comparable with each other as well
as with double. Against PreciseNumber in particular the difference is what
significance tracking adds, since this type is built on that one.

Every other benchmark here answers "how long does this take", which is only
readable beside something. This supplies the something: the primitive a caller
would otherwise have used. The bare method is the BenchmarkDotNet baseline, so
the answer is the Ratio column rather than two rows divided by hand. It is not
expected to be 1.00 and is not a defect when it is not -- every operation pays
twice, once for the arithmetic and again for rounding back to the significance
the operands justify, and both buy something a double cannot do at all. The
number is for watching that cost across releases.

The section derives its ratio from two benchmarks already stored rather than
recording a new field, so there is no schema change, no history to rewrite, and
an entry gathered before the pair existed simply draws as not measured.

Three things decide how the number should be read, and all three are in the
remarks. The loops exist because a single operation over operands that do not
change is loop-invariant and the JIT hoists it out, leaving a double method
indistinguishable from an empty one and a ratio against it meaningless. The
loop's own cost is paid by both sides and pulls the ratio toward 1.00, so a
ratio is a floor rather than the whole cost. And both loops accumulate rather
than compound, because the number underneath carries as many digits as the
arithmetic produces and a compounding chain would measure that growth instead.

The fields are initialised to default! because this type was a class before 2.0,
and the backfill measures those releases too.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017jrnV7N94UGL8fDRRE8Xt8
Six releases measured in one pass against a single reference reading, so the
ratios are comparable as they stand:

            add        multiply
    1.3.0   1377.7x    5809.6x
    1.4.0   1393.4x    5453.4x
    1.4.20  1346.1x    5939.4x
    1.4.40  1352.4x    5944.6x
    2.0.0     89.7x     550.5x
    2.0.1     89.9x     569.3x

Becoming a value type in 2.0 took roughly 15x off add and 10x off multiply.
ktsu.PreciseNumber underneath moved by about 15% across the same change, so
most of what 2.0 recovered here was this layer's own allocation rather than the
number beneath it -- which is a thing the two charts can now be read together to
say, and neither could say alone.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017jrnV7N94UGL8fDRRE8Xt8
@sonarqubecloud

Copy link
Copy Markdown

@matt-edmondson
matt-edmondson merged commit aa8dec7 into main Sep 16, 2026
12 checks passed
@matt-edmondson
matt-edmondson deleted the claude/magical-knuth-idzj5r branch September 16, 2026 12:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants