Chart performance per release, and show it in the README - #242
Conversation
Adds a BenchmarkDotNet suite over the quantity system, and a workflow that measures a fixed subset of it once per release, appends the numbers to a committed history file, and redraws the chart the README shows. Every class is generic and carries GenericTypeArguments for double, float, decimal and PreciseNumber, because that is the axis this library varies along. A quantity is a readonly record struct over its T holding a value in the SI base unit, so an operator is the storage type's arithmetic and a struct initialiser, and a factory is that plus one multiplication. What a quantity costs is therefore mostly what its T costs, and a release changes it per T: building one measures 2 ns as a float and 121 ns as a PreciseNumber, and a vector length measures 1 ns as a double and 520 ns as a decimal, which is the Newton refinement StorageMath.Sqrt falls back to when there is no hardware root to take. Operands are parsed from text rather than converted from a double, so that a storage type wider than 15 digits is measured carrying its own digits rather than a double's. One benchmark cannot be measured and says so. A relationship operator over a binary floating point type is one instruction over operands the loop does not change, so the JIT hoists it out and BenchmarkDotNet reports the method as indistinguishable from an empty one. Nothing fixes that without measuring the fix instead of the operator -- an operand array adds a load, a mutated field adds a store -- so those rows are documented as being below the harness's floor and the chart draws none of them, taking a unit conversion in their place. Times are made comparable across CI hosts by BaselineBenchmarks, a fixed integer loop measured in the same job that the chart divides every time by. Allocation needs none of that, being the same on any machine. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017jrnV7N94UGL8fDRRE8Xt8
Ten releases, 3.3.1 through 5.3.2, all measured in one pass against a single
reference reading so that the times are comparable as they stand. Without this
the README would show an empty chart until the next release, and a one-point
chart after it. Every one of the ten answered all twenty benchmarks; nothing
had to be skipped.
The chart is not all one direction, and the two steps that go the other way
are both deliberate and both already documented:
* a decimal vector length steps from 165ns to 507ns at 5.2.0, where
StorageMath.Sqrt stopped taking a double round trip and started refining
with Newton steps in the storage type's own arithmetic. That is what buys
a decimal length its 28 significant digits rather than a double's 15.
* a decimal unit factory steps from 7.6ns to 15.3ns at 5.2.4, where the
converted factors moved out of a static initializer into a nullable field
per value. That is what stopped one factor too large for a type making
every conversion for that type throw TypeInitializationException; the cost
is a null check per read.
3.3.1 to 4.0.0 is the record struct, and it cuts both ways: building a
quantity went from 27.4ns and 48 bytes to 3.4ns and nothing, while comparing
two through IPhysicalQuantity<T> went from 4.8ns and nothing to 14.2ns and 24
bytes, because what had been a reference conversion on a class became a box on
a struct.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017jrnV7N94UGL8fDRRE8Xt8
|
That is not a compile error in anything this PR adds — it is
So the generator compiled cleanly, its output was removed while a second build of the same project was in flight, and a consumer read the path inside that window. Sonar processed Three things say the tree itself is fine:
I have re-run the failed job once to confirm. If it comes back green this was a build race and I will say so and leave the diff alone; if it fails the same way, it is deterministic and I will root-cause it rather than re-running again. The one way this PR could plausibly be involved is build-graph width: Generated by Claude Code |
|
|
The re-run is green — So that was the build race the timeline described, not the diff, and I have left the change alone. Recording it here because it can recur: the generator's output is written and rewritten within one pipeline pass, and a consumer reading No further re-runs from me on this PR; a second failure would be treated as real. Generated by Claude Code |
2 participants
Adds a BenchmarkDotNet suite over the quantity system, a workflow that measures a fixed subset of it once per release, and a chart in the README built from the accumulated numbers. Seeded with ten releases so it says something the day it merges rather than after the next one.
The axis is the storage type
Every class is generic and carries
[GenericTypeArguments]fordouble,float,decimalandPreciseNumber, and the chart's top row is one construction across the four rather than four operations at oneT.That is not a stylistic choice about charts. A quantity is a
readonly record structover itsTholding a value in the SI base unit, so an operator is the storage type's arithmetic plus a struct initialiser, and a unit factory is that plus one multiplication. What a quantity costs is therefore mostly what itsTcosts — and it varies by four orders of magnitude on the same line of user code:CompareTo(interface)The last row is worth its own line: the
IPhysicalQuantity<T>route allocates on every storage type, includingdouble, where the operator allocates nothing. That is the box that buys the ability to refuse a length compared against a mass.Operands are parsed from text, never converted from a
double— adecimalor aPreciseNumberseeded through a double would be measured carrying a double's worth of digits, which is the opposite of why those types are in the list.One benchmark cannot be measured, and says so
OperatorBenchmarksoverdoubleandfloatcomes back with BenchmarkDotNet's ZeroMeasurement warning: the method is indistinguishable from an empty one. The operands do not change between iterations, so a float multiply is loop-invariant and the JIT hoists it clean out.Nothing fixes that without measuring the fix instead of the operator — an operand array adds a load, a mutated field adds a store, and either would swamp the single instruction being asked about. So the benchmark is left measuring the real thing, those rows are documented as "below what the harness resolves" rather than as numbers, and the chart draws a unit conversion in their place. All eight headline panels are comfortably above the floor.
Making times comparable across CI hosts
BaselineBenchmarks.ReferenceWorkis a fixed integer loop touching none of this library, measured beside the benchmarks in the same job; the chart divides every time by it. Separate CI runs land on different hosts and that difference is larger than most releases are. Its body must never change — editing it rescales the whole history. Allocation needs none of this, being the same on any machine, which is why the chart reads allocation as exact and time as indicative.A backfill therefore measures every version in one job.
What the ten releases show
All ten answered all twenty benchmarks; nothing had to be skipped.
3.3.1 → 4.0.0 is the record struct, and it cuts both ways. Building a quantity went from 27.4 ns and 48 B to 3.4 ns and nothing; comparing two through
IPhysicalQuantity<T>went from 4.8 ns and nothing to 14.2 ns and 24 B, because what had been a reference conversion on a class became a box on a struct.Two steps go the slower way, and both are deliberate and already documented:
StorageMath.Sqrtstopped taking adoubleround trip and started refining with Newton steps in the storage type's own arithmetic — which is what buys adecimallength its 28 significant digits rather than a double's 15.TypeInitializationException. The cost is a null check per read.Neither is a regression; the chart is where the price of each is now visible.
Notes
Semantics.Benchmarksreferencesktsu.PreciseNumberdirectly, since it is one of the four storage types measured. This does not breach the rule that the coreSemantics.Quantitiescarries no PreciseNumber dependency — the benchmark project is not shipped and the reference is confined to it.BenchmarkAgainstVersion, which has to reach the build through the environment, not-p:: BenchmarkDotNet generates a project of its own per run, which a property on the outer command line never reaches. Releases from here on are measured from their own tag through a worktree instead, which keeps that path off the race against the NuGet push beside it.Semantics.sln, sodotnet buildat the root compiles it.Verification
dotnet build -c Release— clean, 0 warnings.dotnet test -c Release— 1273 passed, 0 failed, 8 skipped (Windows-only path tests). The runner also reports one error that is not from this change:Semantics.Cpp.Testmulti-targetsnet10.0;net9.0as it already does onmain, and the container running this only has the .NET 10 runtime, so the net9.0 pass cannot start. CI installs both.🤖 Generated with Claude Code
https://claude.ai/code/session_017jrnV7N94UGL8fDRRE8Xt8
Generated by Claude Code