Repository navigation
Eventual verdict of Predefined and PredefinedWithoutParameterizedType flips between runs (now a ratchet wobble) #91
Description
TestEventualRatchet (maddi-dist, the dogfood eventual ratchet) lost two types in one gate run, and kept them in every rerun at the same commits. They are now listed as a wobble (maddi-dist 295e418), which hides any future loss of them too. This issue is for finding the cause, so they can return to the expected set.
What happened
maddi slowTest gate 41393883829 on laser1 (2026-11-02) ran maddi d694ee7da, maddi-mod 5a58178 and maddi-dist bbaca1c:
The eventual ratchet moved: survivor set and/or keystone levels.
LOST (2):
io.codelaser.maddi.cst.api.runtime.Predefined
io.codelaser.maddi.cst.api.runtime.PredefinedWithoutParameterizedType
Reruns: no loss
The ratchet was run alone with --rerun and EC_RETRACT_DEBUG=1, at the same maddi-mod and maddi-dist:
| maddi | laser1 | Mac |
|---|---|---|
| d694ee7da (the gate's) | 2× | 1× |
| a1697baa5 (before #166) | 1× | |
| af6512c45 (last green) | 2× | 1× |
Every run kept both types. All the survivor sets are byte-identical (352: the 351 baseline plus the ImportComputer.ImportDetails wobble), and no run printed an ECRETRACT line. #166, the only change since the last green in what the dogfood analyses, is ruled out.
So the verdict flips at the same revision. The loss has only been seen inside the whole slowTest battery, where other buckets run in parallel on the same machine. That points at timing or ordering, like the ImportDetails and CompilationUnitPrinterImpl wobbles, but it is not traced.
Why it matters
Predefinedis a large, central interface in cst-api. A run-to-run flip in its eventual verdict means the analysis result depends on scheduling. That is a bug in the analysis, whatever the ratchet does.- As a wobble, a genuine regression of these two types now passes the ratchet unseen.
What to do
- Reproduce: run the dogfood eventual analysis repeatedly under load (alongside other heavy buckets, or with a different parallelism) with
EC_RETRACT_DEBUG=1, untilPredefinedis lost, and capture theECRETRACTbroken-list of that run. - Find what the eventual verdict depends on that is order-dependent: the drift round in
docs/design/eventual-info-hierarchy.md, and the verification-pass residue boundary that explainsImportDetails. - Make the verdict deterministic, then move both types back from
dogfood/eventual-survivor-wobble.txttodogfood/expected-eventual-survivors.txtin maddi-dist.
Done when the two types are back in the expected set and the ratchet stays green across several full slowTest gates.
Activity
Analyzer: SEQCYCLEBREAKING runs the passes after cycle breaking activ…
bnaudts commentedon Nov 4, 2026
Tried to reproduce it by perturbing thread timing on a quiet laser1 (maddi-mod 737913b, maddi-dist 88e2ce4). The dogfood eventual ratchet ran 8 times:
PARALLEL= 8, 3, 2, 5, 1NOSUPERWAIT=1(the #82 supertype fix switched off)CBSWEEP=1at 8 and 3 threadsPredefinedandPredefinedWithoutParameterizedType.Separately, every run fails the ratchet on 4 survivors gained, none lost:
They appear with the #82 supertype fix on and off, and with
CBSWEEPon and off, so they come from another change on devel. The expected set needs updating at the next gate.Analyzer: PARSHUFFLE, a seeded submission order for parallel passes (…
Analyzer: a CBSWEEP sweep repeats in place until it changes nothing (#82
bnaudts commentedon Nov 4, 2026
Reproduced without load.
PARSHUFFLE=<seed>(97007ab, a diagnostic) submits each parallel pass's elements in a seeded random order. Dogfood eventual ratchet, laser1, ungated, 6 seeds:Predefined+PredefinedWithoutParameterizedTypeOutputElement+ 5 output types; 7 keystones (Element,Expression,Info,FieldInfo,ParameterInfo,TypeInfo,TypeInfoImpl) drop to@FinalFieldsafter the markSo the eventual verdicts depend on the order of the parallel pass, not only on load.
Mechanism (found on guava, #82): the eventual analysis keeps a ledger of the types each verdict leaned on. A type is recorded only if it had no verdict of its own at that moment. While that ledger is built inside the parallel loop, what gets recorded depends on thread timing, and the contraction then retracts different sets.
Fix candidate:
CBSWEEP(#82). It moves the type analyzers into the sequential second type pass and applies cycle breaking only in a sequential sweep. A first dogfood run with it was deterministic across seeds, but lost 44 survivors. The cause was not retraction: it ran out of iterations (120) before the final stage converged, because one sweep round per pass made every link of a verdict chain cost a whole round. 3040f72 repeats the sweep in place until it changes nothing; the dogfood run on it is going now.Element.rewire is @notmodified: the rewire cycle kept Element at @Fin…
bnaudts commentedon Nov 5, 2026
Fixed in devel: CBSWEEP is now the default (c3ba74b). The eventual assumption ledger is no longer written inside the parallel loop. In maddi-all's measurement with CBSWEEP, the dogfood keystone drops went from 11 to 1, and maddi 8771dabce (now in maddi) fixes the remaining one,
Element.