Repository navigation
Number spent inputs in ledger order - #14
Open
carbolymer wants to merge 2 commits into
Open
carbolymer wants to merge 2 commits into
carbolymer wants to merge 2 commits into
Conversation
carbolymer
force-pushed
the
mgalazyn/fix/spent-input-index
branch
from
November 11, 2026 13:56
af6fdcf to
3131e78
Compare
carbolymer
changed the base branch from
master
to
mgalazyn/feature/gated-migrations
November 12, 2026 10:09
carbolymer
force-pushed
the
mgalazyn/fix/spent-input-index
branch
from
November 12, 2026 11:13
3131e78 to
7b93ea6
Compare
carbolymer
force-pushed
the
mgalazyn/feature/gated-migrations
branch
2 times, most recently
from
November 13, 2026 10:30
1aaa00c to
804b7de
Compare
carbolymer
force-pushed
the
mgalazyn/fix/spent-input-index
branch
from
November 13, 2026 10:30
7b93ea6 to
c6d8711
Compare
carbolymer
marked this pull request as ready for review
November 13, 2026 12:32
Since v2.10.0 (76013ce), spent_at.input_index and the spend redeemer were numbered from the last input, so the largest input got index 0. The ledger numbers spend redeemers by position in the ascending input set. Number the inputs from the first one, so the index and the redeemer match the ledger. Rows indexed before this fix keep the old values until a resync.
carbolymer
force-pushed
the
mgalazyn/fix/spent-input-index
branch
from
November 13, 2026 12:46
c6d8711 to
6fe0e6f
Compare
carbolymer
force-pushed
the
mgalazyn/feature/gated-migrations
branch
from
November 13, 2026 12:46
804b7de to
3fd8a76
Compare
Register v2.12.0/002, a data repair for the spends that Kupo recorded up to the cutoff slot, with their inputs numbered from the last one. kupo db migrate runs it when the patterns include * and the operator confirms with --confirm-complete-index that the index was synchronised from origin. The repair names v2.10.0/001, which added spent_by, as the origin of its data, so an upgrade from v2.9 or older skips it. The repair reads the spends in windows of 21600 slots, and skips the windows without any. It ranks the inputs of each spending transaction by output reference, which is the ledger order. When the stored indexes run from the last input, it numbers the inputs in ledger order and gives each input the redeemer of its pointer. It leaves alone the transactions already in ledger order, which Kupo records again after a rollback below the cutoff, and those that it cannot verify: their rows do not decode, or their indexes match neither order, for example because the index lacks some of their inputs. The final log line counts the renumbered and the unverified transactions. An interrupted prune can leave only the last inputs of a transaction, which look like a complete set. The precondition message and the documentation therefore ask for an index that was never pruned.
carbolymer
force-pushed
the
mgalazyn/fix/spent-input-index
branch
from
November 13, 2026 12:59
6fe0e6f to
b11a40e
Compare
Jimbo4350
approved these changes
Nov 13, 2026
| , toPolicy = \_ _ -> () | ||
| } | ||
| let patterns' = Set.singleton (MatchAny IncludingBootstrap) | ||
| let got = Map.lookup (txId, getPointSlotNo pt) $ consumed $ matchBlock codecs patterns' blk |
There was a problem hiding this comment.
Can you improve this property to generate varying numbers of inputs to test against?
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacking
This PR stacks on #16 (gated migrations, #15). Its base is that PR's branch,
mgalazyn/feature/gated-migrations, so GitHub shows only this PR's 2 commits: "Number spent inputs in ledger order" and "Repair spent input indexes recorded before the fix". The second is a data repair that uses the newRepairmigration kind.Problem
For each spent output, Kupo records the index of the spending input (
spent_at.input_index) and the spend redeemer (spent_at.redeemer). Since v2.10.0, both are wrong for any transaction that spends two or more inputs. The ledger numbers inputs by their position in the ascending input set, so the smallest input is index 0 and its redeemer is the one for pointer 0. Kupo numbered them from the other end, and looked the redeemer up with that same wrong index.Take a transaction with inputs a < b, where R0 is the redeemer for a and R1 the redeemer for b:
So clients get the wrong index and also the redeemer of a different input. With N inputs, only the middle input of an odd N is correct. Single-input transactions are not affected. Both chain producers are affected: the direct node connection and Ogmios.
Cause
matchBlock(src/Kupo/Data/Pattern.hs) numbers the inputs with afoldrthat carries the counter in its accumulator.foldrreaches the last element first, so the largest input gets 0 and the smallest gets N-1. This came in with 76013ce (upstream CardanoSolutions/kupo, 2025-01-02), first released in v2.10.Fix
Number the inputs left to right with
foldrWithIndexfromKupo.Prelude. The index and the redeemer lookup then both follow the ledger.Existing data
The fix alone does not change rows already stored, so the second commit adds a repair, the
Repairmigrationv2.12.0/002.On the first start of the fixed binary, Kupo records a cutoff slot and logs a warning at every start while the repair is pending. Spends recorded before the cutoff may be wrong. To repair them, stop Kupo and run:
The flag confirms that the index matches
*and was synchronised from origin. Only such an index holds all the inputs of every transaction. Any other index needs a resync.The repair works in batches of 21600-slot windows, and skips windows without spends. For each spending transaction, it ranks the stored inputs by output reference, which is the ledger order, and compares that with the stored indexes:
The final log line counts the renumbered and the incomplete transactions.
Known limitations:
*index, and some of these may be renumbered wrongly.The
CHANGELOG.mdentry under 2.12.0 "Changed" describes this.Testing
matchBlockintest/Test/Kupo/Data/PatternSpec.hs. It builds a transaction that spends two inputs listed as[hi, lo], and checks thatlogets index 0 with redeemer 0 andhigets index 1 with redeemer 1. It fails without the fix.Related
Upstream master has the same code. The open upstream PR CardanoSolutions#210 describes the same right-to-left fold, for redeemers only.