Export every annotation candidate, and say "A or B" in mzTab-M

[htsugawa]

MS-DIAL keeps up to NUMBER_OF_ANNOTATION_RESULTS threshold-passing results per annotator, and alignment carries all of them from the representative peak into the spot. Every export then published only the representative. A product-ion spectrum reports structure indirectly, so a search that cannot separate two references is the common case rather than the corner case, and dropping the alternatives at the file boundary turns "the search did not choose" into "the search chose".

On the FastLC demo, of the 1919 annotated alignment spots only 335 have a single candidate: 115 have two and 1469 have three. Four fifths of the annotations were published as unambiguous when the run had not established that.

This adds the two exports that let a file say otherwise: a per-candidate audit sidecar, and the mzTab-M mechanism that already exists for exactly this.

AlignResult-*.mdcandidate.tsv

One row per spot and candidate, ranked by the same precedence that picks the representative, so rank 1 is the representative in every row of the file. 36 columns: the annotator and database, the library entry, the resolved reference fields, and the full score block. That is what a downstream record needs in order to state an annotation claim and the evidence behind it.

Off by default, requested with Annotation candidates: True in a Console parameter file, alongside the existing Detailed alignment provenance.

Conventions follow the .mdprovenance.tsv sidecar rather than the .mdalign columns, because the consumer is the same audit pipeline: an empty cell wherever the run established nothing, and round-trip precision instead of display rounding. A score that reads 0.977 in .mdalign reads 0.977049112 here; same measurement.

Three things the file is careful about:

• A spectral score is written only when a comparison happened. That condition now lives in one place, AnnotationScoreFormat.IsComputed, which the .mdalign and .mdpeak columns already went through. On the demo, 2744 of the 4972 rows are precursor-only suggestions and carry five empty score cells rather than five zeros.
• The five non-spectral similarity terms have no equivalent of the -1 sentinel. An unused term is stored as 0 and cannot be told apart from a term evaluated as 0. The file says so in a comment rather than implying a measurement it cannot support.
• The annotator identifier is an absolute library path in a Console run. It is kept verbatim because it is the join key back to .mdalign's Comment column and to the parameter file, which both already record it. database_id beside it carries the short database name (LbmDB) for anything that should not repeat a local path.

mzTab-M: SME_ID_REF_ambiguity_code, real ranks, one row per candidate

mzTab-M already has a way to report that a search did not choose: evidence rows sharing an evidence_input_id came from the same input spectrum, rank orders them, and the feature row that references them carries ambiguity code 1. The exporter wrote one evidence row per feature, hardcoded rank to "1" and the ambiguity code to "null".

On the demo the evidence section goes from 848 rows to 2228, and 714 of the 848 identified features have alternatives the search kept and the file used to drop. Feature 108 is a good example: three separate RIKEN N-VS1 records at the same nominal mass, and the third is the one a different run file independently chose, which the old file had no way to show.

SME  11  108  LbmDB:RIKEN N-VS1 ID-97 ...   158.97440  ms_run[1]:... | ms_run[2]:...  rank 1
SME  12  108  LbmDB:RIKEN N-VS1 ID-103 ...  158.97479  null                           rank 2
SME  13  108  LbmDB:RIKEN N-VS1 ID-104 ...  158.97510  ms_run[3]:...                  rank 3
SMF 108  SME_ID_REFS 11|12|13  ambiguity_code 1

Identifiers are laid out before the sections are written

The feature section is written before the evidence section but has to reference it, so a layout pass assigns the evidence identifiers first. Two things follow.

A reference cannot outlive a row that was never written. The two conditions used to be spelled out separately and did not agree: the feature side omitted the MS/MS-assigned, blank-filtered and internal-standard tests, so a feature could reference an evidence row the evidence loop had skipped. The demo does not happen to exercise the difference; sourcing both from one table removes the possibility, and a test asserts the two sets are equal for every row of the file.

The identifiers are file-unique. They used to be the alignment identifier, which is not unique across an ion-mobility run: a drift spot is written with its parent's metadata dictionary and so reported the parent's number. For the same reason evidence_input_id now comes from the spot the row describes — a drift row and its parent are different input spectra and must not be grouped as alternatives for one another.

Each row is resolved from its own candidate

Database identifier, name, formula, SMILES, theoretical m/z and the score block. Two notes:

• theoretical_mass_to_charge was read from the representative's metadata dictionary under a key only the LC and IM accessors provide, so a GC-MS file reported the number 0 for it. It now goes through the same reference lookup for every row, and an unresolvable reference is the null token rather than a mass of zero. This is visible in the updated golden file, where 0 becomes 301.03000.
• spectra_ref keeps its existing rule — the member files whose own top annotation is this library entry — applied to the row's own candidate. 79 of the 1380 lower-ranked rows have one; the rest report null because no file independently chose them. That the alternatives were scored against the same query spectrum is already stated by evidence_input_id.

A trailing separator on every evidence row, removed

The row used to be written without a terminator, the caller closing it afterwards, which appended a trailing tab to every one. The built-in mzTab-M structural check reports

SME column count 27 differs from its header count 26; ... (848 lines total)

on a current master run of this demo, and reports nothing on this branch. With more than one candidate per feature that form would also have run them together on a single physical line.

Verification

Measured against master at 6b8e3e9d3, same demo and method file:

Output	Result
7 per-file .mdpeak, 7 per-file .mdmsp	byte-identical
.mdalign, alignment .mdmsp, .mdpeakid.tsv, .qa.tsv	byte-identical
.mzTab MTD section	differs only in the mzTab-ID line, which is the run timestamp
.mzTab SML section	byte-identical, 2553 rows
.mzTab SMF section	2553 rows, and only fields 2 and 3 move — the evidence reference list on 848 rows and the ambiguity code on 714
.mzTab SME section	848 -> 2228 rows; 26 fields per row against a 26-field header, was 27

mzTab-M validation: passed on this branch, warning on master.

Alignment-light mode produces the same 4972 candidate rows and the same 2228 evidence rows, and its mzTab validates. It diverges from regular mode on 6 of 4972 candidate rows, which are the same 4 spots on which the .mdalign files of the two modes already diverge on master — a pre-existing representative-selection difference, not something this introduces.

• Build: Release, net48, 0 errors, including MsdialGuiApp.
• MsdialCoreTests: 325/325 (316 before, plus six for the candidate exporter and three for the mzTab evidence section).
• MsdialCoreTestAppTests: 14/14, including the new config key.

The new tests assert whole lines, not substrings, for the reason the provenance exporter tests do: a substring assertion cannot see a ragged field count, and this change adds another hand-maintained column list.

Deliberately not in this PR

• Applying the same per-candidate treatment to .mdalign and .mdpeak. Those are widely parsed fixed-column formats; the sidecar carries the alternatives without moving anything in them.
• The drift-spot metadata reuse in the SML and SMF sections. Rows for a drift spot are written with the parent's metadata dictionary, so they carry the parent's identifier and adduct. The evidence section no longer depends on that, but fixing it properly changes ion-mobility output across three sections and belongs in its own change.
• Gating the mzTab-M behaviour behind a flag. A Console parameter cannot reach the GUI exporter, and the previous file asserted an unambiguous identification the run had not made, so this is a correction rather than an option.

🤖 Generated with Claude Code