Skip to content

[Bug] scannum with a bound variable returns each scan once per matching candidate #263

Description

@hsingh84ai

What happens

A query that binds a variable and returns scannum reports the same scan many times over —
once per candidate mass that matched it — so the reported match count is a large multiple of
the number of scans that actually matched.

Measured on a 48-spectrum Thermo file with 14 MS1 scans:

QUERY scannum(MS1DATA) WHERE MS1MZ=X

returns 2,783 rows over those 14 scans. The distinct set of scan numbers is correct; it
is the multiplicity that is wrong.

Why

A variable query is executed once per candidate mass and the per-candidate results are
concatenated (msql_engine.py, the comment column is set to the candidate's m/z at
line ~342). The de-duplication that follows is guarded on that column:

if "comment" in collated_df:
    collated_df["truncated_comment"] = collated_df["comment"].astype(float).astype(int)
    collated_df = collated_df.drop_duplicates(subset=["scan", "truncated_comment"])

Two things combine:

  1. the subset is ["scan", "truncated_comment"], so it de-duplicates per (scan, candidate)
    rather than per scan — by design, since a scaninfo result wants one row per candidate; and
  2. the scannum result frame is rebuilt from scratch and carries only a scan column
    (msql_engine.py:606-614), so "comment" in collated_df is False and no de-duplication
    runs at all.

So for scannum specifically, every candidate that matched a scan contributes another copy
of that scan.

Suggested fix

scannum's contract is a list of scan numbers, so de-duplicating it unconditionally on
scan after collation would match what the function already does within a single execution
(list(set(...)) at line 610/612).

Note

scaninfo is not affected in the same way — its frame does carry comment.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions