Skip to content

fix(lib): emit p-value columns in scientific notation again - #102

Open
SkyeAv wants to merge 1 commit into
mainfrom
fix-broken-scientific-notation
Open

fix(lib): emit p-value columns in scientific notation again#102
SkyeAv wants to merge 1 commit into
mainfrom
fix-broken-scientific-notation

Conversation

@SkyeAv

@SkyeAv SkyeAv commented Aug 14, 2026

Copy link
Copy Markdown
Owner

Restores the controlled scientific-notation output for p-value columns: the KGX-compliance rework (#71) switched format_numeric() to real JSON numbers for p_value / adjusted_p_value, silently retiring the {:.4e} branch for exactly the columns it existed for — edges shipped "p_value":0.0001 instead of the documented "p_value":"1.0000e-03".

What was broken

  • Dead notation branch: format_numeric() consulted numeric_slot_kind() first, which reports "float" for p_value / adjusted_p_value (declared Optional[float] on the base Association class in the pinned biolink-model), so the float-kind arm short-circuited with a cast-only path. The {:.4e} formatting could only fire for p-value columns without a numeric slot — which don't exist.
  • Result: shortest-repr numbers in whatever notation polars/serde chose (0.0001, 0.05, 6.52e-6) instead of controlled four-significant-figure scientific notation.

Fix

  • src/tablassert/lib.py: any p-value-like column (same substring match as numeric_columns) is now formatted {:.4e} before the numeric-slot short-circuit; branch ordering is the guard. Non-p-value numeric columns (effect_size, supporting_study_size) keep the model-typed behavior: a real JSON number once a future biolink-model types the slot (#1770 / #1774), controlled {:.4g} strings until then.
  • KGX validity preserved: validate_record() / validate_kgx() validate in Pydantic lax mode, which coerces the numeric string back for the float slot — verified empirically against the pinned model classes before choosing this fix.

Tests

  • New guard in tests/test_biolink.py: the validate_kgx matrix gains an edge with "p_value":"1.0000e-03" that must count as fully valid (while "not-a-number" still fails).
  • tests/test_lib.py: p-value formatting tests assert scientific-notation strings; the null-stripping test pins a zero p-value surviving strip_nulls as "0.0000e+00".
  • tests/test_e2e_smoke.py: the real-pipeline coercion smoke asserts "p_value" == "1.0000e-02" on the emitted edge.

Docs

  • docs/tutorial.md: the example edge output is accurate again — including dropping the flat supporting_study_size that never reaches the edge on biolink-model 4.4.3 (it rides the inlined supporting study).
  • CHANGELOG.md: Unreleased / Fixed entry documenting the regression and restoration; format_numeric() / numeric_slot_kind() docstrings updated.

Risks / caveats

  • Accepted tension with fix(biolink): make KGX output validate against the Biolink Model #71: item 8 of that PR moved p-values to real numbers for type fidelity; this deliberately trades strict-typed emission back for notation control. Pydantic-lax validation (our validate-kgx, the agent self-audit) stays green; a hypothetical strict-mode downstream consumer would coerce the string.

Testing

  • uv run ruff check . → clean; uv run ruff format --check . → clean; uv run pyright0 errors, 0 warnings.
  • uv run pytest -q898 passed, 35 skipped (identical count to pristine main).

The KGX-compliance rework (#71) made format_numeric() emit p_value /
adjusted_p_value as real JSON numbers because numeric_slot_kind reports
the float-typed Biolink slots -- silently retiring the {:.4e} branch for
exactly the columns it existed for, so edges shipped shortest-repr
numbers ("p_value":0.0001) instead of the controlled notation the
tutorial documents ("p_value":"1.0000e-03").

P-value-like columns are now always formatted as scientific-notation
strings ahead of the numeric-slot short-circuit. Biolink validation runs
in Pydantic's lax mode, which coerces the numeric string back, so
validate-kgx stays green (pinned by a new record in the validate_kgx
test). Non-p-value numeric columns keep the model-typed behavior: a real
JSON number once a future biolink-model types the slot, {:.4g} strings
until then.
@coderabbitai

coderabbitai Bot commented Aug 14, 2026

Copy link
Copy Markdown

Important

Review available on request

  • 🔍 Trigger review

Reviews should be triggered manually for repositories with fewer than 10 stars. Select Trigger review above or comment @coderabbitai review to review the latest changes. For a full review, comment @coderabbitai full review.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 2ac5e607-dd65-43ed-bac8-59cea394ee72


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant