Skip to content

fix(clean): no formula error finding when the formula's inputs are blank - #25

Merged
TatianaTchouakam merged 1 commit into
devfrom
feat/formula-error-empty-sources
Oct 27, 2026
Merged

TatianaTchouakam merged 1 commit into
devfrom
feat/formula-error-empty-sources

Conversation

@TatianaTchouakam

@TatianaTchouakam TatianaTchouakam commented Oct 23, 2026 •

Copy link
Copy Markdown
Collaborator

Based on #24: both add tests at the end of test_converters.py. When #24 is merged, this PR will point to dev.

When a computed field has an Excel error, the source fields of the same row are now checked. If a source field is empty, no finding is reported.

Already in the code

  • normalize_excel_formula_errors (clean/converters.py, ticket 27) turns Excel errors (#DIV/0!, #NUM!…) into empty cells and reports one source_formula_error per cell.
  • Its docstring already gives the cause: age at diagnosis when no diagnosis date was recorded, or BMI when height is blank.

Findings

  • 99.7% of source_formula_error are in 2 columns: t1d_diagnosis_age (11,001) and bmi (5,557).
  • In the 2026 trackers (counts only): every BMI error at one clinic has weight and height blank; every age-at-diagnosis error at another has no diagnosis date. Most of these findings are a missing measurement, not a formula problem.

Changes

  • FORMULA_INPUTS: bmi comes from weight and height; t1d_diagnosis_age from dob and t1d_diagnosis_date.
  • _formula_input_missing(): true when one of these inputs is in the frame and blank on the row (null, empty, or a MISSING_VALUE_MARKERS value such as -).
  • In the loop: no finding when an input is missing. The cell is still made empty, as before.
  • When an input column is not in the frame, the finding stays (the existing test passes unchanged).
  • Other columns with Excel errors (fbg_baseline_mg, age, stock) are not changed.

Tests

  • BMI error with weight and height blank, or height - → no finding; with both recorded → finding (invented data).
  • Age-at-diagnosis error without a diagnosis date → no finding; with both dates → finding.
  • just ci green, golden unchanged.

Result (local run, no upload)

Column Before After
t1d_diagnosis_age 11,001 33
bmi 5,557 16
fbg_baseline_mg 27 27
age 18 18
product_entry_date 1 1
TOTAL 16,604 95
  • 2026: 804 → 9 (−795).
  • All finding types compared by year: nothing else changed.
  • The 49 left in bmi and t1d_diagnosis_age have their inputs recorded: real formula problems, still reported.

Open point

The missing measurements are no longer reported here. If useful for the clinics, they could be reported once as a missing value in a separate PR.

@TatianaTchouakam
TatianaTchouakam changed the base branch from dev to feat/ticket-6-missing-words-empty October 23, 2026 10:26

@pmayd pmayd left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, please use a squash commit

@TatianaTchouakam TatianaTchouakam changed the title : fix(clean): no formula error finding when the formula's inputs are blank fix(clean): no formula error finding when the formula's inputs are blank Oct 27, 2026
Base automatically changed from feat/ticket-6-missing-words-empty to dev October 27, 2026 15:56
@TatianaTchouakam
TatianaTchouakam force-pushed the feat/formula-error-empty-sources branch from 90ed3ec to f7d8227 Compare October 27, 2026 15:58
@TatianaTchouakam
TatianaTchouakam merged commit 2e921ac into dev Oct 27, 2026
2 checks passed
@TatianaTchouakam
TatianaTchouakam deleted the feat/formula-error-empty-sources branch October 27, 2026 16:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants