fix(parallelogram): support quarterly and semiannual origin grains - #1205
fix(parallelogram): support quarterly and semiannual origin grains#1205ppcvote wants to merge 2 commits into
Conversation
ParallelogramOLF raised ValueError on two of the four grains
Triangle.origin_grain can return:
grain result
Y ok
S ValueError: Invalid frequency: S
Q ValueError: unconverted data remains when parsing with format "%Y": "Q1"
M ok
Two independent causes, both in parallelogram_olf.
The leap-year flag was recovered by strptime-parsing the origin label after
it had already been turned into a string, with a format built as
`"%Y" + ("-%m" if grain == "M" else "")`. That covers the "Y" and "M" label
shapes only, so a quarterly label such as "2016Q1" fails to parse. The flag
is now carried off the PeriodIndex while it is still a PeriodIndex, which
removes the dependency on the label shape entirely.
Separately, `index.to_period("S")` is not a thing: pandas reads "S" as
seconds. Nor is "2Q" a substitute. pandas ignores the multiple when
converting to periods, so `to_period("2Q")` yields the same four buckets a
year as `to_period("Q")`, which would return 12 rows for a 6-origin
semiannual triangle. Since parallelogram_olf is broadcast against the
triangle positionally, that would misalign silently instead of raising.
`_origin_periods` therefore builds half years explicitly, labelled by the
quarter they start in, matching Triangle's own "%YQ%q" semiannual format.
This continues the generalisation started in casact#524, which fixed the monthly
grain and stopped there.
Verification:
- Row counts through the public estimator: Y 3, S 6, Q 12, M 36 for a
three-year triangle.
- Values checked against the aggregation identity used to validate casact#524,
OLF(period) == 1 / mean(1 / OLF(month)). Holds to 2.2e-16 for S and Q.
The Y case passes on unfixed code too and is kept as a control, so the
identity is not assumed.
- No behaviour change for Y and M: 72 scenarios covering both grains,
approximation_grain M and D, vertical_line both ways, policy_length 6/12/24
and windows spanning the 2016 and 2020 leap years hash identically before
and after.
- New tests fail 4 of 7 on unmodified code and pass 7 of 7 with the fix.
Full suite 1049 passed, 2 skipped.
Not part of the fix. The ruff workflow lints whole changed files, so any PR touching this file reports these two, and both were already failing on main before this branch: F401 typing.Iterable imported but never used E721 type(out.ddims) == np.ndarray `is` is used rather than `isinstance` so the comparison keeps its exact meaning: `isinstance` would also match ndarray subclasses, which `==` on the type object does not. Drop this commit if you would rather keep the cleanup separate. Full suite passes either way: 1049 passed, 2 skipped.
|
@ppcvote Welcome to the repo and thanks for raising this PR! |
|
Thanks @henrydingliu, glad to be here. Worth saying explicitly since you are the one who is likely to check the math: the numbers in this PR are validated with your own reciprocal-mean check from #524, the one you used to confirm kennethshsu' Werner and Modlin reconciliation: monthly_grain["Recip"] = 1 / monthly_grain["OLF"]
1 / monthly_grain[["EY", "Recip"]].groupby(by="EY").mean()I applied it to the semiannual and quarterly grains rather than re-deriving anything, and it holds to 2.2e-16 for both. The annual case is kept in the test suite as a control because it passes on unfixed code too, so the identity is demonstrated rather than assumed. The one part I would most want a second pair of eyes on is the semiannual bucketing. >>> idx = pd.date_range("2016-01-01", "2017-12-31", freq="MS")
>>> len(set(idx.to_period("Q"))), len(set(idx.to_period("2Q")))
(8, 8)Because Happy to scope this down to the quarterly crash alone and leave semiannual to #1050 if you would rather keep the |
pandas actually picked up the multiple. idx.to_period("Q").values[0].end_time == idx.to_period("2Q").values[0].end_timeFalsethe issue is that the anchoring doesn't quite behave how we want it to in this context. |
Summary of Changes
ParallelogramOLFraisesValueErroron two of the four grainsTriangle.origin_graincan return, so on-leveling a quarterly or semiannual premium triangle is not possible today:mainYSValueError: Invalid frequency: SQValueError: unconverted data remains when parsing with format "%Y": "Q1"MThere are two independent causes, both in
parallelogram_olf.The leap-year flag was recovered from the label after it had been stringified.
utility_functions.py:536parsed the origin string with a format built as"%Y" + ("-%m" if grain == "M" else ""), which only ever describes theYandMlabel shapes, so"2016Q1"fails to parse. The flag is now taken off thePeriodIndexwhile it is still aPeriodIndex, which drops the dependency on label shape entirely.to_period("S")does not exist, and"2Q"is not a substitute. pandas reads"S"as seconds. It also ignores the multiple when converting to periods, soto_period("2Q")returns the same four buckets a year asto_period("Q"):That matters more than it looks.
parallelogram_olfis broadcast against the triangle positionally (parallelogram.py:296takes.values[None, None]), so a semiannual result of length 12 would misalign against a 6-origin triangle silently, rather than raising. The new_origin_periodshelper therefore builds half years explicitly, labelled by the quarter they start in, which matches Triangle's own"%YQ%q"semiannual format (triangle.py:855).Related GitHub Issue(s)
Continues #524, which fixed the monthly grain. The leap-year handling was generalised as far as
Mthere and stopped, leavingQandSuncovered.Related to #1050 (Centralize supported grain values). This PR is compatible with that refactor rather than a substitute for it: if the
S/2Qmapping is centralised later,_origin_periodsis the single place this function would consume it from.Additional Context for Reviewers
Values, not just absence of a crash. Results are checked against the aggregation identity used to validate #524,
OLF(period) == 1 / mean(1 / OLF(month)). It holds to 2.2e-16 for bothSandQ. TheYcase passes on unfixed code as well and is kept in the suite as a control, so the identity is demonstrated rather than assumed.Row counts are asserted, not just values. Y 3, S 6, Q 12, M 36 for a three-year triangle. The
S == 6assertion is the one that catches a wrong semiannual bucketing, for the positional-broadcast reason above.Nothing changes for
YandM. 72 scenarios covering both grains xapproximation_grainM and D xvertical_lineboth ways xpolicy_length6/12/24 x windows spanning the 2016 and 2020 leap years, including a 2020-02-29 effective date so the daily leap branch is exercised, hash identically before and after.The tests discriminate. On unmodified code the new tests are 4 failed / 3 passed, the three passes being the
YandMcontrols; with the fix, 7 passed. Full suite 1049 passed, 2 skipped.About the second commit. The ruff workflow lints whole changed files, and
utility_functions.pyalready carriedF401(unusedtyping.Iterable) andE721onmainbefore this branch, so any PR touching this file reports them. I cleared them in a separate commit so it is easy to drop if you would rather keep that out of scope.iswas used rather thanisinstanceto keep the comparison's exact meaning.Checklist
uv run pytest) and documentation changes