Skip to content

Baseline, LogReg, LLM models for manufacturing stress test project - #203

Open
vbeohar wants to merge 40 commits into
VectorInstitute:mainfrom
minaattari:main
Open

vbeohar wants to merge 40 commits into
VectorInstitute:mainfrom
minaattari:main

Conversation

@vbeohar

@vbeohar vbeohar commented Sep 22, 2026

Copy link
Copy Markdown

Summary

Clickup Ticket(s): Link(s) if applicable.

Type of Change

  • 🐛 Bug fix (non-breaking change that fixes an issue)
  • ✨ New feature (non-breaking change that adds functionality)
  • 💥 Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • 📝 Documentation update
  • 🔧 Refactoring (no functional changes)
  • ⚡ Performance improvement
  • 🧪 Test improvements
  • 🔒 Security fix

Changes Made

Testing

  • Tests pass locally (uv run pytest tests/)
  • Type checking passes (uv run mypy <src_dir>)
  • Linting passes (uv run ruff check src_dir/)
  • Manual testing performed (describe below)

Manual testing details:

Screenshots/Recordings

Related Issues

Deployment Notes

Checklist

  • Code follows the project's style guidelines
  • Self-review of code completed
  • Documentation updated (if applicable)
  • No sensitive information (API keys, credentials) exposed

vbeohar and others added 30 commits September 2, 2026 00:57
Add XGBoost predictor for manufacturing stress forecasts
feat(manufacturing): add notebook workbench and cached LLMP backtest
Expand manufacturing stress data features
…tput

Fix/manufacturing agent json output
FRED now truncates BAMLH0A0HYM2 (ICE BofA HY OAS) to a rolling 3-year
window, which silently zeroed out every fit-at-origin training example
for the logistic/XGBoost/agent predictors across both backtest windows
(build_feature_snapshot requires all features present or drops the row).
Replace it with BAA10Y (Moody's Baa corporate yield minus 10Y Treasury),
which has full history back to 1986, and rename HY_SPREAD to
CREDIT_SPREAD throughout to reflect the new source.

Also removes the GSCPI data source and its test, and finishes the
STATISTICAL_FEATURE_SERIES_IDS -> FEATURE_SERIES_IDS cleanup in the
logistic/XGBoost predictors.
Lets the tune/confirm sweep test alternative stress-label definitions
(e.g. -1.5% trailing 3-month IPMAN decline instead of the -2.0%
default) without editing code. Threshold changes redefine the target,
so results across different thresholds are not directly comparable.
Fix manufacturing-stress credit spread feature (FRED truncation)
minaattari and others added 10 commits October 5, 2026 20:25
Improve adaptive-agent structured response handling and document the paired Lite/Advanced evaluation outputs.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Include remaining notebook updates and generated adaptive-agent and all-model report packages.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…on history

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Backtesting lives in the adaptive model comparison notebook.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
… evaluation

- Walk-forward adaptive agent (run_adaptive_walk_forward.py) with an
  OutcomeLedger: the agent sees only forecast outcomes published by each
  origin, and strategy tools refuse hypothesis outcomes that do not cite a
  resolved origin. Fixes single-forecast self-confirmation of hypotheses.
- Full-history deterministic evaluation (2003-2024, 264 origins, 20 stress
  months) with block-bootstrap intervals, horizon-aware Diebold-Mariano
  tests, and Murphy decomposition (evaluation_stats.py, run_full_history.py).
- Fix logistic/XGBoost fallback forecasting exactly zero when feature-limited
  training rows contain no events; fall back to the visible label base rate.
- Vectorised fit-at-origin training rows (~100x faster, exact match).
- Untrack __pycache__, remove stray nested report path, drop dead code,
  fix adaptive report output path.
- Docs: manufacturing README, root README, implementations README, roadmap,
  AGENTS.md.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A lite-model walk-forward that asked each forecast call to also maintain
the strategy made zero strategy mutations over 76 origins. Learning now
happens in a dedicated review call, run every --review-every origins and
after any newly published stress outcome. Reviews see resolved forecasts
with the signals at each origin, so hypotheses can name signal conditions.
Forecast calls read the strategy but cannot change it; --review-every 0
restores the single-call design for comparison.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants