Skip to content

LLM Backtesting and workbench - #204

Open
minaattari wants to merge 8 commits into
VectorInstitute:mainfrom
minaattari:branch_mina
Open

minaattari wants to merge 8 commits into
VectorInstitute:mainfrom
minaattari:branch_mina

Conversation

@minaattari

Copy link
Copy Markdown

This commit adds a manufacturing-stress forecasting workbench that:

Synchronizes project dependencies with uv sync.
Loads and refreshes FRED data.
Runs historical-frequency, logistic-regression, and XGBoost predictors.
Supports optional LLM backtesting using RUN_LLMP_BACKTEST.
Caches LLM results to avoid repeated token usage.
Uses the same Brier-score evaluation across all predictors.
Supports configurable backtest stride and current forecasts.
Adds the cached LLMP backtest runner, tests, and documentation.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants