diff --git a/README.md b/README.md index b92d316..71a30e3 100644 --- a/README.md +++ b/README.md @@ -1,100 +1,128 @@ # CryptoLivePoolPipelines -CryptoLivePoolPipelines selects and publishes the monthly crypto live-pool used by QuantStrategyLab's crypto strategies. It ranks candidate symbols, decides which ones stay in the published pool and in what order, and produces the validation artifacts that downstream strategy code reads before trading on them. It does not place trades itself; it is the upstream evidence and release pipeline that other repositories depend on. +Pipeline that selects and publishes the monthly crypto live-pool used by QuantStrategyLab crypto strategies. -[Chinese README](README.zh-CN.md) +English | [简体中文](README.zh-CN.md) -> Investing involves risk. This project does not provide investment advice and is for education, research, and engineering review only. +## Status -## What this repository is +Experimental. Use at your own risk. This project is not financial advice and +does not guarantee any trading result. Its outputs are research evidence and +release artifacts, not trading instructions. It does not place trades. -CryptoLivePoolPipelines is the QuantStrategyLab crypto live-pool and release pipeline. It is the authority for monthly crypto live-pool membership, ordering, rankings, shadow candidate tracks, and release artifacts used by CryptoStrategies. +## What this is -It is an evidence-producing repository. It does not place trades and should not be treated as an execution platform. +CryptoLivePoolPipelines owns the monthly crypto live-pool: candidate ranking, +pool membership and order, shadow validation tracks, and the release artifacts +that downstream strategy code reads. It consumes market data and QuantPlatformKit +helpers and writes versioned artifacts under a documented contract. It is the +upstream evidence and release layer for crypto strategy runtimes. -## Strategy and evidence boundary +## What this is not -### Direct runtime strategies +It does not place trades and is not an execution platform. It does not decide +strategy logic; the trading logic lives in `CryptoStrategies`. Downstream systems +must not rebuild the pool from local indicators instead of the published +artifacts. -The trading logic lives in CryptoStrategies. This repository produces the live-pool and validation artifacts that the strategy package reads, and it owns the monthly selection and order of the published pool. - -### Snapshot-backed work handled here - -- core_major live pool artifacts -- monthly live-pool shadow validation -- external-data and candidate-track research outputs +## Quick start -### Downstream use +Requires Python 3.11 or newer. -CryptoStrategies and BinancePlatform should consume only release artifacts that pass the documented contract checks. Downstream systems should preserve `live_pool.json["symbols"]` order and should not rebuild the monthly pool from local indicators. +Create the project virtual environment and install the pinned runtime +requirements into it: -## What the artifacts are for +```bash +python -m venv .venv +.venv/bin/python -m pip install --upgrade pip +.venv/bin/python -m pip install -r requirements-lock.txt +``` -Live-pool artifacts are used to make strategy decisions reproducible: ranking inputs, live-pool snapshots, manifests, validation summaries, and promotion evidence. `live_pool.json` and `artifact_manifest.json` are the stable downstream execution contract; ranking files and research outputs stay upstream evidence unless the contract explicitly promotes them. They are not marketing claims. Before a downstream repository promotes a profile, review the latest artifacts across short, medium, and long windows where applicable. +Run the unit tests: -## QSL architecture role +```bash +.venv/bin/python -m unittest discover -s tests +``` -- **Layer**: `pipeline`. -- **Responsibility**: crypto live-pool and release pipeline. -- **Owns**: monthly live-pool membership, ordering, validation artifacts. -- **Consumes**: market data inputs, QuantPlatformKit helpers. -- **Must not**: place trades or bypass downstream artifact contract checks. +Build and check a monthly shadow release locally. The `make` targets invoke +`.venv/bin/python` (see [`Makefile`](Makefile)), so create `.venv` first as +shown above: -## Monthly API review +```bash +make monthly-shadow-build +make monthly-shadow-check +``` -After a successful monthly publication on `main`, an isolated review job reads that -run's published `ai_review_input.md` and `strategy_metrics.json`. It makes exactly -one single-turn OpenAI request and one single-turn Anthropic request, then posts -their reviews to the existing `monthly-review` issue. Defaults: 16,000 input tokens, -2,500 output tokens, 60 seconds per request, zero retries, and a $1 UTC-month budget. -Missing `OPENAI_API_KEY` or `ANTHROPIC_API_KEY` skips the API job with a clear log and -a `review unavailable` issue comment. See the [operator runbook](docs/operator_runbook.md#monthly-review-only-api-audit) -for configuration, conservative input counting, and the issue-comment budget ledger. -Review results are human research input; the job cannot edit code, open remediation -PRs, change live settings, or place trades. -All approvals archive the verdict and close the issue with `auto-verified`. Flags -receive one deeper Dot task under existing workload grants; only confirmed problems -or severe consensus notify the owner. An independent dead-man check alerts after -three failed/skipped cycles. See the runbook for authorization and pause gaps. +## Configuration + +| Variable | Required | Default | Purpose | +|---|---|---|---| +| `PUBLISH_ENABLED` | No | `false` | Allow publishing release artifacts. | +| `PUBLISH_MODE` | No | `core_major` | Publish mode for release objects. | +| `GCS_BUCKET` | No | none | Bucket for published release objects. | +| `CLOUD_PROJECT_ID` | No | none | Google Cloud project for storage and Firestore. | +| `FIRESTORE_COLLECTION` | No | `strategy` | Collection that receives the current-pointer document. | +| `FIRESTORE_DOCUMENT` | No | `CRYPTO_LIVE_POOL_ROTATION_LIVE_POOL` | Document that receives the current-pointer payload. | +| `UPLOAD_CURRENT_POINTER` | No | `true` | Update the current-pointer pointer after publish. | +| `TELEGRAM_BOT_TOKEN` | No | none | Bot token for monthly notifications. | +| `GLOBAL_TELEGRAM_CHAT_ID` | No | none | Chat that receives monthly notifications. | +| `NOTIFY_LANG` | No | `en` | Notification language. | +| `REVIEW_DEADMAN_THRESHOLD` | No | `3` | Failed or skipped cycles before the dead-man alert. | +| `QSL_DISPLAY_NAMES_JSON` | No | none | Display-name overrides for notifications. | + +Pipeline inputs and defaults live in [`config/default.yaml`](config/default.yaml). +The publish bucket and project identifiers are unset by default; provide them +through the environment variables above or your config file. + +## Monthly review automation + +After a successful monthly publish, a separate review-only job reads the +released monthly report and `strategy_metrics.json` from the same run, issues one +OpenAI and one Anthropic request, and posts the single-turn result as a comment +on the existing `monthly-review` issue. It does not trigger the retired +AIAuditBridge Codex Audit, does not use the Codex or Cursor CLI, and does not +modify code or trading configuration. + +The default per-request input cap is 16,000 tokens, the output cap is 2,500 +tokens, the timeout is 60 seconds, retries are zero, and the natural-month budget +is 1 USD. Input is counted conservatively as UTF-8 byte lengths plus fixed +message overhead, so oversized input is skipped rather than truncated. Missing +`OPENAI_API_KEY` or `ANTHROPIC_API_KEY` skips the API job and posts a +`review unavailable` comment. See the +[operator runbook](docs/operator_runbook.md#monthly-review-only-api-audit) for +the configuration and budget rules. + +Production publish targets such as the GCP project, GCS bucket, and Firestore +document must be read from GitHub variables. This repository no longer keeps a +source-local `ai_review.yml`; the new API review job lives in +`monthly_publish.yml`. It reads the repository and the published report, writes +issue comments, archives the verdict, and updates review state. It marks the +review issue `auto-verified` and closes it when both API reviews approve; when a +review flags a concern, it submits one bounded deeper advisory review if an +approved AIGateway route is configured, and closes the issue unless that review +confirms a real problem. It does not edit code, change live configuration, or +authorize promotion or trading. ## Repository layout -- `src/`: library and runtime code. -- `tests/`: unit, contract, and regression tests. -- `docs/`: runbooks, design notes, evidence, and integration contracts. -- `.github/workflows/`: CI, scheduled jobs, release, or deployment workflows. -- `scripts/`: operator scripts and local helpers. -- `config/`: runtime or pipeline configuration. - -## Quick start - -```bash -python -m pip install -r requirements.txt -python -m pytest -q +``` +src/ pipeline, ranking, validation, and publish code +scripts/ operator scripts and monthly jobs +config/ pipeline configuration +data/ local and generated output +docs/ contracts, roadmap, and operator runbook +tests/ unit and contract tests ``` -## Useful docs - -- [`docs/external_data_roadmap.md`](docs/external_data_roadmap.md) -- [`docs/external_data_validation.md`](docs/external_data_validation.md) -- [`docs/integration_contract.md`](docs/integration_contract.md) -- [`docs/operator_runbook.md`](docs/operator_runbook.md) -- [`docs/validation_status.md`](docs/validation_status.md) - -## Safety and contribution notes +## Contributing -- Keep generated data, credentials, and private account details out of Git unless the artifact is intentionally public and documented. -- Prefer reproducible commands and explicit output directories. -- Do not promote a research artifact to live use without the documented validation evidence. +See [CONTRIBUTING.md](CONTRIBUTING.md). -## Community and security +## Security -- See [CONTRIBUTING.md](CONTRIBUTING.md) for pull request scope, local verification, and documentation expectations. -- Follow [CODE_OF_CONDUCT.md](CODE_OF_CONDUCT.md) for maintainer and contributor conduct. -- Report credential, automation, broker, exchange, or cloud-resource vulnerabilities through [SECURITY.md](SECURITY.md); do not open public issues for secrets or live-execution risk. +See [SECURITY.md](SECURITY.md). Do not open a public issue for a vulnerability. +Keep credentials, private account details, and generated data out of Git. ## License -See [LICENSE](LICENSE). - -Display/notification naming configuration: [display names](docs/display-names.md). +MIT. See [LICENSE](LICENSE). diff --git a/README.zh-CN.md b/README.zh-CN.md index 7a5a280..ba69fb2 100644 --- a/README.zh-CN.md +++ b/README.zh-CN.md @@ -1,89 +1,92 @@ # CryptoLivePoolPipelines -CryptoLivePoolPipelines 负责挑选并发布 QuantStrategyLab 加密策略所用的月度 live-pool。它对候选币种排序，决定哪些币种留在已发布的池子里、顺序如何排列，并产出下游策略代码在下单前要读取的验证产物。它本身不下单，是其它仓库依赖的上游证据与发布流水线。 +选取并发布 QuantStrategyLab 加密策略所用月度 live-pool 的管线。 -[English README](README.md) +[English](README.md) | 简体中文 -> 投资有风险。本项目不构成投资建议，仅用于学习、研究和工程审阅。 +## 状态 -## 这个仓库是什么 +实验性项目，使用风险自负。本项目不构成投资建议，也不保证任何交易结果。它的输出是研究证据与发布产物，不是交易指令。它不提交交易。 -CryptoLivePoolPipelines 是 QuantStrategyLab 的加密货币 live-pool 与发布流水线。它是月度加密 live-pool 成员、顺序、ranking、shadow candidate tracks 和发布产物的权威来源，供 CryptoStrategies 使用。 +## 简介 -这是一个产出证据的仓库，不直接下单，也不应该被当作执行平台。 +CryptoLivePoolPipelines 负责月度加密 live-pool：候选排名、池内成员与顺序、shadow 验证轨道，以及下游策略代码读取的发布产物。它消费行情数据和 QuantPlatformKit 辅助能力，并在有文档的契约下写出带版本号的产物。它是加密策略运行时的上游证据与发布层。 -## 策略和证据边界 +## 不适用范围 -### 普通 runtime 策略 +它不提交交易，也不是执行平台。它不决定策略逻辑；交易逻辑位于 `CryptoStrategies`。下游系统不应依据本地指标重建池，而应使用已发布产物。 -交易逻辑在 CryptoStrategies。本仓库生成策略包读取的 live-pool 和验证产物，并负责已发布池的月度选择和顺序。 +## 快速开始 -### 本仓库处理的 Snapshot-backed 工作 +需要 Python 3.11 或更高版本。 -- core_major live pool 产物 -- 月度 live-pool shadow validation -- external-data 和 candidate-track 研究输出 +创建项目虚拟环境并安装锁定的运行时依赖： -### 下游如何使用 +```bash +python -m venv .venv +.venv/bin/python -m pip install --upgrade pip +.venv/bin/python -m pip install -r requirements-lock.txt +``` -CryptoStrategies 和 BinancePlatform 应只消费通过 contract 检查的发布产物。下游系统应保留 `live_pool.json["symbols"]` 的顺序，不应根据本地指标重建月度池。 +运行单元测试： -## 这些产物用来做什么 +```bash +.venv/bin/python -m unittest discover -s tests +``` -Live-pool artifact 的作用是让策略判断可复现：包括 ranking 输入、live-pool snapshot、manifest、validation summary 和提升证据。`live_pool.json` 和 `artifact_manifest.json` 是稳定的下游执行合约；ranking 文件和研究输出默认留在上游作为证据，除非合约明确提升它们。它们不是宣传式收益承诺。下游仓库提升 profile 前，应在适用场景下检查最新短、中、长周期产物。 +在本地构建并检查一次月度 shadow 发布。`make` 目标调用 `.venv/bin/python`（见 [`Makefile`](Makefile)），因此请先按上面的步骤创建 `.venv`： -## QSL 架构角色 +```bash +make monthly-shadow-build +make monthly-shadow-check +``` -- **层级**：`快照/证据流水线`。 -- **职责**：加密 live-pool 与发布流水线。 -- **事实源/归属**：月度 live-pool 成员、排序、验证 artifacts。 -- **消费对象**：市场数据输入、QuantPlatformKit helpers。 -- **禁止事项**：下单或绕过下游 artifact contract 检查。 +## 配置 + +| 变量 | 必填 | 默认值 | 用途 | +|---|---|---|---| +| `PUBLISH_ENABLED` | 否 | `false` | 是否允许发布产物。 | +| `PUBLISH_MODE` | 否 | `core_major` | 发布对象的发布模式。 | +| `GCS_BUCKET` | 否 | 无 | 存放发布对象的 bucket。 | +| `CLOUD_PROJECT_ID` | 否 | 无 | 用于存储和 Firestore 的 Google Cloud 项目。 | +| `FIRESTORE_COLLECTION` | 否 | `strategy` | 接收 current-pointer 文档的 collection。 | +| `FIRESTORE_DOCUMENT` | 否 | `CRYPTO_LIVE_POOL_ROTATION_LIVE_POOL` | 接收 current-pointer 载荷的 document。 | +| `UPLOAD_CURRENT_POINTER` | 否 | `true` | 发布后是否更新 current-pointer。 | +| `TELEGRAM_BOT_TOKEN` | 否 | 无 | 月度通知所用的 bot token。 | +| `GLOBAL_TELEGRAM_CHAT_ID` | 否 | 无 | 接收月度通知的会话。 | +| `NOTIFY_LANG` | 否 | `en` | 通知使用的语言。 | +| `REVIEW_DEADMAN_THRESHOLD` | 否 | `3` | 触发 dead-man 告警前允许的失败或跳过周期数。 | +| `QSL_DISPLAY_NAMES_JSON` | 否 | 无 | 通知显示名覆盖。 | + +管线输入与默认值见 [`config/default.yaml`](config/default.yaml)。发布所用的 bucket 与项目标识默认未设置，可通过上表中的环境变量或配置文件提供。 ## 月度 review 自动化 -月度发布成功后，独立的 review-only 作业读取同一次运行已发布的月报和 `strategy_metrics.json`，各调用一次 OpenAI 和 Anthropic API，单轮审查结果写入既有 `monthly-review` issue 的评论。不再触发 AIAuditBridge 的旧 Codex Audit，不使用 Codex/Cursor CLI，也不自动修改代码或交易配置。 +月度发布成功后，一个独立的 review-only 作业读取同一次运行已发布的月报和 `strategy_metrics.json`，各调用一次 OpenAI 和 Anthropic API，并把单轮结果作为评论写到既有 `monthly-review` issue 上。它不再触发 AIAuditBridge 的旧 Codex Audit，不使用 Codex 或 Cursor CLI，也不修改代码或交易配置。 -默认每次输入上限 16,000 token、输出上限 2,500 token、超时 60 秒、零重试，UTC 自然月预算 1 美元。输入按 UTF-8 字节数加固定消息开销保守计数，超限直接跳过，不截断材料。调用前在 issue 评论中预留两次请求的最大费用；预算记录不可读、预算不足或同月重复审查同一月报时不调用 API。缺少既有 `OPENAI_API_KEY` 或 `ANTHROPIC_API_KEY` 时，日志列明缺失名称，API 作业跳过，并发布 `review unavailable` 评论。配置和预算规则见 [运行手册](docs/operator_runbook.md#monthly-review-only-api-audit)。 +默认每次请求输入上限 16,000 token、输出上限 2,500 token、超时 60 秒、零重试，UTC 自然月预算 1 美元。输入按 UTF-8 字节数加固定消息开销保守计数，超限直接跳过而不截断。缺少 `OPENAI_API_KEY` 或 `ANTHROPIC_API_KEY` 时 API 作业跳过，并发布 `review unavailable` 评论。配置与预算规则见[运行手册](docs/operator_runbook.md#monthly-review-only-api-audit)。 -生产发布目标如 GCP project、GCS bucket 和 Firestore document 必须从 GitHub variable 读取。本仓库不再保留 source-local `ai_review.yml`；新的 API 审查作业位于 `monthly_publish.yml`，只允许读取仓库、下载报告和写 issue 评论。 +生产发布目标如 GCP project、GCS bucket 和 Firestore document 必须从 GitHub variable 读取。本仓库不再保留 source-local `ai_review.yml`；新的 API 审查作业位于 `monthly_publish.yml`。它读取仓库和已发布报告，写入 issue 评论、归档审查结论，并更新 review 状态。两次 API 审查均 approve 时，它将 review issue 标记为 `auto-verified` 并关闭；有审查 flag 时，若配置了已批准的 AIGateway 路由，它会提交一次有界的深入 advisory 审查，除非该审查确认存在真实问题，否则关闭 issue。它不修改代码、不改动线上配置，也不授权上线或交易。 ## 仓库结构 -- `src/`：库代码和运行时代码。 -- `tests/`：单元测试、契约测试和回归测试。 -- `docs/`：运行手册、设计说明、证据和集成契约。 -- `.github/workflows/`：CI、定时任务、发布或部署 workflow。 -- `scripts/`：运维脚本和本地辅助工具。 -- `config/`：运行或流水线配置。 - -## 快速开始 - -```bash -python -m pip install -r requirements.txt -python -m pytest -q +``` +src/ 管线、排名、验证与发布代码 +scripts/ 运维脚本与月度任务 +config/ 管线配置 +data/ 本地与生成的输出 +docs/ 契约、路线图与运维手册 +tests/ 单元与契约测试 ``` -## 延伸文档 - -- [`docs/external_data_roadmap.md`](docs/external_data_roadmap.md) -- [`docs/external_data_validation.md`](docs/external_data_validation.md) -- [`docs/integration_contract.md`](docs/integration_contract.md) -- [`docs/operator_runbook.md`](docs/operator_runbook.md) -- [`docs/validation_status.md`](docs/validation_status.md) - -## 安全和贡献说明 +## 参与贡献 -- 除非产物明确设计为公开且已有文档说明，否则不要把生成数据、凭据或私人账户信息提交到 Git。 -- 优先提供可复现命令，并显式指定输出目录。 -- 没有完整验证证据时，不要把研究产物提升到 live 使用。 +见 [CONTRIBUTING.md](CONTRIBUTING.md)。 -## 社区和安全 +## 安全 -- 贡献前请阅读 [CONTRIBUTING.md](CONTRIBUTING.md)，确认 PR 范围、本地校验和文档要求。 -- 讨论、issue 和 review 请遵守 [CODE_OF_CONDUCT.md](CODE_OF_CONDUCT.md)。 -- 涉及密钥、自动化、券商/交易所或云资源的漏洞请按 [SECURITY.md](SECURITY.md) 私密报告；不要为 secret 或实盘风险开公开 issue。 +见 [SECURITY.md](SECURITY.md)。安全漏洞不要开公开 issue，也不要让凭据、私有账户信息和生成数据进入 Git。 ## 许可证 -详见 [LICENSE](LICENSE)。 +MIT，见 [LICENSE](LICENSE)。