Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
a70b4ad
fix(strix): bound runner occupancy by progress and a sourced job ceiling
seonghobae Sep 18, 2026
732eb17
fix(strix): renumber occupancy ADR to 0034 and tighten contracts
seonghobae Sep 18, 2026
8456fb7
fix(strix): document ADR-0034 policy and scope 900m header check
seonghobae Sep 18, 2026
a83d6d3
fix(security): allowlist https://api.github.com before urllib urlopen
seonghobae Sep 18, 2026
f659ec4
fix(ci): unblock #2265 Semgrep urllib and Strix binder packaging
seonghobae Sep 18, 2026
225260a
test(security): pin GitHub API redirect credential boundary
seonghobae Sep 18, 2026
0ae2204
fix(security): contain GitHub API redirects to admitted origin
seonghobae Sep 18, 2026
062663a
test(security): pin Strix redirect credential boundary
seonghobae Sep 18, 2026
2708a6b
fix(security): contain Strix GitHub API redirects
seonghobae Sep 18, 2026
3758b89
chore(deps): bump anyio from 4.14.0 to 4.14.2
dependabot[bot] Sep 18, 2026
4dcd25c
test(security): align Strix transport seam with dedicated opener
seonghobae Sep 19, 2026
9dff5a9
merge(security): carry #2269 urllib GitHub API opener into #2265
seonghobae Sep 19, 2026
0712822
merge(deps): carry #2278 anyio 4.14.2 into #2265
seonghobae Sep 19, 2026
3d66adc
test(strix): forbid elapsed job deadline (RED)
seonghobae Sep 19, 2026
d2134f7
fix(strix): remove elapsed job deadline from active model work
seonghobae Sep 19, 2026
bd39803
test(strix): require progress-only occupancy release
seonghobae Sep 19, 2026
d199459
docs(strix): reject elapsed margin and retain no-progress release
seonghobae Sep 19, 2026
9d4c77c
test(strix): bind idle evidence to compatibility adapter
seonghobae Sep 19, 2026
8ca8a61
merge: reconcile Strix occupancy with GitHub API authority owner
seonghobae Sep 19, 2026
d42fea7
test: retire divergent GitHub redirect contracts
seonghobae Sep 19, 2026
dcef71c
fix(strix): isolate occupancy repair scope
seonghobae Sep 19, 2026
c2b2367
fix(strix): restore complete quick-gate contract blob
seonghobae Sep 19, 2026
bde7c10
Merge protected main into Strix occupancy repair
seonghobae Sep 19, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 8 additions & 6 deletions .github/workflows/strix.yml
Original file line number Diff line number Diff line change
Expand Up @@ -369,12 +369,14 @@ jobs:
strix:
needs: [changed-scope, admit-current-head]
if: needs.changed-scope.outputs.code == 'true' && needs.admit-current-head.outputs.admitted == 'true'
# Large, actively-growing repositories (e.g. contextual-orchestrator) can
# legitimately require well over two hours to scan -- this org's own
# standing operating directive accepts that central OpenCode/Strix/Noema
# scans may take more than two hours per model (docs/product-goal-directive.md).
# Inference has no wall-clock deadline; cancellation is reserved for an
# explicit operator action or a superseded head.
# Large repositories can legitimately require more than two hours of model
# work (docs/product-goal-directive.md §8). Total elapsed job deadlines
# therefore remain absent: they would cancel active reasoning, streaming, or
# tool calls without evidence of failure. Runner release is progress-based:
# strix_timeout_compat.py restores the evidence-sourced stream-idle boundary
# after fast-mlsirm holder run 35263416380 demonstrated unbounded no-progress
# occupancy, while STRIX_PROCESS/TOTAL remain 0. Explicit operator cancellation
# and superseded-head concurrency remain separate cancellation authorities.
runs-on: ubuntu-24.04
# Least-privilege token scoped to this job (Scorecard alert #43): the scan
# exchanges an OIDC token (id-token) and publishes same-repo status evidence
Expand Down
160 changes: 160 additions & 0 deletions docs/adr/0034-review-runner-occupancy-progress-bound.md

Large diffs are not rendered by default.

109 changes: 109 additions & 0 deletions docs/doctoring/strix-unbounded-agentic-occupancy-20260918.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,109 @@
# Doctoring record: unbounded Strix agentic occupancy held fast-mlsirm under the plan ceiling (2026-09-18)

- **Date:** 2026-09-18
- **Subject:** `ContextualWisdomLab/fast-mlsirm` Actions queue showed ~179–180
`queued` runs against 1–2 `in_progress` while a single central Strix Security
Scan occupied a hosted runner with no progress or job occupancy bound.
- **Decision records:** ADR-0034 (progress / admission occupancy, not elapsed
inference); ADR-0030 (plan concurrent-job ceiling cannot be lifted by
workflow consolidation); ADR-0028 principle (numeric bounds must be sourced
from measurement, not invented — here applied to Strix occupancy, not the
coalesce-tick max-age).

## Live holder (re-fetched)

| Field | Value |
|---|---|
| Run | [35263416380](https://github.com/ContextualWisdomLab/fast-mlsirm/actions/runs/35263416380) |
| Event | `pull_request_target` (required Strix path) |
| PR | fast-mlsirm#1988 @ `3211659557118354c61a8221119c0dc314f0c90b` |
| Run created / started | 2026-09-17T19:11:44Z (~10h wall at measurement) |
| Job | `strix` id `105414661161` |
| Labels | `ubuntu-24.04` (standard hosted; not larger / self-hosted) |
| Job started | 2026-09-18T03:52:48Z |
| Step holding | `Run Strix (quick)` since 2026-09-18T04:18:54Z |
| Concurrent sibling | CI run [35287625813](https://github.com/ContextualWisdomLab/fast-mlsirm/actions/runs/35287625813) (`push`, rust still in progress / python jobs cycling) |

Timeline split (why "10h" is not 10h of model work):

1. 19:11Z → ~23:00Z: run admitted; `admit-current-head` / `Detect changed scope`
completed in seconds once they got runners.
2. ~23:00Z → 03:52Z: `strix` job waited for a runner under the org plan
concurrent-job ceiling (~60; see
[`actions-plan-concurrency-ceiling-20260903.md`](actions-plan-concurrency-ceiling-20260903.md)).
3. 03:52Z onward: setup + sidecar (~26m) then unbounded `Run Strix (quick)` with
`STRIX_PROCESS_TIMEOUT_SECONDS=0`, `STRIX_TOTAL_TIMEOUT_SECONDS=0`,
`LLM_STREAM_IDLE_TIMEOUT=0` (forced by `strix_timeout_compat.py`), and **no**
job-level `timeout-minutes` (platform default 360m).

## Why concurrency looks like ~1 on a public repo

API sample at measurement (REST `actions/runs?status=`):

| Repository | `in_progress` | `queued` |
|---|---|---|
| `fast-mlsirm` | 1–2 | ~179–180 |
| `.github` | 17 | ~538 |
| `contextual-orchestrator` | 2 | ~151 |
| `naruon` | 0 | ~76 |

That shape matches the **org-wide plan concurrent-job ceiling**, not
per-repository Strix serialization and not a larger-runner pool:

- Central `strix.yml` concurrency group is
`strix-security-scan-<repository>-<pr|push|run_id>` with
`cancel-in-progress: true` — one Strix **per PR**, not one for the whole org.
- Holder labels are `ubuntu-24.04`; org self-hosted runners `total_count=0`.
- Other repos simultaneously hold double-digit `in_progress` counts, so
fast-mlsirm is not uniquely capped at one slot by workflow design.
- Billing plan ceiling is not exposed on the Actions REST API (ADR-0030); the
user-reported 58–60/60 figure remains in the 2026-09-03 doctoring record.

## Root defect in the central Strix path

`#1546` / `#1895` correctly removed **model-path** wall clocks (including the
reverted 900s `#1889` cap). The residual hole was treating that as "no bound
of any kind": an agentic scan with stream-idle forced to `0` and no job
occupancy release can hold a shared runner until the platform 360m kill while
hundreds of unrelated jobs stay `queued`.

Directive §8 still forbids converting elapsed inference into a model-failure
verdict. ADR-0034 names the allowed repairs: **progress (idle-socket)** and
**admission/occupancy release**, with expiry classified as occupancy — not as
"the model was too slow."

## Repair landed with this record

1. **No job `timeout-minutes`** on the central `strix` job. The proposed
180-minute value was derived as measured ~2h plus a 50% margin; that margin
is not a statistical decision rule and would terminate active reasoning or
streaming solely because wall time elapsed. Required check name `strix`
remains unchanged.
2. **`LLM_STREAM_IDLE_TIMEOUT=90`** in `strix_timeout_compat.py` — progress
bound sourced from `#1884` run `34732993973` (8/10 dead-socket attempts at
exactly 90.0s). Request / warm-up elapsed deadlines remain disabled
(`LLM_TIMEOUT=0`, `STRIX_PROCESS/TOTAL_TIMEOUT_SECONDS=0`).
3. **ADR-0034** recorded on `main` so the next change cites the occupancy
decision rather than re-litigating §8.

## Duplicate "Detect changed scope" contexts (separate, non-blocking)

On the hung head, four distinct workflow runs each published a check named
`Detect changed scope` (Strix, and peer required workflows). That multiplies
admission slots under the same ceiling. Reduction without weakening gates is
already scoped by open
[ContextualWisdomLab/.github#1962](https://github.com/ContextualWisdomLab/.github/pull/1962)
(fold admit + changed-scope into one metadata job per workflow; keep required
`strix` / scan job names). Do not rename required contexts in this occupancy
PR.

## Audit trail

- REST: `repos/ContextualWisdomLab/fast-mlsirm/actions/runs/35263416380` and
`/jobs` (job `105414661161`).
- REST queue samples: `actions/runs?status=queued|in_progress` on
`fast-mlsirm`, `.github`, `contextual-orchestrator`, `naruon`, `OriginWeave`.
- OriginWeave completed-scan sample: run `35178432177` job `105140349882`
(~78.1m job / ~31.1m `Run Strix (quick)`).
- `#1884` / run `34732993973` sidecar evidence cited in ADR-0034.
- `docs/product-goal-directive.md` §8; `#1889` / `#1895`; `#1546`.
6 changes: 4 additions & 2 deletions scripts/ci/strix_quick_gate.sh
Original file line number Diff line number Diff line change
Expand Up @@ -239,11 +239,13 @@ PY
}

# Issue #2168: reject "already applied" remediation prose when apply_patch
# missed the materialized scan workspace. Uses scripts/ci/strix_evidence_binding.py.
# missed the materialized scan workspace. Uses the trusted binder beside this
# gate script (SCRIPT_DIR), never STRIX_REPO_ROOT — target/fixture workspaces
# do not ship scripts/ci/strix_evidence_binding.py (#2265 quality-ci).
sanitize_remediation_evidence_claims() {
local log_file="$1"
local report_root="$2"
local binder="$REPO_ROOT/scripts/ci/strix_evidence_binding.py"
local binder="$SCRIPT_DIR/strix_evidence_binding.py"
local report_file

if [ ! -f "$binder" ] || [ -L "$binder" ]; then
Expand Down
19 changes: 14 additions & 5 deletions scripts/ci/strix_timeout_compat.py
Original file line number Diff line number Diff line change
@@ -1,11 +1,13 @@
#!/usr/bin/env python3
"""Launch Strix 1.5.3 with ContextualWisdomLab's unbounded inference contract.
"""Launch Strix 1.5.3 with progress-bounded, elapsed-inference-unbounded policy.

Strix 1.5.3 models ``LLM_TIMEOUT`` as an integer and passes it both to request
settings and to ``asyncio.wait_for`` during model preflight. ``0`` therefore
cancels preflight immediately instead of meaning "no deadline". This trusted,
version-gated launcher keeps Strix's non-model operational timeouts intact while
removing only model-request and model-warm-up wall-clock deadlines.
version-gated launcher keeps Strix's non-model operational timeouts intact,
removes model-request and model-warm-up *elapsed* deadlines (directive §8 /
ADR-0034), and installs a sourced stream-idle occupancy bound so a dead socket
cannot hold a shared runner until GitHub's job ceiling.
"""

from __future__ import annotations
Expand All @@ -20,12 +22,19 @@

SUPPORTED_VERSION = "1.5.3"
STRIX_DISTRIBUTION = "strix-agent"
# Measured no-progress window from ContextualWisdomLab/.github#1884 head
# e85fc437, run 34732993973 / sidecar artifact 10310273053: eight of ten
# attempt-to-failure durations landed on exactly 90.0s against sockets that
# accepted a connection then delivered no response bytes (ADR-0034 §A.1).
# Expiry is occupancy / transport no-progress release, not a model-failure
# verdict for route ranking.
STREAM_IDLE_OCCUPANCY_SECONDS = "90"


def normalize_inference_timeout_environment(environment: MutableMapping[str, str]) -> None:
"""Disable Strix request and stream-idle deadlines before settings import."""
"""Clear elapsed request deadlines; keep a sourced stream-idle occupancy bound."""
environment["LLM_TIMEOUT"] = "0"
environment["LLM_STREAM_IDLE_TIMEOUT"] = "0"
environment["LLM_STREAM_IDLE_TIMEOUT"] = STREAM_IDLE_OCCUPANCY_SECONDS


class UnboundedInferenceAsyncio:
Expand Down
Loading
Loading