[qiuethan]

What this changes

Release the source-refresh runner's exact guard after all enabled API work completes, even when a start or final monitoring heartbeat fails. Monitoring failures still exit 2 and trigger Railway's failure alert; subsequent scheduled runs can acquire the guard without manual cleanup.

Why

The staging run at 17:00 UTC on November 11 completed its document and meeting work, then logged monitor_failed=true and guard_released=false. Later hourly runs and redeploys failed admission with HTTP 409 because the guard persisted in the database.

Lock cleanup now depends on confirmed API completion rather than monitoring delivery. Ambiguous acquisition, refresh/meeting API failures, and unsuccessful release still require the existing recovery procedure. The existing time-budget release exception is preserved. If release also fails, a later successful failure ping no longer clears an earlier monitoring-failure flag.

Zone

services/documentation-system

Changed-file rationale:

• src/source_refresh.py: separate successful API completion from heartbeat delivery when deciding whether to release the guard, and preserve monitoring-failure history.
• tests/test_source_refresh.py: prove failed start/final pings release the exact owner and permit a later run, while API/meeting/release timeouts retain ownership and alerts remain failures.
• docs/API.md: update the runner guard protocol for failed heartbeat delivery.
• docs/ARCHITECTURE.md: explain completion-based cleanup for document and meeting work.
• docs/DEPLOYMENT.md: update recovery guidance and clarify that drained alone does not establish successful completion of the optional meeting phase.

How to verify

The updated test_monitor_outage_never_replays_refresh_or_reports_success and test_guard_recovery_depends_on_api_completion failed against the previous runner because it retained the completed run's guard. They pass with this change.

Validation:

• make check: passed, including repository lint/format checks and fast suites.
• Full documentation-system suite with RUN_PG_TESTS=1 RUN_PG_MIGRATION_TESTS=1: passed against a dedicated disposable pgvector/Postgres 16 container after migrating to head.
• Source, contracts, and tests imports were pinned and verified to come from this checkout.

Checklist

• Branched off current staging and targeting staging.
• Ran the service's full test suite, including Postgres and migration opt-ins.
• Pinned Ruff lint and format checks pass.
• Followed the service contribution guide.
• Updated affected API, architecture and operator documentation.

User documentation and release notes

No member-facing command or wiki changes. Operator release note: monitoring delivery failures no longer strand a completed source-refresh run's guard and block later scheduled work.

Deployment notes

No migration, environment variable, key or API contract shape change. This prevents newly stranded guards after completed work; it does not delete an existing owner or expire guards after crashes. An existing stranded guard still requires the documented quiescence check and exact-owner release. Production receives the fix through the normal staging-to-main promotion.

[angelayzheng]

LGTM! Codex comments:

---

Railway MCP logs confirm this sequence in staging on November 11 (UTC):

Time	Evidence
17:00	API work completed; monitor_failed=true, guard_released=false, exit 2.
18:03–20:03	Three subsequent runs failed guard acquisition with HTTP 409.
20:06	API logs show successful deletion of the original guard.
21:05 onward	Scheduled runs completed and released their guards normally.

The code change directly addresses that sequence: successful API completion releases the exact owner’s guard despite heartbeat failure. Monitoring still produces a failing exit status, and uncertain API outcomes retain the guard.

I independently ran the source-refresh suite: 91 tests passed, including monitor failures, subsequent-run recovery, and API/meeting/release timeouts. Ruff lint and formatting checks also passed.