diff --git a/AGENTS.md b/AGENTS.md index 9e3c09c..eaf0853 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -20,3 +20,31 @@ - Keep development plan/status/operations and public guides consistent when paths/build/CI change. Local/private operating context belongs outside published documents. - Every issue/PR needs actual application verification screenshots: before/after for changed behavior and a confirmation screen for a new feature. Repository screenshots do not substitute. Planned work shows the existing app baseline and explicitly pending confirmation. Remove private source pixels before applying blur; captions state the actual evidence scope. - Create automatic backup commits only after satisfaction is confirmed. A direct PR request authorizes scoped branch commits/push/PR. Merge/release/external messages need explicit scope. A direct owner release request covers only the named version, platforms and verified publication/installation scope; preserve other releases and active Codex sessions. + + + +## VM build and cache storage + +CODEXWORK VM builds use `D:/Workstation/Set-ProjectEnvironment.ps1 -Project codex-switch` and the registered `PROJECT_BUILD_ROOT` / `PROJECT_CACHE_ROOT`. Read `D:/Workstation/BUILD_CACHE_POLICY.md`. Reproducible outputs belong under `W:/Builds/CODEXWORK/codex-switch`; use each tool's real output option. Do not silently create a second output on D: when W: is unavailable. Source, real Git worktrees, credentials, databases, emulator userdata, original assets and final acceptance evidence stay on D:. Keep active Unity Library/Temp and executable virtual environments local for filesystem/locking compatibility; do not move or delete active caches. W: is regenerable storage on the acknowledged HDD; only separately backed-up asset copies belong there. Large build admission requires 40 GiB free on D:; archive verified inactive checkpoints to the reliable HostSourceHistory area first, never discard unique state. + + + +## Codex shared-storage build execution + +Windows elevated Codex sandbox accounts have a separate drive/authentication context: they cannot directly use the interactive user's W: SMB mapping. Do not disable the sandbox, add network drives as native writable roots, or replace missing W: with local D: outputs. Submit registered operations to the ordinary-user build worker: + +`D:/Tools/Python311/python.exe D:/Workstation/Invoke-ProjectBuild.py --project --operation ` + +Operations: `storage-probe` (harmless write/read/delete); `npm-ci`, `npm-build`, `npm-test` (optional `--relative-path` to the package folder); `dotnet-restore`, `dotnet-build`, `dotnet-test` (`--relative-path` to the repository's csproj/sln); `reborn-windows`, `reborn-android`, `reborn-editmode`, `reborn-audit` for DragonMate-Reborn; `infinity-android` for InfinityCrystal-Unity. Only registered projects/operations are accepted; arbitrary shell commands and paths outside the project are rejected. The worker runs as the normal CodexWork user and receives build/cache routing without unrelated credentials. Build admission requires 40 GiB free on local D:. + +The client waits up to 30 seconds, then returns a receipt path while the same build continues. Read that receipt and its existing operation lease; do not start a duplicate. Detailed logs stay in D:/Workstation/private/shared-build-logs. Do not kill unrelated build processes. A routing probe is infrastructure evidence, not product compilation or Player acceptance. + +The permanent ordinary-user task is CodexWork-RegisteredSharedBuildWorker, starts at CodexWork logon, and accepts structured requests only under D:/Caches/Temp/CodexSharedBuildQueue. Preserve this active queue during temp cleanup. The only extra native sandbox writable root is this local queue, not W: or historical/private data. Normal interactive terminals may use Set-ProjectEnvironment and the project tools directly. In CodexSandbox accounts the environment loader skips user DPAPI Git credentials and uses this registered build path for shared I/O. + + +## 사용자 지시: 빌드 산출물은 W:만 사용 (2026-10-22) + +- APK/AAB/Player/배포 ZIP 및 재생성 dist/bin/obj/publish는 `W:/Builds/CODEXWORK/`에만 생성한다. D:/Deliverables 또는 D:/artifacts로 바이너리를 다시 복사하지 않는다. “최종 증거”/durableRoot를 빌드 복사 예외로 해석하지 않는다. +- `D:/Workstation/BuildOutputPolicy.py --project --output `로 실제 경로와 공유 저장소 쓰기/읽기를 검사한다. W: 실패 시 중단하며 D: 대체는 금지한다. 등록 worker의 npm/.NET 출력 검사도 우회하지 않는다. +- D: 검수 증거는 로그·manifest·해시·원본 화면/영상·장부다. 소스·DB·인증·AVD·설치된 도구·원본 에셋/복구 입력을 빌드 정리로 지우지 않는다. Unity Library/Temp와 OS TEMP는 도구 호환성상 로컬 D:를 유지한다. D: 40GiB 여유 조건은 이 로컬 중간 작업 공간 조건이며 APK 출력 위치가 아니다. +- 자세한 정본은 `D:/Workstation/BUILD_CACHE_POLICY.md`와 `build-cache-routing.json`이다. 과거 D: 산출물은 W: 사본 SHA 검증 후 로컬 실데이터만 제거하고 필요한 옛 경로는 링크로 유지한다. 실행 중 쓰기·추적된 원본·검수 증거 혼합 폴더를 일괄 삭제하지 않는다. diff --git a/Directory.Build.props b/Directory.Build.props index 413031d..a024667 100644 --- a/Directory.Build.props +++ b/Directory.Build.props @@ -6,4 +6,8 @@ 10.0.13 false + + W:\Builds\CODEXWORK\codex-switch\$(MSBuildProjectName)\bin\ + W:\Caches\CODEXWORK\projects\codex-switch\msbuild\$(MSBuildProjectName)\obj\ + diff --git a/README.md b/README.md index f1aba73..b45e574 100644 --- a/README.md +++ b/README.md @@ -1,7 +1,7 @@

Codex Switch app icon

-

Codex Switch

+

Codex Switch: OpenAI Codex account switcher and quota monitor

-**A desktop account switcher and quota dashboard for OpenAI Codex.** See your accounts together, choose the next one, and recover an interrupted handoff. Optional quota-based switching starts **Off**. +**Codex Switch (`codex-switch`) is an open-source desktop account switcher and usage quota monitor for OpenAI Codex.** See your accounts together, choose the next one, and recover an interrupted handoff. Optional quota-based switching starts **Off**. Windows · macOS15+ · Linux preview · MIT open source · **1.44.0** diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index eea586a..a06bd2c 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -2,6 +2,10 @@ The optional model gateway uses bounded in-memory request arrays:1GiB aggregate capacity with small-request headroom,16 upload/routing slots, eight large uploads and48independent response streams. Token validation and original-byte slices avoid full JSON history copies for native and custom routes. Array growth reserves overlap; buffers and upload slots release at response headers. Overload never replays inference. [Limits](MODEL_PROVIDERS.md) / [verification](evidence/gateway-load/README.md). +`GatewayUpstreamRetry` adds three bounded retries only for a streaming Responses request explicitly refused with HTTP507before a successful upstream response. Each attempt recreates a message around the same budgeted input and immutable route; retryable refusals dispose their response/content first. Backoff SSE comments detect caller disconnect and preserve one client turn, while authentication is rechecked before resubmission. After final headers the input and upload capacity release normally. Exhaustion is a terminal SSE failure; ambiguous transport/stream failures, images and compaction are never replayed. [Recovery contract](MODEL_PROVIDERS.md#automatic-http507-recovery). + +`DesktopStreamRecoveryMonitor` separately handles a persisted failed turn of any error type after the desktop is stopped. Eligibility uses terminal status rather than error prose, including failed completions with no message; actual interruptions remain excluded. It continues saved chat context through the current owner with inherited settings; it never replays the original HTTP request. `StreamRecoveryCoordinator` persists a three-attempt chain before each identified Continue, retaining ambiguity/exhaustion across restart. Exact SQLite metadata and bounded read-only rollout completion, current model/owner/queue and unchanged authenticated connected route gate submission. First-request failures need no fabricated successful-route record; gateway history checks still apply. Bounded fixed-code decisions expose why recovery ran or skipped without raw errors or chat text. [Stream recovery](MODEL_PROVIDERS.md#continue-after-a-failed-response-stream). + Release1.43.3 adds ModelProviderManager, ModelProviderStore, ProviderCatalog and ModelGateway in the existing application assembly, with WPF/shared desktop editors. Provider keys and connection recovery use the existing native protection identity in a separate encrypted file. The loopback gateway routes each selected alias using an immutable request snapshot; OpenAI authentication is confined to its OpenAI route. Official config RPCs atomically prepare or restore only owned settings. [Compatibility, history and acceptance](MODEL_PROVIDERS.md). Source1.41.0 adds `LinuxTerminalCommands` in the UI-neutral Application assembly. The Linux host dispatches `cli` before Avalonia/native display initialization. It composes the existing vault/account/policy/native executable services, serializes with GUI and terminal operations, refuses conflicting CLI clients, and delegates exact CLI/exec session restoration to the descriptor-bound official executable. It uses no shell, permission override, desktop adoption or plaintext credential path. [Terminal contract and current evidence](development/LINUX_TERMINAL.md). @@ -49,6 +53,7 @@ Source1.32.0 adds `src/application/CodexSwitch.Application.csproj` and `src/desk - Background reads stay asynchronous; login triggers an immediate read for that account. Credentials use serialized revisions to avoid overwriting newer authentication. - Windows vault protection is current-user DPAPI. Mac uses AES-GCM with a Keychain-held key. The existing CodexAccountDesk data paths and IPC identity are retained. - Desktop counts use observed root-chat state. Queue reads combine saved JSON and committed SQLite/WAL, filter desktop roots and deduplicate client/submission IDs. Observer protocol or schema failure hides unsupported counts. +- Errors counts currently stopped observed root chats: runtime systemError, or idle with latest turn failed. It excludes active/interrupted/recovered/child chats and stays unknown until full discovery. Both desktop presentations share this ledger result; no transcript scan or cumulative request error counter is used. - CPU/MEM/GPU/VRAM sample every 2 seconds, drives every 15 seconds, account auto-refresh every 60 seconds. Resource warning/critical thresholds are 70/85%, released 5 percentage points lower. - Windows GPU usage is the busiest aggregated engine; VRAM is the highest supported adapter ratio. Mac driver limitations remain visible as unknown. - Only the active account animates its water surface. Menus/tooltips share theme components and stable hover targets across updates. diff --git a/docs/MODEL_PROVIDERS.md b/docs/MODEL_PROVIDERS.md index f2df7db..6b99e74 100644 --- a/docs/MODEL_PROVIDERS.md +++ b/docs/MODEL_PROVIDERS.md @@ -34,13 +34,29 @@ For a model absent from that catalog or a route with different support, expand * Supported control values are `none`, `minimal`, `low`, `medium`, `high`, `xhigh` and `max`, subject to the installed Codex version and the model's API. Unsupported selections fail before inference rather than silently using another strength. A plain list with no exact match or explicit settings keeps provider-default behavior. No additional reasoning API or token-budget conversion is used. -Switch supports at most32 providers and128 selected models per provider. Discovery reads are bounded at4MiB; the complete Codex-facing catalog must fit the client's1MiB limit. Full-input conversation and image requests are bounded at512MiB, including chunked bodies. Request buffers share a1GiB capacity budget, with64MiB reserved for small requests and resize overlap included. At most eight bodies larger than16MiB or of unknown length upload concurrently, within16 upload/routing slots. Up to48 upstream responses can stream concurrently; those streams do not occupy upload slots. Overload reports503 with `Retry-After: 1` and never retries inference internally. Above-limit bodies report413; unsupported media types report415. Upstream limits still apply. A discovery result alone does not certify tool or conversation compatibility. +Switch supports at most32 providers and128 selected models per provider. Discovery reads are bounded at4MiB; the complete Codex-facing catalog must fit the client's1MiB limit. Full-input conversation and image requests are bounded at512MiB, including chunked bodies. Request buffers share a1GiB capacity budget, with64MiB reserved for small requests and resize overlap included. At most eight bodies larger than16MiB or of unknown length upload concurrently, within16 upload/routing slots. Up to48 upstream responses can stream concurrently; successful streams do not occupy upload slots. Local overload reports503 with `Retry-After: 1` and is never retried internally. Above-limit bodies report413; unsupported media types report415. Upstream limits still apply. A discovery result alone does not certify tool or conversation compatibility. JSON validation reads tokens without a payload-sized document metadata pool. It refuses duplicate properties, nesting above64levels, more than one million tokens, property names over1KiB or total property-name bytes over8MiB. Routing controls (`reasoning` and `text`) are each limited to64KiB. These bounds apply before upstream inference and do not remove conversation items. Unchanged OpenAI requests retain exact body bytes; custom-provider rewrites serialize only small controls and stream original history/image value slices. The buffer budget covers request arrays, not all process/runtime/HTTP allocations. Changing providers sends the replayed conversation to the selected provider. Provider-encrypted reasoning is omitted after a conversation crosses routes. An existing provider-encrypted compaction cannot be reconstructed by another provider: that request fails explicitly and requires a portable text summary. Stateful response references are refused rather than silently dropping history. New gateway sessions are configured for local compaction. -The connection uses existing file-backed OpenAI authentication. The local listener accepts authenticated requests on127.0.0.1 only, refuses browser-origin requests, and never forwards an OpenAI credential or account header to a custom provider. A custom provider receives only its saved key. Redirects are not followed and generation requests are not retried by the gateway. +The connection uses existing file-backed OpenAI authentication. The local listener accepts authenticated requests on127.0.0.1 only, refuses browser-origin requests, and never forwards an OpenAI credential or account header to a custom provider. A custom provider receives only its saved key. Redirects are not followed. The only automatic inference retry is the explicit HTTP507 recovery described below. + +### Automatic HTTP507 recovery + +For streaming `POST /responses`, an upstream HTTP507 refusal before its response begins is retried at most three times, after2,5and10seconds. The original attempt plus retries makes at most four upstream requests. All attempts retain the original selected model, provider URL/key, signed-in account, complete input bytes/history and tool-call identities. The current sign-in is revalidated before another attempt. There is no new chat, `Continue` message, model/account change, history trimming or context compression. + +While waiting, Switch sends only SSE comment keep-alives to the original caller. These are not assistant messages or model response events. Successful upstream events continue in the same request. After exhaustion or a different upstream failure, Switch emits one terminal `response.failed` event with the HTTP status and retry count; upstream error bodies are never echoed. Success alone records conversation provenance. The total request deadline remains30minutes, and request arrays remain within the existing aggregate buffer/admission limits until retry eligibility ends. + +### Continue after a failed response stream + +A later hotfix observes confirmed desktop turn failures of every error type, including HTTP507exhaustion, invalid event streams, authentication, quota and context errors. Eligibility uses the persisted failed status, without matching error words or requiring an error message. After the original turn has stopped, Switch can send one identified `Continue` through that chat's existing desktop owner, inheriting its settings. It does not replay the original Responses input after tools or partial output may already have run. Consecutive failed automatic continuation turns share at most three attempts, with3/10/30second backoff; successful completion closes that recovery chain. This is separate from the pre-response HTTP507request budget, so each new continuation has its own bounded HTTP507retries. A persistent error can still require user action after all three continuations fail. + +The observer requires a recent persisted `task_complete` failure, an observed stopped main chat with no approval/input/queued follow-up, matching current model/provider, a connected original OpenAI/Switch provider and unchanged current sign-in/route during backoff. Custom model aliases must still be registered. An earlier successful request is not required: the first request can fail before successful-route history exists. This never fabricates route history or bypasses encrypted-compaction and portable-history checks in the gateway. It skips active, completed, interrupted, paused/limited-goal and unsupported-provider chats. The bounded read-only rollout tail comes from the existing SQLite thread record; a newer `task_started` invalidates a historical failure. No original transcript is rewritten, no account or model is changed and no window is opened or activated. + +Encrypted recovery receipts persist attempt count and message/turn identities before submission. A lost/ambiguous IPC response or persistence error stops that recovery rather than resending. Exhaustion survives a Switch restart. Reauthentication, route edits, new user turns, queues and actual user interruptions prevent resubmission. Generic idle/EOF alone does not establish a failed turn. The observer applies only to chats observed in this local desktop with a connected OpenAI/Switch provider; it does not revive archived or unloaded historical chats. The local `stream-recovery-state.json` also retains up to128deduplicated decisions with fixed reason codes, hashed chat/turn references and attempt counts. It omits chat text, raw errors, account identities and provider secrets. + +Cancellation, a disconnected client, sign-in changes, gateway shutdown, transport exceptions and any failure after a successful upstream response stop recovery. Images, compaction, nonstream requests and other HTTP errors retain their single-send behavior. A manual new send has a new bounded budget. An already failed turn from before this hotfix needs one manual resume; the gateway does not scan old chats or submit work on its own. This recovery cannot guarantee that a server-side507condition clears and does not prove an upstream performed no internal work. Built-in image generation and editing use the original OpenAI sign-in through `images/generations` and `images/edits`. These requests preserve their JSON body, image references and request IDs and do not change the chat's model route. A custom text model selection does not redirect built-in images to that provider or its API key. Image availability and usage limits belong to the original OpenAI account. This is separate from a text model's ability to receive image attachments. @@ -48,7 +64,7 @@ The image-route and 64MiB corrections are source hotfixes after release1.43.3. O A subsequent source correction raises the64MiB intermediary limit to512MiB because an image-heavy full-input chat can exceed it. The [OpenAI image input guide](https://developers.openai.com/api/docs/guides/images-vision?api-mode=responses) documents up to512MB total payload for the public API; authenticated Codex and custom upstream acceptance must be verified separately. This change preserves image data and encrypted history and does not silently compact or discard them. -The large-upload admission slot and request buffers are released after upstream response headers arrive, so ongoing response streams do not occupy large-upload capacity. +The large-upload admission slot and request buffers are released when final upstream response headers arrive, so ongoing response streams do not occupy large-upload capacity. Explicit retryable507refusals retain the same budgeted input until the next attempt or terminal failure; they never allocate a second full body. Unregistered model names fail explicitly. OpenAI routes are checked against the current sign-in's catalog, while custom routes use their registered aliases. Managed provider/catalog requirements are refused before preparing a local connection. diff --git a/docs/README.ja.md b/docs/README.ja.md index 9d1ba34..90e6113 100644 --- a/docs/README.ja.md +++ b/docs/README.ja.md @@ -1,7 +1,7 @@

Codex Switch app icon

-

Codex Switch

+

Codex Switch: OpenAI Codexのアカウント切替と使用量確認

-**OpenAI Codexのアカウント切替と使用量を一画面で管理するデスクトップアプリです。** 各アカウントの残量を比較して次のアカウントを選択し、中断した切替は保存された記録から復旧できます。任意の残量基準による自動切替は初期状態で**オフ**です。 +**Codex Switch(`codex-switch`)は、OpenAI Codexのアカウント切替と使用量確認のためのオープンソースのデスクトップアプリです。** 各アカウントの残量を比較して次のアカウントを選択し、中断した切替は保存された記録から復旧できます。任意の残量基準による自動切替は初期状態で**オフ**です。 Windows · macOS15以降 · Linuxプレビュー · MITオープンソース · **1.44.0** diff --git a/docs/README.ko.md b/docs/README.ko.md index 1821780..7b9284a 100644 --- a/docs/README.ko.md +++ b/docs/README.ko.md @@ -1,7 +1,7 @@

Codex Switch app icon

-

Codex Switch

+

Codex Switch: OpenAI Codex 계정 전환과 사용량 확인

-**OpenAI Codex 계정 전환과 사용량을 한 화면에서 관리하는 데스크톱 앱입니다.** 여러 계정의 잔여량을 비교하고 다음 계정을 선택하세요. 전환이 중단되면 저장된 기록으로 복구할 수 있습니다. 선택적 사용량 기준 자동 전환은 기본 **꺼짐**입니다. +**Codex Switch(`codex-switch`)는 OpenAI Codex 계정 전환과 사용량 확인을 위한 오픈소스 데스크톱 앱입니다.** 여러 계정의 잔여량을 비교하고 다음 계정을 선택하세요. 전환이 중단되면 저장된 기록으로 복구할 수 있습니다. 선택적 사용량 기준 자동 전환은 기본 **꺼짐**입니다. Windows · macOS15+ · Linux 미리보기 · MIT 오픈소스 · **1.44.0** diff --git a/docs/README.zh-CN.md b/docs/README.zh-CN.md index 5960d45..2436aa6 100644 --- a/docs/README.zh-CN.md +++ b/docs/README.zh-CN.md @@ -1,7 +1,7 @@

Codex Switch app icon

-

Codex Switch

+

Codex Switch: OpenAI Codex账号切换与用量监控

-**在一个桌面界面中管理OpenAI Codex账号切换和使用额度。** 比较各账号的剩余额度、选择下一个账号,并通过已保存的记录恢复中断的切换。可选的额度阈值自动切换默认**关闭**。 +**Codex Switch(`codex-switch`)是一款用于OpenAI Codex账号切换与用量监控的开源桌面应用。** 比较各账号的剩余额度、选择下一个账号,并通过已保存的记录恢复中断的切换。可选的额度阈值自动切换默认**关闭**。 Windows · macOS15及以上 · Linux预览版 · MIT开源 · **1.44.0** diff --git a/docs/development/OPERATIONS.md b/docs/development/OPERATIONS.md index 55f1367..23fbe0c 100644 --- a/docs/development/OPERATIONS.md +++ b/docs/development/OPERATIONS.md @@ -1,12 +1,18 @@ # Operations +Failed-turn recovery uses `DesktopStreamRecoveryMonitor` and a separately encrypted `stream-recovery.vault`, preserving the original vault/key identities. It reads a bounded original rollout tail through exact read-only SQLite metadata and waits for a confirmed failed stopped main turn of any error type. HTTP507, quota, authentication, context, compaction and malformed-stream errors all qualify; missing error prose does not imply success. It sends one inherited-settings identified Continue via the existing desktop owner, without deep-link activation, model changes, account application or original HTTP replay. Three consecutive automatic attempts share persisted3/10/30second backoff and do not reset on app restart; success closes the chain. Ambiguous send/persistence failures stop that chat's recovery. Existing cancellation, queue, approval, route/authentication and completed/active guards stay enforced. An earlier successful-route receipt is not required, and gateway history checks remain enforced. Inspect bounded `decisions` in local `stream-recovery-state.json` for hashed references and reason codes such as `continued`, `attempt-limit`, `selected-model-unregistered` or `state-changed`; no raw error/chat/provider data is logged. [Behavior and limitations](../MODEL_PROVIDERS.md#continue-after-a-failed-response-stream). + +The Home Errors counter reports currently stopped main chats from the same authenticated desktop observation as Run. Count a main chat once when its runtime status is systemError, or idle and its latest turn failed. Active, interrupted, recovered and child chats are excluded; unavailable or incomplete discovery displays an em dash. Counts update on state patches and clear on recovery/removal. It is not a cumulative gateway request/error log. Windows and shared desktop use the same protocol count and localized warning-colored presentation. + +The post1.44.0 HTTP507hotfix retries only an explicitly refused streaming Responses request, at2/5/10seconds and at most three additional attempts. It stays within the original client request using SSE comment keep-alives, preserving selected route/auth/input and cancelling on caller disconnect, sign-in change or shutdown. Retryable refusal responses are disposed before waiting; original request buffers remain within the existing memory budget until retry eligibility ends. A final failure reports its status/count without an upstream body. Compaction/images/nonstream/transport/partial-response failures remain single-send. Historical stopped chats are not automatically resumed. [Exact behavior and limits](../MODEL_PROVIDERS.md#automatic-http507-recovery). + Source1.44.0 keeps Add account and Refresh beside one titlebar Settings gear. Model providers, automatic/credit controls, Restart Codex, language, appearance and drive settings are in that menu. Direct per-account Switch/Recover and warning actions stay on the tanks. [Home contract](HOME_SIMPLIFICATION.md) / [release and installation gates](RELEASE_1_44_0.md). The ordinary Linux desktop-session harness follows the new three-action titlebar and Settings > Start Codex/appearance using real input. Preserve its login/cancel, approved child ownership, rendered theme, original ciphertext and normal close/reopen assertions. The Settings menu ends with Start Codex when available, then Quit; the current-account import and drive entries remain above them. A stale toolbar coordinate is a test-path failure, not evidence of product success. Built-in image generation requires `images/generations` and `images/edits` in addition to Responses/model discovery. The gateway forwards these JSON requests only with the original OpenAI identity and preserves the body and safe request IDs; custom text aliases and saved custom-provider keys do not select the image route. Browser/loopback authentication, redirect refusal and no-retry behavior still apply. -Conversation and image request bodies remain bounded at512MiB, including chunked bodies. The gateway admits eight large/unknown-length uploads within16 upload/routing slots and a1GiB aggregate request-buffer budget.64MiB stays available for small requests; unknown-length growth counts both old/new arrays before resizing. Uploads time out after two minutes, and completed uploads release their buffers/slots at upstream response headers. Up to48 responses stream independently. Bounded overload responses include503/Retry-After; no internal generation retry is added. Disconnected clients cannot fault the listener's accept loop. [Load verification](../evidence/gateway-load/README.md). +Conversation and image request bodies remain bounded at512MiB, including chunked bodies. The gateway admits eight large/unknown-length uploads within16 upload/routing slots and a1GiB aggregate request-buffer budget.64MiB stays available for small requests; unknown-length growth counts both old/new arrays before resizing. Uploads time out after two minutes, and completed uploads release their buffers/slots at final upstream response headers. Only retryable507refusals retain their budgeted input. Up to48 responses stream independently. Bounded local overload responses include503/Retry-After and never trigger inference retries. Disconnected clients cannot fault the listener's accept loop. [Load verification](../evidence/gateway-load/README.md). Token validation avoids the payload-sized metadata pool used by JsonDocument. Full history/images remain original byte slices even when custom routing changes small controls. Strict duplicate/nesting/token/property/control bounds are documented in [model-provider limits](../MODEL_PROVIDERS.md). A body above512MiB reports413; unsupported media reports415. Preserve the chat and update the gateway before retrying a local size refusal; never remove its history to bypass it. Upstream limits can differ. [Earlier hotfix evidence](../evidence/image-gateway/README.md). @@ -95,6 +101,8 @@ Issue81 moves the shared desktop clock into the normal window lifetime so a fail ## Build and verify +The offline Linux desktop runner retains three recent `codex-switch-desktop-session` test images. Images referenced by any container, including stopped owner-authenticated sessions, and images younger than six hours are protected. `python scripts/retain-desktop-images.py --keep 3` applies the same policy manually. It does not remove containers, volumes, credentials, or interactive-session images. Source-bound image IDs in older receipts may describe history after an unused image has been retired. + The Linux source1.40.0 preview enables ordinary account/usage and [managed desktop integration](LINUX.md#source1400-ordinary-integration--issue74--pr75). Start Codex from this utility before working; an external/unmanaged client cannot be adopted. Existing credential/context policy is respected. Explicit checksum CLI-only mode has no desktop resumption. `scripts/run-local-linux-desktop.py` exercises the production backend plus fictional auth/offline native client in an owned container; authenticated two-account continuation and full desktop/Release acceptance remain separate. Run from the repository root with .NET10.0.4xx, currently SDK10.0.402/runtime10.0.13. `global.json` allows current patch updates. Use clean output folders for package candidates. .NET10's reviewed current Mac support baseline is macOS15+; the bundle minimum and app version match source1.40.2. [KDE6 provider contracts and native gates](KDE_SECRET_SERVICE.md) describe explicit package/checksum support; runtime providers are not installed or altered by the app. [Linux IPC](LINUX_IPC.md) records controlled launch, native peer trust, kernel/display/container requirements and the remaining lifecycle/account gates. [Local desktop-session verification](LINUX_DESKTOP_SESSION.md) exercises ordinary installed UI and actual provider/client lifecycles without host homes or credentials. diff --git a/docs/development/PLAN.md b/docs/development/PLAN.md index 2a6515d..351f9d0 100644 --- a/docs/development/PLAN.md +++ b/docs/development/PLAN.md @@ -1,5 +1,9 @@ # Current plan +Extend recovery beyond the pre-response507case to confirmed failed turns of every error type. Preserve original tools/context by continuing the saved chat once through its desktop owner, with three persisted attempts across a failure chain. Verify all-error eligibility, message-less failures, actual interruption exclusion, latest-turn/queue/identity/model/provider checks, no duplicate after IPC ambiguity, restart/exhaustion, bounded decision diagnostics, real rollout/SQLite path handling and actual Codex failed-turn followed by same-chat completion. Existing observed active/completed work is left alone. + +Bounded HTTP507recovery to streaming Responses passes231local provider/official-CLI checks: original plus at most three requests after2/5/10seconds, retained native/custom body/route identity, early success, terminal exhaustion, cancellation/disconnect, changed sign-in and shutdown. Images/compaction and ambiguous transport/partial-stream failures remain single-send. Preserve the existing input buffer budget, and verify local installation separately from source integration or a new public release. [Recovery contract](../MODEL_PROVIDERS.md#automatic-http507-recovery). + Issue151's [Home simplification](HOME_SIMPLIFICATION.md) and merged image/concurrent-load gateway corrections are published in1.44.0 after six protected checks, author review, matching-tree merge and public package/download verification. Tanks, direct switching, Add/Refresh and enabled automatic-state visibility remain. Keep owner installation and its idle-gateway/data-preservation receipt separate from public delivery; retain broader provider/controller/Linux acceptance in the existing issues. [Publication record](RELEASE_1_44_0.md). Issue149 strengthens concurrent gateway load with an aggregate request-buffer budget, separate upload/stream limits, sliced custom rewrites and token-based validation. Verify exact512MiB bodies, eight65MiB custom uploads,48open response streams, overload/disconnect recovery and retained-memory measurements. Require final-head protected/native CI and privacy review before repository integration. Local installation and actual provider/account acceptance remain separate. diff --git a/docs/development/STATUS.md b/docs/development/STATUS.md index 4085349..9cfaa5e 100644 --- a/docs/development/STATUS.md +++ b/docs/development/STATUS.md @@ -1,5 +1,14 @@ # Current status +Unreleased: the chat date button independently updates active root chat titles with the existing KST prefix and verified metadata path. It does not switch accounts or restart Codex. Errors stop the operation and report the number already verified. Native UI screenshots and real-user acceptance remain pending. +The all-error recovery follow-up passes346account/recovery contracts,234provider/official-CLI checks and99Windows package self-tests on runtime10.0.13. Failed terminal status now controls eligibility across HTTP, stream, authentication, quota, context and compaction errors, including absent error prose. First-request failures no longer need an earlier successful-route receipt; current owner/settings/authentication and gateway history checks remain enforced. Bounded hashed decision diagnostics distinguish continuation, exhaustion and guard exclusions. Source checks and Windows/shared builds are verified. A local installed package also confirmed an identified automatic continuation followed by an active error-free turn and completed tool work; this does not establish completion of the continued project or public release availability. + +Post-response stream continuation passes329account/recovery contracts and234provider/official-CLI checks. Coverage includes bounded continuation chains, persisted exhaustion, ambiguous-send prevention, identity/model/owner changes, read-only rollout completion and newer-turn invalidation. The actual CLI reports a post-response failed turn, classifies it for continuation and completes its following turn in the same chat. Actual owner rollout/SQLite/current desktop-model metadata were read without task mutation; a later completed turn prevents duplicate Continue. Installed acceptance and external server recovery remain separate checks. [Stream continuation contract](../MODEL_PROVIDERS.md#continue-after-a-failed-response-stream). + +Home adds Errors beside Run/Queue/Goals on Windows and shared desktop hosts. It counts observed main chats currently in systemError or idle with a failed latest turn, excludes children and active/interrupted/recovered work, and stays unknown until discovery is complete. Positive values use warning color. Production WPF component verification passes108checks across nine languages, both themes and unknown/zero/nonzero counts; protocol verification passes12checks including deduplication, owner filtering, recovery and patch updates. Native owner installation remains separately verified; component images are not native window acceptance. + +The post1.44.0 HTTP507recovery hotfix passes231local provider/official-CLI checks, including49isolated recovery checks. Explicit streaming Responses refusals get at most three additional attempts using original route/auth/input. The actual CLI completes the same turn after three507s and ends in failure after exhaustion without multiplying the budget. Cancellation, disconnect, sign-in change, gateway shutdown, partial-stream and excluded endpoint/error checks pass. The Windows package compiles with reviewed runtime10.0.13. Source integration, installed-process verification and public release remain separate. [Contract](../MODEL_PROVIDERS.md#automatic-http507-recovery). + Release1.44.0 is published from reviewed source `d97f1bfa94b933db9e64c2242324d8ff50ab89a6`, merged as `c58d87a8cadc3ee02930bdfff36984d549fcffd2` with an identical tree. All six protected checks passed. Windows/Mac actual app/menu evidence, Linux ordinary installed desktop71checks, nine runtime/license/provenance-verified packages and twelve anonymous public download hashes passed. The public Windows ZIP passes99self-tests. Home retains its tanks and direct switching with global actions in Settings. [Publication record](RELEASE_1_44_0.md) / [actual screenshots](../evidence/home-simplification/README.md). Owner installation remains a separate data-preservation/idle-gateway check; public delivery does not imply authenticated provider or broader Linux support acceptance. Issue149's implementation passes184local official-client/provider checks,74separate generated-byte stress checks and99Windows package self-tests. It bounds request arrays at1GiB with small-request headroom, allows eight large uploads and48response streams, and forwards custom history/image slices without a full mutable JSON copy. Exact512MiB Responses/compaction/image-edit and eight simultaneous65MiB custom uploads succeed; token validation removes payload-sized pooled metadata retention. Actual hosted Windows before/after images confirm app continuity. Protected final-head CI, installed-process state and external-provider acceptance are separate gates. [Measurements, evidence and delivery scope](../evidence/gateway-load/README.md). diff --git a/localization/packs/de.json b/localization/packs/de.json index db31912..b34c784 100644 --- a/localization/packs/de.json +++ b/localization/packs/de.json @@ -6,6 +6,8 @@ "culture": "de-DE", "direction": "ltr", "strings": { + "sessions.errors": "Fehler", + "sessions.errors_help": "Zählt lokale Hauptchats, die durch einen Systemfehler oder einen fehlgeschlagenen letzten Schritt angehalten wurden. Unteragenten, aktive Chats, Abbrüche und behobene Fehler sind ausgeschlossen.", "providers.reasoning_title": "GPT-Denkintensität in Codex", "providers.reasoning_model": "Modell", "providers.reasoning_override": "Unterstützte Denkstufen festlegen", @@ -513,7 +515,12 @@ "history.selected_range": "{0:yyyy/MM/dd} – {1:yyyy/MM/dd} (KST)", "history.credit_range": "Guthabenbereich", "history.credit_reading": "Messung {0:MM/dd HH:mm:ss} (KST)\nGuthabenbereich: {1:0.##}% über dem Kontingent", - "history.retention": "Nur erfasste Beobachtungen der letzten 30 Tage werden angezeigt." + "history.retention": "Nur erfasste Beobachtungen der letzten 30 Tage werden angezeigt.", + "titles.button": "Chat-Datum", + "titles.help": "Datum nach koreanischer Zeit bei laufenden Hauptchats hinzufügen oder aktualisieren.", + "titles.working": "Chat-Daten werden aktualisiert…", + "titles.done": "Datum für {0} Chats aktualisiert.", + "titles.failed": "Nach {0} aktualisierten Chats ist ein Fehler aufgetreten: {1}" }, "plurals": { "msg.accounts": { diff --git a/localization/packs/en.json b/localization/packs/en.json index 7428c16..b4df741 100644 --- a/localization/packs/en.json +++ b/localization/packs/en.json @@ -6,6 +6,8 @@ "culture": "en-US", "direction": "ltr", "strings": { + "sessions.errors": "Errors", + "sessions.errors_help": "Counts local main chats currently stopped by a system error or failed latest turn. Excludes subagents, active chats, cancellations and past errors after recovery.", "providers.reasoning_title": "GPT reasoning in Codex", "providers.reasoning_model": "Model", "providers.reasoning_override": "Set supported reasoning levels", @@ -513,7 +515,12 @@ "history.selected_range": "{0:yyyy/MM/dd} – {1:yyyy/MM/dd} (KST)", "history.credit_range": "Credit range", "history.credit_reading": "Observed at {0:MM/dd HH:mm:ss} (KST)\nCredit range: {1:0.##}% above quota", - "history.retention": "History includes only recorded observations from the last 30 days." + "history.retention": "History includes only recorded observations from the last 30 days.", + "titles.button": "Date chats", + "titles.help": "Add or update the KST date prefix on running main chats.", + "titles.working": "Updating chat dates…", + "titles.done": "Updated dates on {0} chats.", + "titles.failed": "Updated {0} chats before an error: {1}" }, "plurals": { "msg.accounts": { diff --git a/localization/packs/es.json b/localization/packs/es.json index 0018757..02bf6c9 100644 --- a/localization/packs/es.json +++ b/localization/packs/es.json @@ -6,6 +6,8 @@ "culture": "es-ES", "direction": "ltr", "strings": { + "sessions.errors": "Errores", + "sessions.errors_help": "Cuenta los chats principales locales detenidos por un error del sistema o el fallo del último turno. Excluye subagentes, chats activos, cancelaciones y errores ya recuperados.", "providers.reasoning_title": "Razonamiento de GPT en Codex", "providers.reasoning_model": "Modelo", "providers.reasoning_override": "Definir niveles de razonamiento compatibles", @@ -513,7 +515,12 @@ "history.selected_range": "{0:yyyy/MM/dd} – {1:yyyy/MM/dd} (KST)", "history.credit_range": "Zona de créditos", "history.credit_reading": "Observado {0:MM/dd HH:mm:ss} (KST)\nZona de créditos: {1:0.##}% sobre la cuota", - "history.retention": "Solo se muestran las observaciones registradas en los últimos 30 días." + "history.retention": "Solo se muestran las observaciones registradas en los últimos 30 días.", + "titles.button": "Fechar chats", + "titles.help": "Añade o actualiza la fecha coreana en los chats principales activos.", + "titles.working": "Actualizando fechas de chats…", + "titles.done": "Fechas actualizadas en {0} chats.", + "titles.failed": "Error tras actualizar {0} chats: {1}" }, "plurals": { "msg.accounts": { diff --git a/localization/packs/fr.json b/localization/packs/fr.json index 9e04e74..a400c76 100644 --- a/localization/packs/fr.json +++ b/localization/packs/fr.json @@ -6,6 +6,8 @@ "culture": "fr-FR", "direction": "ltr", "strings": { + "sessions.errors": "Erreurs", + "sessions.errors_help": "Nombre de chats principaux locaux arrêtés par une erreur système ou un échec du dernier tour. Exclut les sous-agents, les chats actifs, les annulations et les erreurs résolues.", "providers.reasoning_title": "Raisonnement GPT dans Codex", "providers.reasoning_model": "Modèle", "providers.reasoning_override": "Définir les niveaux de raisonnement pris en charge", @@ -513,7 +515,12 @@ "history.selected_range": "{0:yyyy/MM/dd} – {1:yyyy/MM/dd} (KST)", "history.credit_range": "Zone de crédits", "history.credit_reading": "Relevé {0:MM/dd HH:mm:ss} (KST)\nZone de crédits : {1:0.##}% au-delà du quota", - "history.retention": "Seules les observations enregistrées au cours des 30 derniers jours sont affichées." + "history.retention": "Seules les observations enregistrées au cours des 30 derniers jours sont affichées.", + "titles.button": "Dater les chats", + "titles.help": "Ajouter ou actualiser la date coréenne des conversations principales en cours.", + "titles.working": "Mise à jour des dates…", + "titles.done": "Dates mises à jour pour {0} conversations.", + "titles.failed": "Erreur après la mise à jour de {0} conversations : {1}" }, "plurals": { "msg.accounts": { diff --git a/localization/packs/ja.json b/localization/packs/ja.json index 3a20e77..ed8b898 100644 --- a/localization/packs/ja.json +++ b/localization/packs/ja.json @@ -6,6 +6,8 @@ "culture": "ja-JP", "direction": "ltr", "strings": { + "sessions.errors": "エラー", + "sessions.errors_help": "システムエラーまたは最新ターンの失敗で停止したローカルのメインチャット数です。サブエージェント、実行中のチャット、キャンセルと復旧済みの過去エラーは除きます。", "providers.reasoning_title": "Codex で GPT の推論強度を使う", "providers.reasoning_model": "モデル", "providers.reasoning_override": "対応する推論レベルを指定", @@ -513,7 +515,12 @@ "history.selected_range": "{0:yyyy/MM/dd} ~ {1:yyyy/MM/dd} (韓国時間)", "history.credit_range": "クレジット区間", "history.credit_reading": "実測 {0:MM/dd HH:mm:ss} (韓国時間)\nクレジット区間: 上限を {1:0.##}% 超過", - "history.retention": "直近30日間に実際に収集された記録のみ表示します。" + "history.retention": "直近30日間に実際に収集された記録のみ表示します。", + "titles.button": "日付更新", + "titles.help": "実行中のメインチャットの先頭に韓国時間の日付を追加または更新します。", + "titles.working": "チャットの日付を更新中…", + "titles.done": "{0}件のチャットの日付を更新しました。", + "titles.failed": "{0}件を更新した後にエラーが発生しました: {1}" }, "plurals": { "msg.accounts": { diff --git a/localization/packs/ko.json b/localization/packs/ko.json index 2607c34..53aa568 100644 --- a/localization/packs/ko.json +++ b/localization/packs/ko.json @@ -6,6 +6,8 @@ "culture": "ko-KR", "direction": "ltr", "strings": { + "sessions.errors": "오류", + "sessions.errors_help": "시스템 오류 또는 최근 턴 실패로 멈춘 로컬 주 대화 수입니다. 하위 에이전트, 실행 중인 대화, 사용자 중단과 복구된 과거 오류는 제외합니다.", "providers.reasoning_title": "Codex에서 GPT 추론 강도 사용", "providers.reasoning_model": "모델", "providers.reasoning_override": "지원하는 추론 단계 직접 설정", @@ -513,7 +515,12 @@ "history.selected_range": "{0:yyyy/MM/dd} ~ {1:yyyy/MM/dd} (한국 시간)", "history.credit_range": "크레딧 구간", "history.credit_reading": "실측 {0:MM/dd HH:mm:ss} (한국 시간)\n크레딧 구간: 한도 대비 {1:0.##}% 초과", - "history.retention": "최근 30일 동안 실제로 수집된 기록만 표시합니다." + "history.retention": "최근 30일 동안 실제로 수집된 기록만 표시합니다.", + "titles.button": "채팅 날짜", + "titles.help": "실행 중인 주 채팅 제목에 한국 시간 날짜를 추가하거나 갱신합니다.", + "titles.working": "채팅 날짜 갱신 중…", + "titles.done": "채팅 {0}개의 날짜를 갱신했습니다.", + "titles.failed": "채팅 {0}개를 갱신한 뒤 오류가 발생했습니다: {1}" }, "plurals": { "msg.accounts": { diff --git a/localization/packs/pt-BR.json b/localization/packs/pt-BR.json index 1df57cf..9f2634c 100644 --- a/localization/packs/pt-BR.json +++ b/localization/packs/pt-BR.json @@ -6,6 +6,8 @@ "culture": "pt-BR", "direction": "ltr", "strings": { + "sessions.errors": "Erros", + "sessions.errors_help": "Conta conversas principais locais paradas por erro de sistema ou falha no último turno. Exclui subagentes, conversas ativas, cancelamentos e erros já resolvidos.", "providers.reasoning_title": "Raciocínio GPT no Codex", "providers.reasoning_model": "Modelo", "providers.reasoning_override": "Definir níveis de raciocínio compatíveis", @@ -513,7 +515,12 @@ "history.selected_range": "{0:yyyy/MM/dd} – {1:yyyy/MM/dd} (KST)", "history.credit_range": "Faixa de créditos", "history.credit_reading": "Observado em {0:MM/dd HH:mm:ss} (KST)\nFaixa de créditos: {1:0.##}% acima da cota", - "history.retention": "Apenas observações registradas nos últimos 30 dias são exibidas." + "history.retention": "Apenas observações registradas nos últimos 30 dias são exibidas.", + "titles.button": "Datar chats", + "titles.help": "Adiciona ou atualiza a data coreana nos chats principais em execução.", + "titles.working": "Atualizando datas dos chats…", + "titles.done": "Datas atualizadas em {0} chats.", + "titles.failed": "Erro após atualizar {0} chats: {1}" }, "plurals": { "msg.accounts": { diff --git a/localization/packs/zh-CN.json b/localization/packs/zh-CN.json index 5539464..50345d6 100644 --- a/localization/packs/zh-CN.json +++ b/localization/packs/zh-CN.json @@ -6,6 +6,8 @@ "culture": "zh-CN", "direction": "ltr", "strings": { + "sessions.errors": "错误", + "sessions.errors_help": "因系统错误或最新轮次失败而停止的本地主对话数。不包含子代理、正在运行的对话、用户取消或已恢复的历史错误。", "providers.reasoning_title": "在 Codex 中使用 GPT 推理强度", "providers.reasoning_model": "模型", "providers.reasoning_override": "指定支持的推理级别", @@ -513,7 +515,12 @@ "history.selected_range": "{0:yyyy/MM/dd} – {1:yyyy/MM/dd}(韩国时间)", "history.credit_range": "额度用尽后的点数区间", "history.credit_reading": "实测 {0:MM/dd HH:mm:ss}(韩国时间)\n点数区间:超出额度 {1:0.##}%", - "history.retention": "仅显示最近30天实际收集的记录。" + "history.retention": "仅显示最近30天实际收集的记录。", + "titles.button": "更新日期", + "titles.help": "为运行中的主聊天添加或更新韩国时间日期。", + "titles.working": "正在更新聊天日期…", + "titles.done": "已更新 {0} 个聊天的日期。", + "titles.failed": "更新 {0} 个聊天后出错:{1}" }, "plurals": { "msg.accounts": { diff --git a/localization/packs/zh-TW.json b/localization/packs/zh-TW.json index c2dd98e..9bbd5bd 100644 --- a/localization/packs/zh-TW.json +++ b/localization/packs/zh-TW.json @@ -6,6 +6,8 @@ "culture": "zh-TW", "direction": "ltr", "strings": { + "sessions.errors": "錯誤", + "sessions.errors_help": "因系統錯誤或最新回合失敗而停止的本機主對話數。不包含子代理、執行中的對話、使用者取消或已復原的歷史錯誤。", "providers.reasoning_title": "在 Codex 中使用 GPT 推理強度", "providers.reasoning_model": "模型", "providers.reasoning_override": "指定支援的推理級別", @@ -513,7 +515,12 @@ "history.selected_range": "{0:yyyy/MM/dd} – {1:yyyy/MM/dd}(韓國時間)", "history.credit_range": "額度用盡後的點數區間", "history.credit_reading": "實測 {0:MM/dd HH:mm:ss}(韓國時間)\n點數區間:超出額度 {1:0.##}%", - "history.retention": "僅顯示最近30天實際收集的紀錄。" + "history.retention": "僅顯示最近30天實際收集的紀錄。", + "titles.button": "更新日期", + "titles.help": "為執行中的主聊天新增或更新韓國時間日期。", + "titles.working": "正在更新聊天日期…", + "titles.done": "已更新 {0} 個聊天的日期。", + "titles.failed": "更新 {0} 個聊天後發生錯誤:{1}" }, "plurals": { "msg.accounts": { diff --git a/scripts/build.ps1 b/scripts/build.ps1 index 8d1905a..e024d77 100644 --- a/scripts/build.ps1 +++ b/scripts/build.ps1 @@ -1,17 +1,28 @@ param([switch]$Verify) $ErrorActionPreference = 'Stop' $taskRoot = Split-Path -Parent $PSScriptRoot -$taskOutput = Join-Path $taskRoot 'artifacts\win-x64' -& dotnet publish (Join-Path $taskRoot 'src/sensors/CodexSwitch.Sensors.csproj') -c Release -r win-x64 --self-contained true --nologo -p:UseSharedCompilation=false -o (Join-Path $taskOutput 'sensors') +$storageArguments = @() +if ($env:COMPUTERNAME -eq 'CODEXWORK') { + . D:/Workstation/Set-ProjectEnvironment.ps1 -Project codex-switch + $taskRun = Join-Path $env:PROJECT_BUILD_ROOT ('local-windows/'+[DateTime]::UtcNow.ToString('yyyyMMdd-HHmmss-fff')+'-'+[guid]::NewGuid().ToString('N').Substring(0,8)) + $taskOutput = Join-Path $taskRun 'win-x64' + & D:/Tools/Python311/python.exe D:/Workstation/BuildOutputPolicy.py --project codex-switch --output $taskOutput + if ($LASTEXITCODE -ne 0) { throw 'W-only output admission failed' } + $storageArguments = @('-p:UseArtifactsOutput=true', ('-p:ArtifactsPath='+(Join-Path $taskRun 'managed'))) +} else { + # Public Windows CI retains its existing package interface outside this VM. + $taskOutput = Join-Path $taskRoot 'artifacts\win-x64' +} +& dotnet publish (Join-Path $taskRoot 'src/sensors/CodexSwitch.Sensors.csproj') -c Release -r win-x64 --self-contained true --nologo -p:UseSharedCompilation=false @storageArguments -o (Join-Path $taskOutput 'sensors') if ($LASTEXITCODE -ne 0) { throw 'Native CPU sensor worker publish failed' } $taskSensorHash = (Get-FileHash -LiteralPath (Join-Path $taskOutput 'sensors/CodexSwitch.Sensors.exe') -Algorithm SHA256).Hash -& dotnet publish (Join-Path $taskRoot 'src/windows/CodexSwitch.Windows.csproj') -c Release -r win-x64 --self-contained true --nologo -p:UseSharedCompilation=false "-p:CpuSensorWorkerSha256=$taskSensorHash" -o $taskOutput +& dotnet publish (Join-Path $taskRoot 'src/windows/CodexSwitch.Windows.csproj') -c Release -r win-x64 --self-contained true --nologo -p:UseSharedCompilation=false "-p:CpuSensorWorkerSha256=$taskSensorHash" @storageArguments -o $taskOutput if ($LASTEXITCODE -ne 0) { throw 'Windows publish failed' } Copy-Item -LiteralPath (Join-Path $taskRoot 'licenses/PawnIO-Modules.LGPL-2.1.txt') -Destination (Join-Path $taskOutput 'sensors/PawnIO-Modules.LGPL-2.1.txt') Copy-Item -LiteralPath (Join-Path $taskRoot 'src/sensors/modules/SOURCE.md') -Destination (Join-Path $taskOutput 'sensors/SOURCE.md') Copy-Item -LiteralPath (Join-Path $taskRoot 'src/sensors/modules/source') -Destination (Join-Path $taskOutput 'sensors/module-source') -Recurse -Force if ($Verify) { - $taskReport = Join-Path $taskRoot 'artifacts\self-tests.json' + $taskReport = if ($taskRun) { Join-Path $taskRun 'self-tests.json' } else { Join-Path $taskRoot 'artifacts\self-tests.json' } $taskProcess = Start-Process -FilePath (Join-Path $taskOutput 'CodexSwitch.exe') -ArgumentList @('--self-test','--verify-readonly',('"'+$taskReport+'"')) -WindowStyle Hidden -PassThru if (!$taskProcess.WaitForExit(60000)) { throw 'Internal verification did not finish' } $taskResult = Get-Content -LiteralPath $taskReport -Raw | ConvertFrom-Json diff --git a/scripts/linux-desktop-session/verify-session.py b/scripts/linux-desktop-session/verify-session.py index 357f014..ac86fb7 100644 --- a/scripts/linux-desktop-session/verify-session.py +++ b/scripts/linux-desktop-session/verify-session.py @@ -240,6 +240,9 @@ def login_controls(pending): check('Ordinary account addition exposes native pending-login cancellation at minimum size', not (ordinary_home / 'auth.json').exists()) capture(window, pid, 'ordinary-pending-login-500x360') run(['xdotool', 'mousemove', '--window', window, '32', '17', 'click', '1']) + # The title-update button becomes visible beneath the pointer when Cancel + # disappears. Compare the restored toolbar with the same unhovered state. + run(['xdotool', 'mousemove', '--window', window, '250', '180']) wait_for(lambda: login_controls(False), 20) check('Ordinary Cancel restores the ready toolbar without changing existing ciphertext or creating authentication', not (ordinary_home / 'auth.json').exists() and (root / 'share/CodexAccountDesk/accounts.vault').read_bytes() == before_login_vault) run(['xdotool', 'windowsize', '--sync', window, '800', '600']) diff --git a/scripts/retain-desktop-images.py b/scripts/retain-desktop-images.py new file mode 100644 index 0000000..3e11f24 --- /dev/null +++ b/scripts/retain-desktop-images.py @@ -0,0 +1,48 @@ +"""Retain three offline test images; never remove images referenced by containers.""" +import argparse +import json +import re +import subprocess +from datetime import datetime, timezone, timedelta + +def docker(*arguments): + return subprocess.run(['docker', *arguments], check=True, capture_output=True, + text=True, encoding='utf-8', timeout=120).stdout + +def retain(keep=3, minimum_hours=6): + rows = [json.loads(line) for line in docker('image', 'ls', '--no-trunc', '--format', '{{json .}}').splitlines() if line] + rows = [row for row in rows if row['Repository'] == 'codex-switch-desktop-session' + and re.fullmatch(r'[0-9a-f]{12}', row['Tag'])] + images = {} + for row in rows: + images.setdefault(row['ID'], row) + ordered = [] + for identifier, row in images.items(): + created = json.loads(docker('image', 'inspect', '--format', '{{json .Created}}', identifier)) + # Docker emits nanosecond fractions; datetime accepts microsecond precision. + moment = datetime.fromisoformat(created.replace('Z', '+00:00')) + ordered.append((moment, identifier)) + ordered.sort(reverse=True) + containers = docker('ps', '-aq').splitlines() + referenced = set() + for identifier in containers: + referenced.add(docker('inspect', '--format', '{{.Image}}', identifier).strip()) + cutoff = datetime.now(timezone.utc) - timedelta(hours=minimum_hours) + removed = [] + for created, identifier in ordered[keep:]: + if created >= cutoff or identifier in referenced: + continue + # Without --force Docker also refuses a container-reference race. + docker('image', 'rm', identifier) + removed.append(identifier) + return {'family': 'codex-switch-desktop-session', 'keep': keep, + 'minimumHours': minimum_hours, 'removedImages': removed, + 'containersAndVolumesRemoved': False} + +if __name__ == '__main__': + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument('--keep', type=int, default=3) + args = parser.parse_args() + if args.keep < 1: + parser.error('--keep must be positive') + print(json.dumps(retain(args.keep))) diff --git a/scripts/run-local-linux-desktop.py b/scripts/run-local-linux-desktop.py index ba0673c..e819201 100644 --- a/scripts/run-local-linux-desktop.py +++ b/scripts/run-local-linux-desktop.py @@ -116,3 +116,5 @@ def git(*arguments): if item['Name'] != '/' + name or item['Config']['Labels'].get('codex-switch-desktop-session') != head or item['Mounts']: raise RuntimeError('Refuse cleanup of an unexpected container') run(['docker', 'rm', '--force', identifier], capture_output=True) + run([sys.executable, str(repository / 'scripts/retain-desktop-images.py'), '--keep', '3'], + cwd=repository) diff --git a/src/application/DesktopStreamRecovery.cs b/src/application/DesktopStreamRecovery.cs new file mode 100644 index 0000000..3eae032 --- /dev/null +++ b/src/application/DesktopStreamRecovery.cs @@ -0,0 +1,110 @@ +using System; +using System.Collections.Generic; +using System.IO; +using System.Linq; +using System.Security.Cryptography; +using System.Text.Json; +using System.Threading; +using System.Threading.Tasks; + +namespace CodexAccountDesk; + +internal sealed record StreamRecoveryObservation(string Thread, string Owner, string Turn, string Status, string? Error, + string Model, string Provider, string Identity, string Route, DateTimeOffset At, bool Ready); +internal sealed class StreamRecoveryReceipt +{ + public string Thread { get; set; } = ""; + public string Failure { get; set; } = ""; + public string? SubmittedTurn { get; set; } + public string Message { get; set; } = ""; + public string Model { get; set; } = ""; + public string Provider { get; set; } = ""; + public string Identity { get; set; } = ""; + public string Route { get; set; } = ""; + public int Attempts { get; set; } + public bool Sending { get; set; } + public bool Closed { get; set; } +} +internal static class StreamRecoveryPolicy +{ + internal const int MaximumAttempts = 3; + // Eligibility comes from the persisted terminal status, never error prose. + // Even an absent message or a message mentioning cancellation can describe + // a failed request. Actual user interruptions remain a separate turn state. + internal static bool Failed(string status) => status == "failed"; + internal static bool RegisteredModel(string model, ModelProviderSettings providers) => + !model.StartsWith("cs_", StringComparison.Ordinal) || providers.Providers.Any(provider => + provider.Models.Any(item => item.Alias(provider.Id) == model)); + internal static bool Same(StreamRecoveryObservation a, StreamRecoveryObservation b) => b.Ready && + a.Thread == b.Thread && a.Owner == b.Owner && a.Turn == b.Turn && Failed(b.Status) && + a.Model == b.Model && a.Provider == b.Provider && a.Identity == b.Identity && a.Route == b.Route; +} +internal sealed record StreamRecoveryDecision(string Reason, int Attempts = 0); +internal sealed class StreamRecoveryCoordinator +{ + private readonly Dictionary _receipts; + private readonly Action _save; + private readonly Func _delay; + internal StreamRecoveryCoordinator(Dictionary receipts, Action save, Func? delay = null) + { _receipts = receipts; _save = save; _delay = delay ?? ((attempt, token) => Task.Delay(TimeSpan.FromSeconds(attempt switch { 1 => 3, 2 => 10, _ => 30 }), token)); } + internal async Task CheckAsync(StreamRecoveryObservation observed, Func> refresh, + Func> send, CancellationToken cancellation) + { + _receipts.TryGetValue(observed.Thread, out var receipt); + if (observed.Status == "completed") + { + if (receipt != null && !receipt.Closed && !receipt.Sending && receipt.Identity == observed.Identity && receipt.Route == observed.Route) + { receipt.Closed = true; _save(); } + return new("completed", receipt?.Attempts ?? 0); + } + if (receipt?.Closed == true && receipt.Failure == observed.Turn) return new("already-handled", receipt.Attempts); + if (!observed.Ready || !StreamRecoveryPolicy.Failed(observed.Status)) return new("not-failed-or-not-ready"); + if (DateTimeOffset.UtcNow - observed.At > TimeSpan.FromDays(1) || observed.At > DateTimeOffset.UtcNow.AddMinutes(1)) return new("outside-recovery-window"); + if (receipt != null && !receipt.Closed) + { + if (receipt.Sending) return new("unconfirmed-send", receipt.Attempts); + if (receipt.Attempts >= StreamRecoveryPolicy.MaximumAttempts) return new("attempt-limit", receipt.Attempts); + if (receipt.Failure == observed.Turn) return new("already-handled", receipt.Attempts); + if (receipt.SubmittedTurn != observed.Turn) return new("different-turn", receipt.Attempts); + if (receipt.Model != observed.Model || receipt.Provider != observed.Provider || + receipt.Identity != observed.Identity || receipt.Route != observed.Route) return new("settings-changed", receipt.Attempts); + } + else + { + if (_receipts.Count >= 10000 && receipt == null) return new("receipt-limit"); + receipt = new() { Thread = observed.Thread, Model = observed.Model, Provider = observed.Provider, Identity = observed.Identity, Route = observed.Route }; + } + await _delay(receipt.Attempts + 1, cancellation).ConfigureAwait(false); + var current = await refresh(cancellation).ConfigureAwait(false); + if (current == null || !StreamRecoveryPolicy.Same(observed, current)) return new("state-changed", receipt.Attempts); + cancellation.ThrowIfCancellationRequested(); + // Save before IPC. An interrupted/ambiguous send is never resent on restart. + receipt.Attempts++; receipt.Failure = observed.Turn; receipt.Message = Guid.NewGuid().ToString(); receipt.Sending = true; + _receipts[observed.Thread] = receipt; _save(); + var submitted = await send(receipt.Message, cancellation).ConfigureAwait(false); + if (!Guid.TryParse(submitted, out _)) throw new InvalidDataException("Automatic continuation returned an invalid turn identity."); + receipt.SubmittedTurn = submitted; receipt.Sending = false; _save(); + return new("continued", receipt.Attempts); + } +} +internal sealed class StreamRecoveryStore +{ + private readonly string _directory, _path; + internal Dictionary Receipts { get; } + internal StreamRecoveryStore(string directory) + { + _directory = directory; _path = Path.Combine(directory, "stream-recovery.vault"); + if (!File.Exists(_path)) { Receipts = new(); return; } + var plain = VaultProtection.Unprotect(SecureFiles.ReadPrivateBytes(_path), directory); + try { Receipts = JsonSerializer.Deserialize>(plain) ?? throw new InvalidDataException("Recovery receipt is empty."); } + finally { CryptographicOperations.ZeroMemory(plain); } + if (Receipts.Count > 10000 || Receipts.Any(pair => !Guid.TryParse(pair.Key, out _) || pair.Key != pair.Value.Thread || pair.Value.Attempts is < 0 or > 3)) + throw new InvalidDataException("Recovery receipt is invalid."); + } + internal void Save() + { + var bytes = JsonSerializer.SerializeToUtf8Bytes(Receipts); + try { SecureFiles.CreatePrivateDirectory(_directory); SecureFiles.AtomicWrite(_path, VaultProtection.Protect(bytes, _directory)); } + finally { CryptographicOperations.ZeroMemory(bytes); } + } +} diff --git a/src/application/DesktopStreamRecoveryMonitor.cs b/src/application/DesktopStreamRecoveryMonitor.cs new file mode 100644 index 0000000..a4e41e5 --- /dev/null +++ b/src/application/DesktopStreamRecoveryMonitor.cs @@ -0,0 +1,143 @@ +using System; +using System.Collections.Generic; +using System.IO; +using System.Linq; +using System.Security.Cryptography; +using System.Text; +using System.Text.Json; +using System.Threading; +using System.Threading.Tasks; + +namespace CodexAccountDesk; + +// Continue any persisted failed turn through its current desktop owner. The +// original Responses stream is never replayed after tools/output may have run. +internal sealed class DesktopStreamRecoveryMonitor : IAsyncDisposable +{ + private readonly AccountService _service; + private readonly DesktopSessionMonitor _sessions; + private readonly Func _ready; + private readonly CancellationTokenSource _stop = new(); + private Task? _run; + private readonly System.Collections.Generic.HashSet _blocked = new(StringComparer.Ordinal); + private DateTimeOffset _lastStatus; + private sealed record Diagnostic(DateTimeOffset AtUtc, string Chat, string? Turn, string Reason, int Attempts); + private readonly Queue _decisions = new(); + internal string? LastError { get; private set; } + internal DesktopStreamRecoveryMonitor(AccountService service, DesktopSessionMonitor sessions, Func ready) + { _service = service; _sessions = sessions; _ready = ready; } + internal void Start() => _run ??= Task.Run(RunAsync); + private async Task RunAsync() + { + try + { + var store = new StreamRecoveryStore(_service.Vault.DirectoryPath); + var recovery = new StreamRecoveryCoordinator(store.Receipts, store.Save); + while (!_stop.IsCancellationRequested) + { + if (_ready() && _sessions.Current.Available) + foreach (var thread in _sessions.Threads.Where(Stopped)) + { + if (_blocked.Contains(thread.Id)) continue; + try + { + var observed = await ObserveAsync(thread.Id, _stop.Token).ConfigureAwait(false); + if (observed == null) continue; + var decision = await recovery.CheckAsync(observed, token => ObserveAsync(thread.Id, token), async (message, token) => + { + // Re-read immediately before submitting. This never opens or + // activates a window, applies credentials or changes models. + var current = await ObserveAsync(thread.Id, token).ConfigureAwait(false); + if (current == null || !StreamRecoveryPolicy.Same(observed, current)) throw new InvalidOperationException("The chat changed before automatic continuation."); + var owner = _sessions.Threads.Single(item => item.Id == observed.Thread); + var result = await _sessions.SendToOwnerAsync(owner, "thread-follower-start-turn", StartParameters(observed.Thread, message), 2, token).ConfigureAwait(false); + return JsonValue.String(result.GetProperty("result").GetProperty("turn"), "id") ?? throw new InvalidDataException("Automatic continuation has no confirmed turn."); + }, _stop.Token).ConfigureAwait(false); + Record(thread.Id, observed.Turn, decision.Reason, decision.Attempts); + } + catch (OperationCanceledException) when (_stop.IsCancellationRequested) { throw; } + catch (Exception error) when (error is IOException or InvalidDataException or JsonException or InvalidOperationException or UnauthorizedAccessException or TimeoutException) + { + // A failed observation/send never triggers a retry of that IPC. + // Isolate this chat; other verified chats can still recover. + _blocked.Add(thread.Id); + LastError = "Automatic stream recovery stopped for a chat (" + error.GetType().Name + ")."; + Record(thread.Id, null, "blocked-" + error.GetType().Name); + } + } + if (DateTimeOffset.UtcNow - _lastStatus > TimeSpan.FromSeconds(10)) + { + WriteStatus("monitoring", store.Receipts.Values.Sum(item => item.Attempts), store.Receipts.Values.Count(item => item.Sending)); + _lastStatus = DateTimeOffset.UtcNow; + } + await Task.Delay(3000, _stop.Token).ConfigureAwait(false); + } + } + catch (OperationCanceledException) when (_stop.IsCancellationRequested) { } + catch (Exception error) + { + // Never resend after an ambiguous IPC or persistence failure. The + // saved Sending receipt prevents it even after an app restart. + LastError = "Automatic stream recovery stopped (" + error.GetType().Name + ")."; + // Keep the failure inspectable without writing chats, keys or raw upstream errors. + try { WriteStatus("stopped", null, null); } + catch (Exception storage) when (storage is IOException or UnauthorizedAccessException) { LastError += " Status storage unavailable."; } + } + } + private void WriteStatus(string state, int? attempts, int? unconfirmed) => SecureFiles.AtomicWriteText( + Path.Combine(_service.Vault.DirectoryPath, "stream-recovery-state.json"), JsonSerializer.Serialize(new { + schema = 1, processId = Environment.ProcessId, atUtc = DateTimeOffset.UtcNow, state, ready = _ready(), + desktopAvailable = _sessions.Current.Available, observedChats = _sessions.Threads.Length, + blockedChats = _blocked.Count, attempts, unconfirmed, error = LastError, decisions = _decisions.ToArray() })); + private void Record(string id, string? turn, string reason, int attempts = 0) + { + // Bounded fixed-code diagnostics: no chat text, raw errors, account IDs, + // provider destinations or credentials. Suppress identical polling entries. + var chat = SecureFiles.Hash(id); + var key = turn == null ? null : SecureFiles.Hash(turn); + var last = _decisions.LastOrDefault(item => item.Chat == chat); + if (last != null && last.Turn == key && last.Reason == reason && last.Attempts == attempts) return; + _decisions.Enqueue(new(DateTimeOffset.UtcNow, chat, key, reason, attempts)); + while (_decisions.Count > 128) _decisions.Dequeue(); + } + private static bool Stopped(DesktopThreadState thread) => !thread.Waiting && thread.ResumeState == "resumed" && + thread.Status is "systemError" or "idle" && thread.LastTurnStatus != "interrupted" && thread.Goal is null or "active"; + private async Task ObserveAsync(string id, CancellationToken cancellation) + { + StreamRecoveryObservation? Skip(string reason, string? turn = null) { Record(id, turn, reason); return null; } + if (!_ready() || !_sessions.Current.Available) return Skip("not-ready"); + if (_sessions.QueuedFor(id) != 0) return Skip("queued-or-queue-unknown"); + var thread = _sessions.Threads.SingleOrDefault(item => item.Id == id); + if (thread == null || !Stopped(thread)) return Skip("not-stopped"); + var outcome = await Task.Run(() => DesktopTurnOutcomes.Read(_service.CodexHome, id), cancellation).ConfigureAwait(false); + if (outcome == null) return Skip("terminal-outcome-unknown"); + if (outcome.Status is not ("failed" or "completed")) return Skip("not-failed", outcome.Id); + var result = await _sessions.SendToOwnerAsync(thread, "thread-follower-read-model-settings", new { conversationId = id }, 1, cancellation).ConfigureAwait(false); + var settings = result.GetProperty("settings"); + if (JsonValue.String(settings, "model") != outcome.Model || JsonValue.String(settings, "modelProvider") != outcome.Provider || + JsonValue.String(settings, "resumeState") != "resumed" || JsonValue.String(settings, "threadSource") != "user") return Skip("model-or-owner-settings-changed", outcome.Id); + var auth = _service.AuthPath; + if (!File.Exists(auth) || (File.GetAttributes(auth) & FileAttributes.ReparsePoint) != 0) return Skip("authentication-unavailable", outcome.Id); + var signedIn = Encoding.UTF8.GetString(SecureFiles.ReadPrivateBytes(auth)); + var identity = SecureFiles.Hash(signedIn); + // Pin the actual provider destination/key set across backoff as well as + // the chat's model-provider selection. Secrets never enter the receipt. + var providers = _service.ModelProviders.Settings; + if (!providers.Connected || !_service.ModelProviders.Running || outcome.Provider is not ("openai" or "codex_switch_router")) return Skip("route-unavailable", outcome.Id); + // A first request may fail before any successful-route receipt exists. + // Continue through the owner's unchanged current settings; do not invent + // successful provenance or bypass the gateway's portable-history checks. + if (!StreamRecoveryPolicy.RegisteredModel(outcome.Model, providers)) return Skip("selected-model-unregistered", outcome.Id); + var rechecked = await Task.Run(() => DesktopTurnOutcomes.Read(_service.CodexHome, id), cancellation).ConfigureAwait(false); + if (rechecked == null || rechecked.Id != outcome.Id || rechecked.Status != outcome.Status) return Skip("turn-changed", outcome.Id); + var route = SecureFiles.Hash(JsonSerializer.Serialize(providers.Providers)); + var current = _sessions.Threads.SingleOrDefault(item => item.Id == id); + if (current == null || current.Owner != thread.Owner || !Stopped(current) || _sessions.QueuedFor(id) != 0) return Skip("state-changed", outcome.Id); + return new(id, thread.Owner, outcome.Id, outcome.Status, outcome.Error, outcome.Model, outcome.Provider, identity, route, outcome.At, true); + } + internal static object StartParameters(string thread, string message) => new { conversationId = thread, + turnStart = new { request = new { threadId = thread, clientUserMessageId = message, + input = new[] { new { type = "text", text = "Continue", text_elements = Array.Empty() } } }, context = new { inheritThreadSettings = true } } }; + public async ValueTask DisposeAsync() + { _stop.Cancel(); if (_run != null) await _run.ConfigureAwait(false); _stop.Dispose(); } +} diff --git a/src/application/GatewayJsonRequest.cs b/src/application/GatewayJsonRequest.cs index 55f3a82..b34042f 100644 --- a/src/application/GatewayJsonRequest.cs +++ b/src/application/GatewayJsonRequest.cs @@ -40,6 +40,7 @@ internal GatewayJsonRequest(ReadOnlyMemory bytes) internal bool Present(string name) { var value = Get(name); return !value.IsEmpty && !value.Span.SequenceEqual("null"u8); } internal string? Model => String(Get("model")); + internal bool Streaming => Get("stream").Span.SequenceEqual("true"u8); internal static string? String(ReadOnlyMemory value) { if (value.IsEmpty || value.Length > 1024) return null; diff --git a/src/application/GatewayUpstreamRetry.cs b/src/application/GatewayUpstreamRetry.cs new file mode 100644 index 0000000..fc68840 --- /dev/null +++ b/src/application/GatewayUpstreamRetry.cs @@ -0,0 +1,30 @@ +using System; +using System.Net.Http; +using System.Threading; +using System.Threading.Tasks; + +namespace CodexAccountDesk; + +// Retry only an explicit refusal before any successful upstream response. +// A fresh message wraps the same owned input and immutable route on each attempt. +internal static class GatewayUpstreamRetry +{ + internal const int MaximumRetries = 3; + internal static TimeSpan Delay(int retry) => TimeSpan.FromSeconds(retry switch { 1 => 2, 2 => 5, 3 => 10, _ => throw new ArgumentOutOfRangeException(nameof(retry)) }); + + internal static async Task SendAsync(HttpClient http, Func createRequest, bool enabled, + Func beforeRetry, CancellationToken cancellation) + { + for (var attempt = 0; ; attempt++) + { + cancellation.ThrowIfCancellationRequested(); + using var request = createRequest(); + HttpResponseMessage result; + try { result = await http.SendAsync(request, HttpCompletionOption.ResponseHeadersRead, cancellation).ConfigureAwait(false); } + finally { request.Content?.Dispose(); request.Content = null; } + if (!enabled || (int)result.StatusCode != 507 || attempt == MaximumRetries) return result; + result.Dispose(); + await beforeRetry(attempt + 1, Delay(attempt + 1), cancellation).ConfigureAwait(false); + } + } +} diff --git a/src/application/ModelGateway.cs b/src/application/ModelGateway.cs index 401168f..5f9afbf 100644 --- a/src/application/ModelGateway.cs +++ b/src/application/ModelGateway.cs @@ -130,11 +130,10 @@ private async Task ServeAsync(HttpListenerContext context) var bytesIn = input.Bytes; var original = new GatewayJsonRequest(bytesIn); JsonObject? body = null; - void ReleaseInput(HttpRequestMessage outgoing) + void ReleaseInput() { - // Upload is complete once response headers arrive. Do not retain its buffers - // or admission slot during a potentially long-running response stream. - outgoing.Content?.Dispose(); outgoing.Content = null; + // Keep the budgeted input only while a pre-response 507 retry is possible. + // Successful response streams never retain request buffers/upload slots. original.Release(); body = null; bytesIn = default; input.Dispose(); if (largeBody) { _largeBodies.Release(); largeBody = false; } @@ -146,7 +145,7 @@ void ReleaseInput(HttpRequestMessage outgoing) CopyOpenAiHeaders(request, image, identity); image.Content = new ReadOnlyMemoryContent(bytesIn); image.Content.Headers.ContentType = new MediaTypeHeaderValue("application/json"); using var imageResult = await _http.SendAsync(image, HttpCompletionOption.ResponseHeadersRead, cancellation).ConfigureAwait(false); - ReleaseInput(image); + image.Content?.Dispose(); image.Content = null; ReleaseInput(); if (!imageResult.IsSuccessStatusCode) { await ErrorAsync(response, (int)imageResult.StatusCode is >= 300 and < 400 ? 502 : (int)imageResult.StatusCode, "The OpenAI image endpoint returned HTTP " + (int)imageResult.StatusCode + ". No request was retried.", cancellation).ConfigureAwait(false); return; } response.StatusCode = (int)imageResult.StatusCode; response.ContentType = imageResult.Content.Headers.ContentType?.ToString() ?? "application/json"; @@ -202,20 +201,63 @@ void ReleaseInput(HttpRequestMessage outgoing) destination = new Uri(ProviderCatalog.Endpoint(route.Provider.BaseUrl), path.EndsWith("/compact", StringComparison.Ordinal) ? "responses/compact" : "responses"); } else destination = new Uri((identity.ApiKey ? "https://api.openai.com/v1/" : "https://chatgpt.com/backend-api/codex/") + (path.EndsWith("/compact", StringComparison.Ordinal) ? "responses/compact" : "responses")); - using var upstream = new HttpRequestMessage(HttpMethod.Post, destination); - if (route.Provider == null) CopyOpenAiHeaders(request, upstream, identity); - else upstream.Headers.Authorization = new AuthenticationHeaderValue("Bearer", route.Provider.ApiKey); - upstream.Content = route.Provider != null || removeReasoning ? - new GatewayRoutedContent(bytesIn, body ?? new JsonObject(), route.Provider != null, route.Model?.GenericMetadata == true, removeReasoning) : - new ReadOnlyMemoryContent(bytesIn); - upstream.Content.Headers.ContentType = new MediaTypeHeaderValue("application/json"); - using var result = await _http.SendAsync(upstream, HttpCompletionOption.ResponseHeadersRead, cancellation).ConfigureAwait(false); - ReleaseInput(upstream); + var retries = 0; + HttpRequestMessage CreateUpstream() + { + var upstream = new HttpRequestMessage(HttpMethod.Post, destination); + if (route.Provider == null) CopyOpenAiHeaders(request, upstream, identity); + else upstream.Headers.Authorization = new AuthenticationHeaderValue("Bearer", route.Provider.ApiKey); + upstream.Content = route.Provider != null || removeReasoning ? + new GatewayRoutedContent(bytesIn, body ?? new JsonObject(), route.Provider != null, route.Model?.GenericMetadata == true, removeReasoning) : + new ReadOnlyMemoryContent(bytesIn); + upstream.Content.Headers.ContentType = new MediaTypeHeaderValue("application/json"); + return upstream; + } + async Task BeforeRetry(int retry, TimeSpan delay, CancellationToken token) + { + retries = retry; + LastError = "Upstream HTTP 507; automatic recovery " + retry + "/" + GatewayUpstreamRetry.MaximumRetries + "."; + if (!started) + { + response.StatusCode = 200; response.ContentType = "text/event-stream"; + response.Headers["Cache-Control"] = "no-store"; response.SendChunked = true; started = true; + } + // Real writes during backoff detect a cancelled/disconnected caller before + // another inference request. Comments never create a response or chat turn. + await RetryCommentAsync(response, ": Switch HTTP 507 recovery " + retry + "/3\n\n", token).ConfigureAwait(false); + var until = DateTimeOffset.UtcNow + delay; + while (DateTimeOffset.UtcNow < until) + { + var remaining = until - DateTimeOffset.UtcNow; + if (remaining > TimeSpan.Zero) await Task.Delay(remaining < TimeSpan.FromMilliseconds(250) ? remaining : TimeSpan.FromMilliseconds(250), token).ConfigureAwait(false); + await RetryCommentAsync(response, ": Switch waiting for upstream recovery\n\n", token).ConfigureAwait(false); + } + var current = _authenticate(request.Headers["Authorization"], request.Headers["ChatGPT-Account-ID"]); + if (current == null || current.Token != identity.Token || current.AccountId != identity.AccountId || current.ApiKey != identity.ApiKey) + throw new InvalidOperationException("The original sign-in changed before automatic recovery."); + } + using var result = await GatewayUpstreamRetry.SendAsync(_http, CreateUpstream, + path == "/backend-api/codex/responses" && original.Streaming, BeforeRetry, cancellation).ConfigureAwait(false); + ReleaseInput(); if (!result.IsSuccessStatusCode) - { await ErrorAsync(response, (int)result.StatusCode is >= 300 and < 400 ? 502 : (int)result.StatusCode, "The selected upstream returned HTTP " + (int)result.StatusCode + ". No request was retried.", cancellation).ConfigureAwait(false); return; } + { + LastError = "The selected upstream returned HTTP " + (int)result.StatusCode + ". " + + (retries == 0 ? "No request was retried." : "Automatic HTTP 507 recovery stopped after " + retries + " of 3 retries."); + if (started) await RetryFailureAsync(response, LastError, cancellation).ConfigureAwait(false); + else await ErrorAsync(response, (int)result.StatusCode is >= 300 and < 400 ? 502 : (int)result.StatusCode, LastError, cancellation).ConfigureAwait(false); + return; + } + if (started && result.Content.Headers.ContentType?.MediaType != "text/event-stream") + { + LastError = "The selected upstream did not return the requested event stream after HTTP 507 recovery."; + await RetryFailureAsync(response, LastError, cancellation).ConfigureAwait(false); return; + } _store.RecordSuccessfulRoute(request.Headers["session_id"] ?? request.Headers["conversation_id"], identity, routeIdentity); - response.StatusCode = (int)result.StatusCode; response.ContentType = result.Content.Headers.ContentType?.ToString() ?? "application/json"; - response.Headers["Cache-Control"] = "no-store"; response.SendChunked = true; started = true; + if (!started) + { + response.StatusCode = (int)result.StatusCode; response.ContentType = result.Content.Headers.ContentType?.ToString() ?? "application/json"; + response.Headers["Cache-Control"] = "no-store"; response.SendChunked = true; started = true; + } await using var stream = await result.Content.ReadAsStreamAsync(cancellation).ConfigureAwait(false); var buffer = new byte[16384]; int count; while ((count = await stream.ReadAsync(buffer, cancellation).ConfigureAwait(false)) > 0) @@ -224,15 +266,15 @@ void ReleaseInput(HttpRequestMessage outgoing) } catch (GatewayBusyException) { if (!started) await TryErrorAsync(context.Response, 503, "The local model gateway's request buffer budget is busy. Try again when uploads finish.", cancellation).ConfigureAwait(false); } - catch (OperationCanceledException) { LastError = "The model request was cancelled or timed out."; context.Response.Abort(); } + catch (OperationCanceledException) { LastError = "The model request was cancelled or timed out."; AbortResponse(context.Response); } catch (Exception error) { LastError = "The model request or upstream connection failed (" + error.GetType().Name + ")."; - if (started) context.Response.Abort(); + if (started) AbortResponse(context.Response); else { try { await ErrorAsync(context.Response, error is InvalidDataException or JsonException or ArgumentException ? 400 : 502, LastError, cancellation).ConfigureAwait(false); } - catch (Exception transport) when (transport is IOException or HttpListenerException or OperationCanceledException or ObjectDisposedException) { context.Response.Abort(); } + catch (Exception transport) when (transport is IOException or HttpListenerException or OperationCanceledException or ObjectDisposedException) { AbortResponse(context.Response); } } } finally { if (largeBody) _largeBodies.Release(); if (workSlot) _slots.Release(); if (streamSlot) _streamSlots.Release(); context.Response.Close(); } @@ -309,10 +351,25 @@ private static async Task ErrorAsync(HttpListenerResponse response, int status, try { await response.OutputStream.WriteAsync(bytes, cancellation).ConfigureAwait(false); } finally { response.Close(); } } + private static async Task RetryCommentAsync(HttpListenerResponse response, string comment, CancellationToken cancellation) + { + await response.OutputStream.WriteAsync(Encoding.UTF8.GetBytes(comment), cancellation).ConfigureAwait(false); + await response.OutputStream.FlushAsync(cancellation).ConfigureAwait(false); + } + private static Task RetryFailureAsync(HttpListenerResponse response, string message, CancellationToken cancellation) => + RetryCommentAsync(response, "data: " + JsonSerializer.Serialize(new { type = "response.failed", response = new { + id = "switch_507_" + Guid.NewGuid().ToString("N"), @object = "response", status = "failed", output = Array.Empty(), + error = new { code = "switch_upstream_recovery_stopped", message } } }) + "\n\n", cancellation); private static async Task TryErrorAsync(HttpListenerResponse response, int status, string message, CancellationToken cancellation) { try { await ErrorAsync(response, status, message, cancellation).ConfigureAwait(false); } - catch (Exception error) when (error is IOException or HttpListenerException or ObjectDisposedException or OperationCanceledException) { response.Abort(); } + catch (Exception error) when (error is IOException or HttpListenerException or ObjectDisposedException or OperationCanceledException) { AbortResponse(response); } + } + private static void AbortResponse(HttpListenerResponse response) + { + try { response.Abort(); } + // Listener shutdown can dispose HTTP.sys before the cancelled request unwinds. + catch (ObjectDisposedException) { } } private async Task RejectAsync(HttpListenerContext context) { diff --git a/src/application/ModelProviders.cs b/src/application/ModelProviders.cs index 9dc50dd..9fc8891 100644 --- a/src/application/ModelProviders.cs +++ b/src/application/ModelProviders.cs @@ -100,6 +100,16 @@ internal void RecordSuccessfulRoute(string? session, GatewayIdentity identity, s Update(next => next.ConversationRoutes[key] = new GatewayConversationRoute { Route = route, Mixed = mixed }); } } + internal static string? RecoveryConversationKey(string? session, string auth) + { + if (string.IsNullOrEmpty(session) || session.Length > 256) return null; + using var json = JsonDocument.Parse(auth); + var root = json.RootElement; string? identity = null; + if (root.TryGetProperty("tokens", out var tokens)) identity = JsonValue.String(tokens, "account_id"); + identity ??= JsonValue.String(root, "OPENAI_API_KEY"); + if (string.IsNullOrEmpty(identity)) return null; + return Convert.ToHexString(SHA256.HashData(Encoding.UTF8.GetBytes(identity + "\n" + session))); + } private static bool IsMixed(GatewayConversationRoute? previous, string route) => previous?.Mixed == true || (previous != null ? previous.Route != route : route != "openai"); private static string? ConversationKey(string? session, GatewayIdentity identity) => diff --git a/src/core/AccountSwitchCoordinator.cs b/src/core/AccountSwitchCoordinator.cs index dc937f3..34ed73f 100644 --- a/src/core/AccountSwitchCoordinator.cs +++ b/src/core/AccountSwitchCoordinator.cs @@ -73,6 +73,21 @@ internal interface IAccountSwitchBackend : IAsyncDisposable Task ResumeAsync(SwitchChat chat, Action sending, CancellationToken cancellation); } +internal static class ActiveChatTitleUpdate +{ + public static async Task RunAsync(IAccountSwitchBackend backend, DateTimeOffset at, Action progress, CancellationToken cancellation) + { + var chats = await backend.CaptureAsync(cancellation); + var saved = 0; + foreach (var chat in chats) + { + await backend.BackupNameAsync(chat, at, cancellation); + progress(++saved); + } + return saved; + } +} + internal sealed class AccountSwitchCoordinator { public const int MaximumRecoveryAttempts = 2; diff --git a/src/desktop/DeskWindow.cs b/src/desktop/DeskWindow.cs index 77541e1..8cad742 100644 --- a/src/desktop/DeskWindow.cs +++ b/src/desktop/DeskWindow.cs @@ -33,6 +33,7 @@ internal sealed class DeskWindow : Window private readonly Dictionary _volumeTemperaturePressure = new(); private readonly List _operations = new(); private AccountService? _service; + private DesktopStreamRecoveryMonitor? _streamRecovery; private DeskTheme _theme; private Grid _accounts = new(), _hardware = new(); private ScrollViewer _accountScroll = new(); @@ -40,12 +41,13 @@ internal sealed class DeskWindow : Window private TextBlock _status = new(); private Button _add = new(), _cancel = new(), _refresh = new(), _settingsButton = new(), _startCodex = new(); private MenuItem _restartCodex = new(); + private Button _stampTitles = new(); private IDesktopMenuBar? _tray; private readonly Dictionary _nativeItems = new(); private CancellationTokenSource? _login, _switch; private string? _switchTarget; private SwitchPhase _phase; - private bool _ready; + private bool _ready, _stampingTitles; private bool _initializing; private bool _providerEditing; private string _statusMessage = ""; @@ -105,6 +107,9 @@ private async Task InitializeAsync(CancellationToken token) await _service.InitializeAsync(token); _ready = true; if (OperatingSystem.IsLinux()) _statusMessage = Loc.T(_service.HasLinuxDesktop ? "msg.linux_desktop_ready" : "msg.linux_cli_only"); _sessions.Start(); _resources.Start(); _drives.Configure(_service.Vault.Data.Drives.ToArray()); _drives.Start(); + _streamRecovery ??= new DesktopStreamRecoveryMonitor(_service, _sessions, () => _ready && !Exiting && _login == null && _switch == null && + !_providerEditing && !_service.IsBusy && _service.Vault.Data.ActivePendingSwitch == null); + _streamRecovery.Start(); UpdateTray(); RenderAccounts(); await _service.RefreshAllAsync(null, token); await RecoverAutomaticSwitchAsync(); @@ -154,8 +159,12 @@ private void Build() var toolbar = new Grid { Margin = new Thickness(8, 1), ColumnDefinitions = new ColumnDefinitions(nativeTitle ? "Auto,*,Auto" : "*,Auto,*") }; _cancel = _theme.Button(Loc.T("취소"), () => { _login?.Cancel(); _switch?.Cancel(); }); _cancel.IsVisible = _login != null || _switch != null; _cancel.HorizontalAlignment = HorizontalAlignment.Left; - toolbar.Children.Add(_cancel); - _sessionValues = new StackPanel { Orientation = Orientation.Horizontal, Spacing = nativeTitle ? 12 : 24, HorizontalAlignment = HorizontalAlignment.Center }; + var actions = new StackPanel { Orientation = Orientation.Horizontal, HorizontalAlignment = HorizontalAlignment.Left }; + _stampTitles = _theme.Button(Loc.T("titles.button"), () => _ = StampTitlesAsync()); + _stampTitles.Name = "StampActiveTitlesButton"; _stampTitles.FontSize = 10; _stampTitles.Padding = new Thickness(5, 1); _stampTitles.MinHeight = 20; + ToolTip.SetTip(_stampTitles, Loc.T("titles.help")); actions.Children.Add(_cancel); actions.Children.Add(_stampTitles); + toolbar.Children.Add(actions); + _sessionValues = new StackPanel { Orientation = Orientation.Horizontal, Spacing = 12, HorizontalAlignment = HorizontalAlignment.Center }; Grid.SetColumn(_sessionValues, 1); toolbar.Children.Add(_sessionValues); var automaticSwitchLabel = Loc.T(_service?.Vault.Data.AutomaticSwitchEnabled == true ? "auto.on" : "auto.off"); @@ -228,9 +237,10 @@ private void RenderAccounts() { _accounts.Children.Clear(); _accounts.ColumnDefinitions.Clear(); _accounts.RowDefinitions.Clear(); _cards.Clear(); if (_tray != null) UpdateTray(); - _add.IsEnabled = _ready && _login == null; _cancel.IsVisible = _login != null || _switch != null; + _add.IsEnabled = _ready && _login == null && !_stampingTitles; _cancel.IsVisible = _login != null || _switch != null; _startCodex.IsEnabled = _ready && _login == null && _switch == null; _restartCodex.IsEnabled = _ready && !Exiting && _switch == null && _login == null && !_providerEditing && _service?.Accounts.Any(profile => profile.IsCurrent) == true && _service.Vault.Data.ActivePendingSwitch is not { RestartOnly: false }; + _stampTitles.IsEnabled = _ready && _preview == null && _login == null && _switch == null && !_stampingTitles && _service?.Vault.Data.ActivePendingSwitch == null; _refresh.IsEnabled = !_initializing && _preview == null && _host.AccountUnavailableReason == null; var profiles = (_preview ?? _service?.Accounts ?? Array.Empty()).OrderByDescending(p => p.IsCurrent).ThenBy(p => p.DisplayName, StringComparer.CurrentCulture).ToArray(); if (profiles.Length == 0) @@ -251,7 +261,7 @@ private void RenderAccounts() resume ? "switch.recover" : profile.RequiresApply ? "로그인 적용" : profile.IsCurrent ? "사용 중" : "전환"; var button = _theme.Button(Loc.T(text), () => _ = SwitchAsync(profile)); if (resume) ToolTip.SetTip(button, pending?.Error); - button.IsEnabled = _ready && _switch == null && _login == null && (!profile.IsCurrent || profile.RequiresApply || resume); + button.IsEnabled = _ready && _switch == null && _login == null && !_stampingTitles && (!profile.IsCurrent || profile.RequiresApply || resume); button.FontSize = 10; button.Padding = new Thickness(5, 2); var more = _theme.IconButton(DeskButtonIconKind.More, Loc.T("계정 메뉴"), () => { }); more.Width = 25; more.Padding = new Thickness(3, 2); more.Click += (_, _) => AccountMenu(more, profile); @@ -286,7 +296,7 @@ private async Task RunAsync(Func action) } private async Task AddAccountAsync(string? expectedAccountKey = null) { - if (!_ready || _login != null || _switch != null) return; + if (!_ready || _login != null || _switch != null || _stampingTitles) return; _login = CancellationTokenSource.CreateLinkedTokenSource(_stop.Token); RenderAccounts(); try { @@ -300,7 +310,7 @@ await RunAsync(async _ => } private async Task SwitchAsync(AccountProfile profile, bool automatic = false, bool creditPause = false, bool restart = false) { - if (!_ready || _switch != null || _login != null) return; + if (!_ready || _switch != null || _login != null || _stampingTitles) return; var creditPending = _service!.Vault.Data.ActivePendingSwitch; restart |= creditPending?.RestartOnly == true; if (restart && creditPending != null && !creditPending.RestartOnly) { _statusMessage = Loc.T("남은 채팅 재개 작업을 먼저 처리해 주세요."); _status.Text = _statusMessage; return; } @@ -340,6 +350,27 @@ await RunAsync(async _ => } finally { if (automatic && _service!.Vault.Data.ActivePendingSwitch != null) AccountAutomation.Pause(_service); _switch.Dispose(); _switch = null; _switchTarget = null; if (!Exiting) { Build(); } } } + private async Task StampTitlesAsync() + { + if (!_ready || Exiting || _preview != null || _service == null || _login != null || _switch != null || _stampingTitles || _service.Vault.Data.ActivePendingSwitch != null) return; + _stampingTitles = true; RenderAccounts(); + var saved = 0; + try + { + await RunAsync(async token => + { + try + { + await using var backend = new DesktopSwitchBackend(_service, _sessions); + await ActiveChatTitleUpdate.RunAsync(backend, DateTimeOffset.UtcNow, count => saved = count, token); + _statusMessage = Loc.F("titles.done", saved); + } + catch (OperationCanceledException) { throw; } + catch (Exception error) { _statusMessage = Loc.F("titles.failed", saved, Loc.Error(UserErrors.Describe(error))); } + }); + } + finally { _stampingTitles = false; if (!Exiting) RenderAccounts(); } + } internal ContextMenu BuildAutomaticSwitchMenu() { var service = _service; @@ -375,7 +406,7 @@ internal ContextMenu BuildAutomaticSwitchMenu() } private async Task TryAutomaticSwitchAsync() { - if (!_ready || Exiting || _preview != null || _service == null || _login != null || _switch != null || _providerEditing) return; + if (!_ready || Exiting || _preview != null || _service == null || _login != null || _switch != null || _providerEditing || _stampingTitles) return; if (!_sessions.Current.Available && _service.Vault.Data.ActivePendingSwitch?.CreditProtectionPause != true) { if (!_service.Vault.Data.AllowCreditUse && _service.Accounts.Any(p => p.IsCurrent && AutomaticSwitchPolicy.Exhausted(p, DateTimeOffset.UtcNow)) && @@ -400,7 +431,7 @@ await RunAsync(async _ => } private async Task RecoverAutomaticSwitchAsync() { - if (!_ready || Exiting || _preview != null || _service == null || _login != null || _switch != null || _service.IsBusy) return; + if (!_ready || Exiting || _preview != null || _service == null || _login != null || _switch != null || _service.IsBusy || _stampingTitles) return; var pending = _service.Vault.Data.ActivePendingSwitch; if (pending is not { Automatic: true, AutomaticRecoveryAttempts: 0, CreditProtectionPause: false }) return; var target = _service.Accounts.FirstOrDefault(p => p.Key == pending.TargetAccountKey); @@ -423,23 +454,23 @@ private void AccountMenu(Button target, AccountProfile profile) Item(Loc.T("추이 보기"), () => new HistoryWindow(_service!, profile.Key, _theme).Show(this)), Item(Loc.T("이 계정 즉시 새로 고침"), () => _ = RunAsync(token => _service!.RefreshAccountAsync(profile.Key, token, true))), Item(Loc.T("이름 변경"), async () => { var value = await DeskDialogs.Prompt(this, _theme, Loc.T("이름 변경"), profile.Alias); if (value != null) await RunAsync(_ => { _service!.Rename(profile.Key, value); return Task.CompletedTask; }); }), - Item(Loc.T("삭제"), async () => { if (_switch != null) return; if (await DeskDialogs.Confirm(this, _theme, Loc.T("삭제"), profile.DisplayName)) await RunAsync(_ => _service!.RemoveAsync(profile.Key)); }) }; + Item(Loc.T("삭제"), async () => { if (_switch != null || _stampingTitles) return; if (await DeskDialogs.Confirm(this, _theme, Loc.T("삭제"), profile.DisplayName) && !_stampingTitles) await RunAsync(_ => _service!.RemoveAsync(profile.Key)); }) }; menu.Open(target); } private void RenderSessions() { _sessionValues.Children.Clear(); var current = _sessions.Current; - var numbers = new int?[] { current.Available ? current.Running : null, current.Available ? current.Queued : null, current.Available ? current.Goals : null }; - var labels = new[] { "실행", "큐", "목표" }; - for (var i = 0; i < 3; i++) + var numbers = new int?[] { current.Available ? current.Running : null, current.Available ? current.Queued : null, current.Available ? current.Goals : null, current.Available ? current.Errors : null }; + var labels = new[] { "실행", "큐", "목표", "sessions.errors" }; + for (var i = 0; i < labels.Length; i++) { var cell = new StackPanel { Orientation = Orientation.Horizontal, Spacing = 6, VerticalAlignment = VerticalAlignment.Center }; var label = _theme.Text(Loc.T(labels[i]), 10, "muted"); label.VerticalAlignment = VerticalAlignment.Center; - var value = _theme.Text(numbers[i]?.ToString(Loc.Culture) ?? "—", 16); value.VerticalAlignment = VerticalAlignment.Center; value.FontWeight = FontWeight.SemiBold; - cell.Children.Add(label); cell.Children.Add(value); _sessionValues.Children.Add(cell); + var value = _theme.Text(numbers[i]?.ToString(Loc.Culture) ?? "—", 16, i == 3 && numbers[i] > 0 ? "warning" : numbers[i].HasValue ? "text" : "muted"); value.VerticalAlignment = VerticalAlignment.Center; value.FontWeight = FontWeight.SemiBold; + cell.Children.Add(label); cell.Children.Add(value); if (i == 3) ToolTip.SetTip(cell, Loc.T("sessions.errors_help")); _sessionValues.Children.Add(cell); } ToolTip.SetTip(_sessionValues, current.Available ? Loc.F("승인/입력 대기 {0}", current.Waiting) + "\n" + - (current.Queued.HasValue ? Loc.F("큐는 예약 메시지 수\n보류된 큐 {0}", current.PausedQueued) : Loc.Error(current.QueueProblem ?? "큐 상태 확인 중…")) + "\n" + Loc.F("목표는 진행 중인 Goal 수\n멈춘 목표 {0}", current.PausedGoals) : Loc.Error(current.Problem ?? "채팅 상태 확인 중…")); + (current.Queued.HasValue ? Loc.F("큐는 예약 메시지 수\n보류된 큐 {0}", current.PausedQueued) : Loc.Error(current.QueueProblem ?? "큐 상태 확인 중…")) + "\n" + Loc.F("목표는 진행 중인 Goal 수\n멈춘 목표 {0}", current.PausedGoals) + "\n" + Loc.T("sessions.errors_help") : Loc.Error(current.Problem ?? "채팅 상태 확인 중…")); } private void RenderHardware() { @@ -601,6 +632,7 @@ public async Task ExitAsync() { if (Exiting) return; Exiting = true; _clock.Stop(); _stop.Cancel(); _login?.Cancel(); _switch?.Cancel(); await Task.WhenAll(_operations.ToArray()); + if (_streamRecovery != null) await _streamRecovery.DisposeAsync(); if (_service != null) await _service.StopAsync(); await _sessions.DisposeAsync(); await _resources.DisposeAsync(); await _drives.DisposeAsync(); _tray?.Dispose(); _service?.ReleaseDesktop(); diff --git a/src/platforms/protocol/DesktopSessionMonitor.cs b/src/platforms/protocol/DesktopSessionMonitor.cs index d7e6039..8e667cb 100644 --- a/src/platforms/protocol/DesktopSessionMonitor.cs +++ b/src/platforms/protocol/DesktopSessionMonitor.cs @@ -35,6 +35,7 @@ public void Start() _run ??= Task.Run(RunAsync); } public DesktopThreadState[] Threads => _ledger.Threads(); + internal int? QueuedFor(string id) => _ledger.QueuedFor(id); public void RestartDiscovery() { _ledger.Reset(); Publish(_ledger.Read()); diff --git a/src/platforms/protocol/DesktopTurnOutcome.cs b/src/platforms/protocol/DesktopTurnOutcome.cs new file mode 100644 index 0000000..b2d3101 --- /dev/null +++ b/src/platforms/protocol/DesktopTurnOutcome.cs @@ -0,0 +1,70 @@ +using System; +using System.IO; +using System.Linq; +using System.Text; +using System.Text.Json; + +namespace CodexAccountDesk; + +internal sealed record DesktopTurnOutcome(string Id, string Status, string? Error, string Model, string Provider, DateTimeOffset At); + +// Read only the bounded end of the original rollout. Never modify a transcript or +// infer failure from idle/EOF alone: a persisted task_complete must name the turn. +internal static class DesktopTurnOutcomes +{ + internal static DesktopTurnOutcome? Read(string home, string id) + { + if (!Guid.TryParse(id, out _)) return null; + using var state = new ReadOnlySqlite(Path.Combine(home, "state_5.sqlite")); + var rows = state.Read("SELECT rollout_path, model, model_provider, archived, originator, source, agent_path FROM threads WHERE id = ?", id); + if (rows.Count != 1) return null; + var row = rows[0]; + if (row[3] != "0" || row[4] != "Codex Desktop" || row[5] != "vscode" || row[6] != null || string.IsNullOrEmpty(row[1]) || string.IsNullOrEmpty(row[2])) return null; + var root = new DirectoryInfo(Path.Combine(home, "sessions")); + var sessions = (root.ResolveLinkTarget(true)?.FullName ?? root.FullName).TrimEnd(Path.DirectorySeparatorChar) + Path.DirectorySeparatorChar; + static string Normal(string path) => Path.GetFullPath(path).Replace("\\\\?\\", "", StringComparison.Ordinal); + var path = Normal(row[0]!); + if (!path.StartsWith(Normal(sessions).TrimEnd(Path.DirectorySeparatorChar) + Path.DirectorySeparatorChar, + OperatingSystem.IsWindows() ? StringComparison.OrdinalIgnoreCase : StringComparison.Ordinal) || Path.GetExtension(path) != ".jsonl" || + !Path.GetFileName(path).EndsWith("-" + id + ".jsonl", StringComparison.OrdinalIgnoreCase) || + (File.GetAttributes(path) & FileAttributes.ReparsePoint) != 0) return null; + using var file = new FileStream(path, FileMode.Open, FileAccess.Read, FileShare.ReadWrite | FileShare.Delete); + var offset = Math.Max(0, file.Length - 4 * 1024 * 1024); + file.Position = offset; + var bytes = new byte[checked((int)(file.Length - offset))]; file.ReadExactly(bytes); + return ParseTail(bytes, offset != 0, row[1]!, row[2]!); + } + internal static DesktopTurnOutcome? ParseTail(byte[] bytes, bool partialFirstLine, string model, string provider) + { + DesktopTurnOutcome? latest = null; + var start = 0; + for (var i = 0; i < bytes.Length; i++) + { + if (bytes[i] != (byte)'\n') continue; + var length = i - start; var line = start; start = i + 1; + if (partialFirstLine) { partialFirstLine = false; continue; } + // Very large tool/assistant items cannot be completion metadata. If a + // completion itself exceeds the bound, leave the outcome unknown. + if (length > 512 * 1024) { latest = null; continue; } + using var document = JsonDocument.Parse(bytes.AsMemory(line, length)); + var root = document.RootElement; + if (JsonValue.String(root, "type") != "event_msg" || !root.TryGetProperty("payload", out var payload)) continue; + var type = JsonValue.String(payload, "type"); + if (type is "task_started" or "turn_aborted" or "task_interrupted") { latest = null; continue; } + if (type != "task_complete") continue; + var turn = JsonValue.String(payload, "turn_id"); + if (turn == null || !Guid.TryParse(turn, out _) || !DateTimeOffset.TryParse(JsonValue.String(root, "timestamp"), out var at)) { latest = null; continue; } + var hasError = payload.TryGetProperty("error", out var value) && value.ValueKind != JsonValueKind.Null; + var error = value.ValueKind == JsonValueKind.Object ? JsonValue.String(value, "message") : + value.ValueKind == JsonValueKind.String ? value.GetString() : null; + var status = JsonValue.String(payload, "status"); + // An explicit interruption wins over any attached error text. Missing + // error prose must not turn a persisted failure into a success. + if (status == "interrupted") latest = new(turn, status, error, model, provider, at); + else if (status == "failed" || hasError) latest = new(turn, "failed", error, model, provider, at); + else if (status is null or "completed") latest = new(turn, "completed", null, model, provider, at); + else latest = null; + } + return latest; + } +} diff --git a/src/platforms/protocol/SessionActivity.cs b/src/platforms/protocol/SessionActivity.cs index ac4d5d1..1a6c291 100644 --- a/src/platforms/protocol/SessionActivity.cs +++ b/src/platforms/protocol/SessionActivity.cs @@ -6,7 +6,7 @@ namespace CodexAccountDesk; internal sealed record SessionActivity(bool Available, int Running, int Waiting, int Observed, string? Problem = null, - int? Queued = null, int? Goals = null, int PausedQueued = 0, int PausedGoals = 0, string? QueueProblem = null); + int? Queued = null, int? Goals = null, int PausedQueued = 0, int PausedGoals = 0, string? QueueProblem = null, int? Errors = null); internal sealed record DesktopThreadState(string Id, string Owner, string Status, bool Waiting, string? Goal, string? Title, string? ResumeState, string? LastTurnId, string? LastTurnStatus, string? LastMessageId); @@ -56,6 +56,15 @@ public DesktopThreadState[] Threads() return new DesktopThreadState(pair.Key, e.Owner, e.Status!, e.Waiting, e.Goal, e.Title, e.ResumeState, e.LastTurnId, e.LastTurnStatus, e.LastMessageId); }).ToArray(); } + internal int? QueuedFor(string id) + { + lock (_sync) + { + if (!_discovered || _queues == null) return null; + return DesktopQueueState.Count(_queueOverrides.TryGetValue(id, out var update) ? update.Messages : _queues.Saved.GetValueOrDefault(id) ?? Array.Empty(), + _queues.Server.GetValueOrDefault(id) ?? Array.Empty()).Total; + } + } public SessionActivity Read() { lock (_sync) @@ -71,7 +80,8 @@ public SessionActivity Read() roots.Count(e => e.Status == "active" && e.Waiting), roots.Length, Queued: queues?.Sum(q => q.Total), Goals: available && roots.All(e => e.GoalKnown) ? roots.Count(e => e.Goal == "active") : null, PausedQueued: queues?.Sum(q => q.Paused) ?? 0, - PausedGoals: roots.Count(e => e.Goal is "paused" or "blocked" or "usageLimited" or "budgetLimited"), QueueProblem: _queueProblem); + PausedGoals: roots.Count(e => e.Goal is "paused" or "blocked" or "usageLimited" or "budgetLimited"), QueueProblem: _queueProblem, + Errors: available ? roots.Count(e => e.Status == "systemError" || e.Status == "idle" && e.LastTurnStatus == "failed") : null); } } public bool Apply(string id, string owner, JsonElement change) diff --git a/src/windows/MainWindow.cs b/src/windows/MainWindow.cs index abef9bf..ff02d83 100644 --- a/src/windows/MainWindow.cs +++ b/src/windows/MainWindow.cs @@ -23,6 +23,7 @@ internal sealed class MainWindow : Window { private readonly AccountService _service; private readonly DesktopSessionMonitor _sessions; + private readonly DesktopStreamRecoveryMonitor _streamRecovery; private readonly HardwareResourceMonitor _hardware = new(); private readonly HardwareUsagePanel _hardwarePanel = new(); private readonly DriveMonitor _drives = new(WindowsDiskTemperature.Read); @@ -40,6 +41,7 @@ internal sealed class MainWindow : Window private readonly MenuItem _restartCodex = new(); private readonly Button _homeActions = new(); private bool _providerEditing; + private readonly Button _stampTitles = new(); private MenuItem? _importCurrent; private readonly Border _loginPanel = new(); private readonly List _tankCards = new(); @@ -53,7 +55,7 @@ internal sealed class MainWindow : Window private readonly DispatcherTimer _clockTimer = new() { Interval = TimeSpan.FromSeconds(1) }; private Forms.NotifyIcon? _tray; private bool _ready, _loggingIn, _switching, _closing, _exit; - private bool _started; + private bool _started, _stampingTitles; private bool _cpuTemperatureChanging; private bool CanRefresh => !_closing && (_ready || _initialization?.IsCompleted != false); private string? _queuedKey; @@ -72,6 +74,8 @@ public MainWindow(AccountService service, Program.Options options) _service = service; _options = options; _drivePanel = new(OpenDriveSettings); _sessions = new DesktopSessionMonitor(options.CodexHome); + _streamRecovery = new DesktopStreamRecoveryMonitor(service, _sessions, () => _ready && !_closing && !_switching && !_loggingIn && !_providerEditing && + !_service.IsBusy && _service.Vault.Data.ActivePendingSwitch == null); _theme = new Burn0Theme(service.Vault.Data.Theme != "light"); Title = "Codex Switch"; Icon = AppIdentity.WindowIcon; Width = 600; Height = 400; MinWidth = 500; MinHeight = 360; WindowStartupLocation = WindowStartupLocation.CenterScreen; @@ -86,6 +90,7 @@ public MainWindow(AccountService service, Program.Options options) _add.Click += async (_, _) => await AddAsync(); _cancelLogin.Click += (_, _) => _loginCancellation?.Cancel(); _cancelSwitch.Click += (_, _) => CancelSwitch(); + _stampTitles.Click += async (_, _) => await StampTitlesAsync(); _service.Changed += ServiceChanged; _sessions.Changed += SessionChanged; _hardware.Changed += HardwareChanged; @@ -127,7 +132,7 @@ private async Task EnableCpuTemperatureAsync() } private void ApplyTheme() { - foreach (var element in new FrameworkElement[] { _homeActions, _automaticState, _sessionCount, _hardwarePanel, _drivePanel, _status, _loginText, _add, _refresh, _cancelLogin, _cancelSwitch, _loginPanel, _scroll }) + foreach (var element in new FrameworkElement[] { _stampTitles, _homeActions, _automaticState, _sessionCount, _hardwarePanel, _drivePanel, _status, _loginText, _add, _refresh, _cancelLogin, _cancelSwitch, _loginPanel, _scroll }) { if (element.Parent is Panel panel) panel.Children.Remove(element); else if (element.Parent is ContentControl control && ReferenceEquals(control.Content, element)) control.Content = null; @@ -170,6 +175,10 @@ private UIElement BuildRoot() toolbarRow.ColumnDefinitions.Add(new ColumnDefinition()); toolbarRow.ColumnDefinitions.Add(new ColumnDefinition { Width = GridLength.Auto }); toolbarRow.ColumnDefinitions.Add(new ColumnDefinition()); + ConfigureButton(_stampTitles, "titles.button", null); + _stampTitles.Name = "StampActiveTitlesButton"; _stampTitles.HorizontalAlignment = HorizontalAlignment.Left; + _stampTitles.FontSize = 10; _stampTitles.Padding = new Thickness(5, 2, 5, 2); + _stampTitles.ToolTip = Loc.T("titles.help"); toolbarRow.Children.Add(_stampTitles); _sessionCount.HorizontalAlignment = HorizontalAlignment.Center; Grid.SetColumn(_sessionCount, 1); toolbarRow.Children.Add(_sessionCount); body.Children.Add(toolbarRow); @@ -373,6 +382,7 @@ private async Task StartAsync() _started = true; if (_options.UiEvidenceDirectory != null) return; _sessions.Start(); + if (_options.ReportPath == null) _streamRecovery.Start(); _hardware.Start(); _drives.Configure(_service.Vault.Data.Drives.ToArray()); _drives.Start(); if (_options.ReportPath == null) { CreateTray(); _refreshTimer.Start(); _clockTimer.Start(); } @@ -514,8 +524,30 @@ private async Task RefreshAsync() await RecoverAutomaticSwitchAsync(); await TryAutomaticSwitchAsync(); } + private async Task StampTitlesAsync() + { + if (!_ready || _closing || _switching || _loggingIn || _stampingTitles || PendingSwitch() != null) return; + _stampingTitles = true; UpdateControls(); RenderAccounts(); + var saved = 0; + try + { + await RunAsync("titles.working", async token => + { + try + { + await using var backend = new DesktopSwitchBackend(_service, _sessions); + await ActiveChatTitleUpdate.RunAsync(backend, DateTimeOffset.UtcNow, count => saved = count, token); + SetStatus("titles.done", false, saved); + } + catch (OperationCanceledException) { throw; } + catch (Exception error) { SetStatus("titles.failed", true, saved, Loc.Error(UserErrors.Describe(error))); } + }); + } + finally { _stampingTitles = false; UpdateControls(); RenderAccounts(); } + } private Task ImportCurrentAsync() => RunAsync("현재 로그인 가져오는 중…", async token => { + if (_stampingTitles) return; await EnsureInitializedAsync(token); var profile = await Task.Run(() => { var p = _service.SynchronizeCurrent(); _service.Vault.Save(); return p; }, token); if (profile == null) throw new InvalidOperationException("Codex에서 먼저 로그인하거나 계정 추가를 선택해 주세요."); @@ -529,7 +561,7 @@ private Task ReauthenticateAsync(string key) } private async Task AddAsync(string? expectedAccountKey = null) { - if (!_ready || _loggingIn || _closing || _switching) return; + if (!_ready || _loggingIn || _closing || _switching || _stampingTitles) return; using var cancellation = CancellationTokenSource.CreateLinkedTokenSource(_lifetime.Token); _loginCancellation = cancellation; _loggingIn = true; _loginText.Text = Loc.T("브라우저에서 추가할 계정으로 로그인해 주세요."); UpdateControls(); await RunAsync("계정 로그인 중…", async _ => @@ -584,7 +616,7 @@ private void RenderSessions(SessionActivity? illustrative = null) _sessionCount.ToolTip = activity.Available ? Loc.T("이 PC의 로컬 Codex 주 채팅\n하위 에이전트 제외\n계정별 구분 없음") + "\n" + Loc.F("승인/입력 대기 {0}", activity.Waiting) + "\n" + (activity.Queued.HasValue ? Loc.F("큐는 예약 메시지 수\n보류된 큐 {0}", activity.PausedQueued) : Loc.T(activity.QueueProblem ?? "큐 상태 확인 중…")) + "\n" + - (activity.Goals.HasValue ? Loc.F("목표는 진행 중인 Goal 수\n멈춘 목표 {0}", activity.PausedGoals) : Loc.T("목표 상태 확인 중…")) : Loc.T(activity.Problem ?? "채팅 상태 확인 중…"); + (activity.Goals.HasValue ? Loc.F("목표는 진행 중인 Goal 수\n멈춘 목표 {0}", activity.PausedGoals) : Loc.T("목표 상태 확인 중…")) + "\n" + Loc.T("sessions.errors_help") : Loc.T(activity.Problem ?? "채팅 상태 확인 중…"); } private void RenderAccounts() { @@ -603,7 +635,7 @@ private void RenderAccounts() switchButton.Tag = "account-switch-action"; if (pendingResume) switchButton.ToolTip = PendingSwitch()?.Error; switchButton.Margin = new Thickness(0); switchButton.FontSize = 12; switchButton.Padding = new Thickness(8, 5, 8, 5); - switchButton.IsEnabled = _ready && !_switching && !_loggingIn && !_closing && (!profile.IsCurrent || profile.RequiresApply || pendingResume); + switchButton.IsEnabled = _ready && !_switching && !_stampingTitles && !_loggingIn && !_closing && (!profile.IsCurrent || profile.RequiresApply || pendingResume); switchButton.Click += async (_, _) => await QueueSwitchAsync(profile); actions.Children.Add(switchButton); var more = NewButton("", "more"); more.Padding = new Thickness(6); more.Margin = new Thickness(4, 0, 0, 0); more.ToolTip = Loc.T("계정 메뉴"); more.Click += (_, _) => OpenAccountMenu(more, profile); actions.Children.Add(more); var details = profile.RequiresApply ? Loc.T("새 로그인 적용 대기") : ""; @@ -728,7 +760,7 @@ private void OpenAccountMenu(Button owner, AccountProfile profile) } var rename = new MenuItem { Header = Loc.T("표시 이름 변경") }; rename.Click += (_, _) => { var alias = PromptForAlias(profile); if (alias != null) _service.Rename(profile.Key, alias); }; var remove = new MenuItem { Header = Loc.T("등록 제거"), IsEnabled = !profile.IsCurrent && profile.Key != _queuedKey }; - remove.Click += async (_, _) => { if (Confirm(Loc.F("{0}의 등록 정보를 제거할까요?", profile.DisplayName), Loc.T("계정 등록 제거"), "등록 제거")) await RunAsync("등록 제거 중…", async _ => { await _service.RemoveAsync(profile.Key); SetStatus("등록 정보를 제거했습니다."); }); }; + remove.Click += async (_, _) => { if (!_stampingTitles && Confirm(Loc.F("{0}의 등록 정보를 제거할까요?", profile.DisplayName), Loc.T("계정 등록 제거"), "등록 제거")) await RunAsync("등록 제거 중…", async _ => { await _service.RemoveAsync(profile.Key); SetStatus("등록 정보를 제거했습니다."); }); }; menu.Items.Add(rename); menu.Items.Add(remove); owner.ContextMenu = menu; menu.IsOpen = true; } private string? PromptForAlias(AccountProfile profile) @@ -811,7 +843,7 @@ private async Task QueueSwitchAsync(AccountProfile profile, bool automatic = fal { var pending = PendingSwitch(); restart |= pending?.RestartOnly == true; - if (_closing || _switching || _loggingIn || !restart && !creditPause && profile.IsCurrent && !profile.RequiresApply && pending?.TargetAccountKey != profile.Key) return; + if (_closing || _switching || _stampingTitles || _loggingIn || !restart && !creditPause && profile.IsCurrent && !profile.RequiresApply && pending?.TargetAccountKey != profile.Key) return; if (restart && pending != null && !pending.RestartOnly) { SetStatus("남은 채팅 재개 작업을 먼저 처리해 주세요.", true); return; } if (pending != null && !pending.CreditProtectionPause && pending.TargetAccountKey != profile.Key) { SetStatus("남은 채팅 재개 작업을 먼저 처리해 주세요.", true); return; } creditPause |= pending is { CreditProtectionPause: true, Phase: not SwitchPhase.CreditPaused } && pending.TargetAccountKey == profile.Key && !AccountAutomation.CanUse(_service, profile, DateTimeOffset.UtcNow); @@ -902,7 +934,7 @@ internal ContextMenu BuildAutomaticSwitchMenu() } private async Task TryAutomaticSwitchAsync() { - if (_options.ReportPath != null || _options.UiEvidenceDirectory != null || !_ready || _closing || _switching || _loggingIn || _providerEditing) return; + if (_options.ReportPath != null || _options.UiEvidenceDirectory != null || !_ready || _closing || _switching || _loggingIn || _providerEditing || _stampingTitles) return; if (!_sessions.Current.Available && PendingSwitch()?.CreditProtectionPause != true) { if (!_service.Vault.Data.AllowCreditUse && _service.Accounts.Any(p => p.IsCurrent && AutomaticSwitchPolicy.Exhausted(p, DateTimeOffset.UtcNow)) && RuntimeLocator.IsDesktopRunning(_service.Installation.DesktopPath)) @@ -928,7 +960,7 @@ await RunAsync("", async _ => } private async Task RecoverAutomaticSwitchAsync() { - if (_options.ReportPath != null || _options.UiEvidenceDirectory != null || !_ready || _closing || _switching || _loggingIn || _service.IsBusy) return; + if (_options.ReportPath != null || _options.UiEvidenceDirectory != null || !_ready || _closing || _switching || _loggingIn || _stampingTitles || _service.IsBusy) return; var pending = PendingSwitch(); if (pending is not { Automatic: true, AutomaticRecoveryAttempts: 0, CreditProtectionPause: false }) return; var target = _service.Accounts.FirstOrDefault(p => p.Key == pending.TargetAccountKey); @@ -950,7 +982,7 @@ private void RenderStatus() _status.ToolTip = _status.Text; _status.Foreground = Color(_statusWarning ? "warning" : "muted"); var empty = _service.Vault.Data.Drives.Count == 0; - _status.Visibility = _status.Text.Length > 0 && (_statusWarning || empty) ? Visibility.Visible : Visibility.Collapsed; + _status.Visibility = _status.Text.Length > 0 && (_statusWarning || empty || _statusSource == "titles.done") ? Visibility.Visible : Visibility.Collapsed; _drivePanel.Visibility = Visibility.Visible; _drivePanel.HorizontalAlignment = empty ? HorizontalAlignment.Right : HorizontalAlignment.Stretch; _drivePanel.Width = empty ? 22 : double.NaN; @@ -968,11 +1000,12 @@ private void UpdateControls() _automaticSwitch.ToolTip = automaticSwitchLabel + "\n" + AccountAutomation.NextLabel(_service, DateTimeOffset.UtcNow) + "\n" + Loc.F("auto.help", _service.Vault.Data.AutomaticSwitchThreshold) + "\n" + Loc.T(_service.Vault.Data.AutomaticSwitchUsePriority ? "auto.custom" : "auto.optimal") + "\n" + Loc.T("auto.priority_help"); _automaticSwitch.IsEnabled = _ready && !_closing; + _stampTitles.IsEnabled = _ready && !_closing && !_switching && !_loggingIn && !_providerEditing && !_stampingTitles && PendingSwitch() == null; if (_homeActions.ContextMenu is { IsOpen: true }) foreach (var next in _automaticSwitch.Items.OfType().Where(item => item.Name == "NextAutomaticAccount")) next.Header = AccountAutomation.NextLabel(_service, DateTimeOffset.UtcNow); - _refresh.IsEnabled = CanRefresh; _add.IsEnabled = _ready && !_loggingIn && !_closing; - if (_importCurrent != null) _importCurrent.IsEnabled = !_closing; + _refresh.IsEnabled = CanRefresh; _add.IsEnabled = _ready && !_loggingIn && !_closing && !_stampingTitles; + if (_importCurrent != null) _importCurrent.IsEnabled = !_closing && !_stampingTitles; _loginPanel.Visibility = _loggingIn ? Visibility.Visible : Visibility.Collapsed; _cancelSwitch.Visibility = Visibility.Collapsed; } @@ -1021,6 +1054,7 @@ internal async Task ExitAsync() await Task.WhenAll(_operations.ToArray()); WriteRunningState("stopping-drives"); await _drives.DisposeAsync(); WriteRunningState("stopping-hardware"); await _hardware.DisposeAsync(); + WriteRunningState("stopping-stream-recovery"); await _streamRecovery.DisposeAsync(); WriteRunningState("stopping-sessions"); await _sessions.DisposeAsync(); WriteRunningState("stopping-service"); await _service.StopAsync(); _exit = true; _tray?.Dispose(); WriteRunningState("stopped"); Application.Current.Shutdown(); diff --git a/src/windows/SessionUsagePanel.cs b/src/windows/SessionUsagePanel.cs index 058a301..12fe040 100644 --- a/src/windows/SessionUsagePanel.cs +++ b/src/windows/SessionUsagePanel.cs @@ -7,16 +7,16 @@ namespace CodexAccountDesk; internal sealed class SessionUsagePanel : Grid { - private readonly TextBlock[] _labels = new TextBlock[3], _values = new TextBlock[3]; - private readonly StackPanel[] _groups = new StackPanel[3]; + private readonly TextBlock[] _labels = new TextBlock[4], _values = new TextBlock[4]; + private readonly StackPanel[] _groups = new StackPanel[4]; public string Text => string.Join(" ", _labels.Select((label, index) => label.Text + " " + _values[index].Text)); public SessionUsagePanel() { Height = 20; HorizontalAlignment = HorizontalAlignment.Center; VerticalAlignment = VerticalAlignment.Center; - for (var index = 0; index < 3; index++) + for (var index = 0; index < _groups.Length; index++) { ColumnDefinitions.Add(new() { Width = GridLength.Auto }); - _groups[index] = new() { Orientation = Orientation.Horizontal, Margin = new Thickness(0, 0, index < 2 ? 16 : 0, 0) }; + _groups[index] = new() { Orientation = Orientation.Horizontal, Margin = new Thickness(0, 0, index < _groups.Length - 1 ? 12 : 0, 0) }; _labels[index] = new() { FontSize = 11, LineHeight = 14, LineStackingStrategy = LineStackingStrategy.BlockLineHeight, VerticalAlignment = VerticalAlignment.Center }; _values[index] = new() { FontSize = 14, LineHeight = 20, LineStackingStrategy = LineStackingStrategy.BlockLineHeight, FontWeight = FontWeights.SemiBold, VerticalAlignment = VerticalAlignment.Center, Margin = new Thickness(6, 0, 0, 0) }; @@ -26,14 +26,15 @@ public SessionUsagePanel() } public void Refresh(SessionActivity activity, Burn0Theme theme) { - var labels = new[] { "실행", "큐", "목표" }; - var values = new int?[] { activity.Available ? activity.Running : null, activity.Available ? activity.Queued : null, activity.Available ? activity.Goals : null }; - for (var index = 0; index < 3; index++) + var labels = new[] { "실행", "큐", "목표", "sessions.errors" }; + var values = new int?[] { activity.Available ? activity.Running : null, activity.Available ? activity.Queued : null, activity.Available ? activity.Goals : null, activity.Available ? activity.Errors : null }; + for (var index = 0; index < _groups.Length; index++) { _labels[index].Text = Loc.T(labels[index]); _labels[index].FontFamily = theme.Font; _labels[index].Foreground = theme.Color("muted"); _values[index].Text = values[index]?.ToString(Loc.Culture) ?? "—"; _values[index].FontFamily = theme.Font; - _values[index].Foreground = theme.Color(values[index].HasValue ? "text" : "muted"); + _values[index].Foreground = theme.Color(index == 3 && values[index] > 0 ? "warning" : values[index].HasValue ? "text" : "muted"); } + _groups[3].ToolTip = Loc.T("sessions.errors_help"); } public bool Fits() => _groups.Select((group, index) => _labels[index].DesiredSize.Width + _values[index].DesiredSize.Width <= group.ActualWidth + 0.5).All(fits => fits) && _labels.Concat(_values).All(element => element.ActualHeight <= 20.5); diff --git a/tests/account-automation/Program.cs b/tests/account-automation/Program.cs index 0213bb8..149618a 100644 --- a/tests/account-automation/Program.cs +++ b/tests/account-automation/Program.cs @@ -5,6 +5,7 @@ var checks = new List(); void Check(string name, bool condition) { if (!condition) throw new Exception(name); checks.Add(name); } var now = DateTimeOffset.UtcNow; +await StreamRecoveryContracts.Run(Check); CreditHistoryContracts.Run(Check); HistoryRangeContracts.Run(Check); CreditOverlayContracts.Run(Check); @@ -29,6 +30,21 @@ completed.AccountApplied = true; completed.CreditProtectionPause = true; Check("Credit protection remains blocking", vault.ActivePendingSwitch == completed); } +{ + var backend = new FakeBackend(); var saved = 0; + var count = await ActiveChatTitleUpdate.RunAsync(backend, DateTimeOffset.UtcNow, value => saved = value, CancellationToken.None); + Check("Manual title update only captures and verifies names", count == 1 && saved == 1 && backend.Names == 1 && backend.Closes == 0 && backend.Applies == 0 && backend.Resumes == 0); + backend = new FakeBackend { NameFailures = 1 }; saved = 0; var failed = false; + try { await ActiveChatTitleUpdate.RunAsync(backend, DateTimeOffset.UtcNow, value => saved = value, CancellationToken.None); } + catch (IOException) { failed = true; } + Check("Manual title update reports a naming failure without claiming success", failed && saved == 0 && backend.Closes == 0 && backend.Applies == 0); + backend = new FakeBackend { CapturedChats = new[] { new SwitchChat { Id = "one" }, new SwitchChat { Id = "two" } }, FailNameAt = 2 }; saved = 0; failed = false; + try { await ActiveChatTitleUpdate.RunAsync(backend, DateTimeOffset.UtcNow, value => saved = value, CancellationToken.None); } + catch (IOException) { failed = true; } + Check("Partial title update reports only names verified before failure", failed && saved == 1 && backend.Names == 2); + var at = DateTimeOffset.Parse("2026-10-31T15:00:00Z"); + Check("Date prefix is replaced at the KST day boundary", SwitchTitle.Stamp("[261031] 채팅", at) == "[261101] 채팅"); +} JsonElement Hello(string version, string home = "/fictional/home", string os = "linux", string family = "unix", string prefix = "codex_account_desk/") => JsonSerializer.SerializeToElement(new { codexHome = home, platformOs = os, platformFamily = family, userAgent = prefix + version + " (fixture)" }); foreach (var version in new[] { "0.162.0-alpha.17.2", "0.162.1", "0.163.0", "0.164.0" }) @@ -351,13 +367,15 @@ sealed class FakeBackend : IAccountSwitchBackend public CancellationTokenSource? Cancel; public int Launches, Applies, Resumes, Closes, Names, LaunchFailures, ResumeFailures, ResumeCalls, CloseFailures, NameFailures, ApplyFailures, ApplyCalls; public Exception? LaunchError; + public SwitchChat[]? CapturedChats; + public int FailNameAt; public bool AmbiguousContinue; public string? LastMarker; public Task CurrentAccountAsync(CancellationToken cancellation) => Task.FromResult(Current); - public Task CaptureAsync(CancellationToken cancellation) => Task.FromResult(new[] { new SwitchChat { Id = "chat", OriginalTitle = "Fictional work" } }); + public Task CaptureAsync(CancellationToken cancellation) => Task.FromResult(CapturedChats ?? new[] { new SwitchChat { Id = "chat", OriginalTitle = "Fictional work" } }); public int TargetValidations; public Task ValidateChatsForTargetAsync(SwitchChat[] chats, string target, CancellationToken cancellation) { TargetValidations++; if (Failure == "target-refused") throw new InvalidDataException("Synthetic workspace refusal"); return Task.CompletedTask; } - public Task BackupNameAsync(SwitchChat chat, DateTimeOffset at, CancellationToken cancellation) { Names++; if (Names <= NameFailures) throw new IOException("Synthetic name failure"); return Task.CompletedTask; } + public Task BackupNameAsync(SwitchChat chat, DateTimeOffset at, CancellationToken cancellation) { Names++; if (Names <= NameFailures || Names == FailNameAt) throw new IOException("Synthetic name failure"); return Task.CompletedTask; } public Task CloseDesktopAsync(CancellationToken cancellation) { Closes++; if (Failure == "close" || Closes <= CloseFailures) throw new IOException("Synthetic shutdown remains incomplete"); if (Failure == "cancel-after-close") Cancel!.Cancel(); return Task.CompletedTask; } public Task ApplyAccountAsync(string key, string? source, CancellationToken cancellation) { ApplyCalls++; if (Failure == "apply" || ApplyCalls <= ApplyFailures) throw new IOException("Synthetic apply failure"); Current = key; Applies++; if (Failure == "apply-after-write" && ApplyCalls == 1) throw new IOException("Synthetic failure after writing credentials"); return Task.CompletedTask; } public Task LaunchDesktopAsync(CancellationToken cancellation) { Launches++; if (Failure == "launch-always" || Launches <= LaunchFailures) throw LaunchError ?? new IOException("Synthetic launch failure"); return Task.CompletedTask; } diff --git a/tests/account-automation/StreamRecoveryContracts.cs b/tests/account-automation/StreamRecoveryContracts.cs new file mode 100644 index 0000000..3c25b12 --- /dev/null +++ b/tests/account-automation/StreamRecoveryContracts.cs @@ -0,0 +1,103 @@ +using CodexAccountDesk; +using System.Text; +using System.Text.Json; + +internal static class StreamRecoveryContracts +{ + internal static async Task Run(Action check) + { + const string failure = "stream disconnected before completion: An error occurred while processing your request. Please include the request ID fixture."; + foreach (var status in new[] { "active", "idle", "completed", "interrupted", "inProgress", "unknown" }) + check("Only a persisted failed terminal status permits recovery: " + status, !StreamRecoveryPolicy.Failed(status)); + var routes = new ModelProviderSettings(); + check("First-request failure does not require fabricated successful-route provenance", StreamRecoveryPolicy.RegisteredModel("fixture-openai", routes) && routes.ConversationRoutes.Count == 0); + check("A removed custom model cannot be continued to a fallback provider", !StreamRecoveryPolicy.RegisteredModel("cs_removed_model", routes)); + var id = Guid.NewGuid().ToString(); + string Complete(string? error) => JsonSerializer.Serialize(new { type = "event_msg", timestamp = DateTimeOffset.UtcNow, + payload = new { type = "task_complete", turn_id = id, error = error == null ? null : new { message = error } } }) + "\n"; + var tail = DesktopTurnOutcomes.ParseTail(Encoding.UTF8.GetBytes(Complete(failure)), false, "fixture-model", "codex_switch_router"); + check("Actual rollout completion parser retains failed turn identity and model", tail?.Id == id && tail.Status == "failed" && tail.Error == failure && tail.Model == "fixture-model"); + check("A persisted successful completion has no stream error", DesktopTurnOutcomes.ParseTail(Encoding.UTF8.GetBytes(Complete(null)), false, "m", "p")?.Status == "completed"); + string Terminal(string status, object? error) => JsonSerializer.Serialize(new { type = "event_msg", timestamp = DateTimeOffset.UtcNow, + payload = new { type = "task_complete", turn_id = id, status, error } }) + "\n"; + check("Failure without error prose remains failed", DesktopTurnOutcomes.ParseTail(Encoding.UTF8.GetBytes(Terminal("failed", null)), false, "m", "p")?.Status == "failed"); + check("An error object without a message remains failed", DesktopTurnOutcomes.ParseTail(Encoding.UTF8.GetBytes(Terminal("failed", new { code = "server_error" })), false, "m", "p")?.Status == "failed"); + check("Actual interrupted status wins over attached error prose", DesktopTurnOutcomes.ParseTail(Encoding.UTF8.GetBytes(Terminal("interrupted", new { message = failure })), false, "m", "p")?.Status == "interrupted"); + check("Unknown terminal status cannot imply completion", DesktopTurnOutcomes.ParseTail(Encoding.UTF8.GetBytes(Terminal("unknown", null)), false, "m", "p") == null); + var next = JsonSerializer.Serialize(new { type = "event_msg", payload = new { type = "task_started", turn_id = Guid.NewGuid().ToString() } }) + "\n"; + check("A newer started turn invalidates historical failure before any continuation", DesktopTurnOutcomes.ParseTail(Encoding.UTF8.GetBytes(Complete(failure) + next), false, "m", "p") == null); + check("A newer user abort invalidates historical failure", DesktopTurnOutcomes.ParseTail(Encoding.UTF8.GetBytes(Complete(failure) + "{\"type\":\"event_msg\",\"payload\":{\"type\":\"turn_aborted\"}}\n"), false, "m", "p") == null); + check("Incomplete appended JSON is not interpreted as another failure", DesktopTurnOutcomes.ParseTail(Encoding.UTF8.GetBytes("{\"type\":\"event_msg\""), false, "m", "p") == null); + check("Tail starts in the middle of an image/tool line without parsing that item", DesktopTurnOutcomes.ParseTail(Encoding.UTF8.GetBytes("partial-non-json\n" + Complete(failure)), true, "m", "p")?.Id == id); + + StreamRecoveryObservation Observation(string? turn=null) => new(Guid.NewGuid().ToString(), "owner", turn ?? Guid.NewGuid().ToString(), "failed", failure, + "fixture-model", "codex_switch_router", "identity-hash", "route-hash", DateTimeOffset.UtcNow, true); + foreach (var error in new string?[] { failure, "unexpected status 401 Unauthorized", "HTTP 403", "HTTP 413 Payload Too Large", + "HTTP 429", "insufficient quota", "context limit", "provider-encrypted compaction", "HTTP 507", "switch_upstream_recovery_stopped", + "stream disconnected before completion: The selected upstream did not return the requested event stream after HTTP 507 recovery.", + "A bounded JSON Responses request is required.", "upstream interrupted its response", "connection closed", null }) + { + var candidate = Observation() with { Error = error }; var data = new Dictionary(); var sent = 0; + var recovery = new StreamRecoveryCoordinator(data, () => {}, (_,_) => Task.CompletedTask); + var decision = await recovery.CheckAsync(candidate, _ => Task.FromResult(candidate), (_,_) => { sent++; return Task.FromResult(Guid.NewGuid().ToString()); }, CancellationToken.None); + check("Every confirmed error family gets one identified continuation: " + (error ?? "missing error message"), sent == 1 && decision.Reason == "continued" && decision.Attempts == 1); + } + var receipts = new Dictionary(); var saved = 0; var sends = 0; var delays = new List(); + var coordinator = new StreamRecoveryCoordinator(receipts, () => saved++, (attempt, _) => { delays.Add(attempt); return Task.CompletedTask; }); + var observed = Observation(); + for (var attempt = 1; attempt <= 4; attempt++) + { + var oldSends = sends; var nextTurn = Guid.NewGuid().ToString(); + await coordinator.CheckAsync(observed, _ => Task.FromResult(observed), (message, _) => + { + check("Receipt persists before send with a stable message identity: " + attempt, saved > 0 && receipts[observed.Thread].Sending && receipts[observed.Thread].Message == message && receipts[observed.Thread].Attempts == attempt); + sends++; return Task.FromResult(nextTurn); + }, CancellationToken.None); + check("Consecutive failed continuations share one three-attempt budget: " + attempt, sends == oldSends + (attempt <= 3 ? 1 : 0)); + if (attempt <= 3) observed = observed with { Turn = nextTurn }; + } + check("Automatic continuation budget is exactly three and backoff is bounded", sends == 3 && delays.SequenceEqual(new[] {1,2,3})); + var reloaded = JsonSerializer.Deserialize>(JsonSerializer.Serialize(receipts))!; + coordinator = new StreamRecoveryCoordinator(reloaded, () => saved++, (_,_) => Task.CompletedTask); + await coordinator.CheckAsync(observed, _ => Task.FromResult(observed), (_,_) => { sends++; return Task.FromResult(Guid.NewGuid().ToString()); }, CancellationToken.None); + check("Restart cannot reset an exhausted stream-recovery budget", sends == 3); + await coordinator.CheckAsync(observed with { Status = "completed", Error = null }, _ => throw new Exception(), (_,_) => throw new Exception(), CancellationToken.None); + observed = observed with { Turn = Guid.NewGuid().ToString() }; + await coordinator.CheckAsync(observed, _ => Task.FromResult(observed), (_,_) => { sends++; return Task.FromResult(Guid.NewGuid().ToString()); }, CancellationToken.None); + check("Confirmed success permits a new independent failure budget", sends == 4 && reloaded[observed.Thread].Attempts == 1); + foreach (var change in new Func[] { + o=>o with { Turn = Guid.NewGuid().ToString() }, o=>o with { Status="active" }, o=>o with { Ready=false }, + o=>o with { Owner="another" }, o=>o with { Model="another" }, o=>o with { Provider="another" }, + o=>o with { Identity="another" }, o=>o with { Route="another" }, o=>o with { Status="interrupted" }, o=>o with { Status="completed" } }) + { + var data = new Dictionary(); var candidate = Observation(); var called = false; + var recovery = new StreamRecoveryCoordinator(data, ()=>{}, (_,_)=>Task.CompletedTask); + await recovery.CheckAsync(candidate, _=>Task.FromResult(change(candidate)), (_,_)=>{called=true;return Task.FromResult(Guid.NewGuid().ToString());},CancellationToken.None); + check("Changed owner/turn/model/account/queue/runtime state prevents automatic send", !called && data.Count == 0); + } + { + var data = new Dictionary(); var candidate = Observation(); var count = 0; + var recovery = new StreamRecoveryCoordinator(data, ()=>{}, (_,_)=>Task.CompletedTask); + try { await recovery.CheckAsync(candidate,_=>Task.FromResult(candidate),(_,_)=>{count++;throw new IOException("Lost IPC response");},CancellationToken.None); } + catch(IOException) { } + var loaded = JsonSerializer.Deserialize>(JsonSerializer.Serialize(data))!; + recovery = new StreamRecoveryCoordinator(loaded,()=>{},(_,_)=>Task.CompletedTask); + await recovery.CheckAsync(candidate,_=>Task.FromResult(candidate),(_,_)=>{count++;return Task.FromResult(Guid.NewGuid().ToString());},CancellationToken.None); + check("Ambiguous continuation is persisted and never resent on restart", count==1 && loaded[candidate.Thread].Sending); + } + { + var data = new Dictionary(); var candidate=Observation();var called=false; + var recovery = new StreamRecoveryCoordinator(data,()=>throw new IOException("Disk write failed"),(_,_)=>Task.CompletedTask); + try { await recovery.CheckAsync(candidate,_=>Task.FromResult(candidate),(_,_)=>{called=true;return Task.FromResult(Guid.NewGuid().ToString());},CancellationToken.None); } catch(IOException) { } + check("Persistence failure prevents continuation submission",!called); + using var stop=new CancellationTokenSource();stop.Cancel(); + recovery=new StreamRecoveryCoordinator(new(),()=>{},(_,token)=>Task.Delay(100,token)); + try { await recovery.CheckAsync(candidate,_=>Task.FromResult(candidate),(_,_)=>{called=true;return Task.FromResult(Guid.NewGuid().ToString());},stop.Token); }catch(OperationCanceledException) { } + check("Cancellation during backoff never submits a continuation",!called); + } + var parameters=JsonSerializer.SerializeToElement(DesktopStreamRecoveryMonitor.StartParameters("fixture-thread","fixture-message")); + check("Automatic continuation inherits existing settings and uses one identified Continue",parameters.GetProperty("turnStart").GetProperty("context").GetProperty("inheritThreadSettings").GetBoolean() && + parameters.GetProperty("turnStart").GetProperty("request").GetProperty("clientUserMessageId").GetString()=="fixture-message" && + parameters.GetProperty("turnStart").GetProperty("request").GetProperty("input")[0].GetProperty("text").GetString()=="Continue"); + } +} diff --git a/tests/model-providers-ui/Program.cs b/tests/model-providers-ui/Program.cs index ddcbf9f..c6f5390 100644 --- a/tests/model-providers-ui/Program.cs +++ b/tests/model-providers-ui/Program.cs @@ -142,6 +142,7 @@ private static int Main(string[] args) checks.Add(language + ": duplicate restart action disabled while busy"); } HomeLayoutContracts.Verify(service, args[0], checks); + SessionCounterContracts.Verify(args[0], checks); File.WriteAllText(Path.Combine(args[0], "ui-verification.json"), JsonSerializer.Serialize(new { checks, diff --git a/tests/model-providers-ui/SessionCounterContracts.cs b/tests/model-providers-ui/SessionCounterContracts.cs new file mode 100644 index 0000000..64b7a49 --- /dev/null +++ b/tests/model-providers-ui/SessionCounterContracts.cs @@ -0,0 +1,37 @@ +using CodexAccountDesk; +using System.Globalization; +using System.IO; +using System.Windows; +using System.Windows.Controls; +using System.Windows.Media; +using System.Windows.Media.Imaging; + +internal static class SessionCounterContracts +{ + internal static void Verify(string output, List checks) + { + foreach (var language in new[] { "en", "ko", "ja", "zh-CN", "zh-TW", "es", "fr", "de", "pt-BR" }) + foreach (var dark in new[] { true, false }) + foreach (var count in new int?[] { null, 0, 3 }) + { + Loc.SetLanguage(language); var theme = new Burn0Theme(dark); + var panel = new SessionUsagePanel(); + panel.Refresh(new SessionActivity(true, 3, 0, 6, Queued: 0, Goals: null, Errors: count), theme); + panel.Measure(new Size(476, 30)); panel.Arrange(new Rect(0, 0, panel.DesiredSize.Width, panel.DesiredSize.Height)); panel.UpdateLayout(); + if (panel.Children.Count != 4 || !panel.Fits() || panel.ActualWidth > 476 || panel.ActualHeight > 21) throw new Exception("Four session counters must fit the minimum window."); + var cell = (StackPanel)panel.Children[3]; var value = (TextBlock)cell.Children[1]; var label = (TextBlock)cell.Children[0]; + if (label.Text != Loc.T("sessions.errors") || value.Text != (count?.ToString(Loc.Culture) ?? "—")) throw new Exception("Error count/unknown translation is incorrect."); + var expected = ((SolidColorBrush)theme.Color(count > 0 ? "warning" : count.HasValue ? "text" : "muted")).Color; + if (((SolidColorBrush)value.Foreground).Color != expected || cell.ToolTip == null) throw new Exception("Errors must use warning color and explain their scope."); + if (count == 3 && language is "en" or "ko") + { + var background = new Border { Background = theme.Color("canvas"), Padding = new Thickness(12, 8, 12, 8), Child = panel }; + background.Measure(new Size(500, 40)); background.Arrange(new Rect(0, 0, 500, 40)); background.UpdateLayout(); + var bitmap = new RenderTargetBitmap(500, 40, 96, 96, PixelFormats.Pbgra32); bitmap.Render(background); + var encoder = new PngBitmapEncoder(); encoder.Frames.Add(BitmapFrame.Create(bitmap)); + using var file = File.Create(Path.Combine(output, "session-errors-" + language + "-" + (dark ? "dark" : "light") + ".png")); encoder.Save(file); + } + checks.Add(language + "/" + (dark ? "dark" : "light") + "/errors " + (count?.ToString(CultureInfo.InvariantCulture) ?? "unknown") + ": current count, warning color and four-counter bounds"); + } + } +} diff --git a/tests/model-providers/GatewayRetryContracts.cs b/tests/model-providers/GatewayRetryContracts.cs new file mode 100644 index 0000000..b74ac71 --- /dev/null +++ b/tests/model-providers/GatewayRetryContracts.cs @@ -0,0 +1,231 @@ +using CodexAccountDesk; +using System.Collections.Concurrent; +using System.Net; +using System.Net.Http.Headers; +using System.Net.Sockets; +using System.Text; +using System.Text.Json; +using System.Text.Json.Nodes; + +internal static class GatewayRetryContracts +{ + internal static async Task Run(string root, Action check) + { + check("507 recovery has exactly three increasing cancellable delays", GatewayUpstreamRetry.MaximumRetries == 3 && + Enumerable.Range(1, 3).Select(GatewayUpstreamRetry.Delay).SequenceEqual(new[] { TimeSpan.FromSeconds(2), TimeSpan.FromSeconds(5), TimeSpan.FromSeconds(10) })); + foreach (var failures in new[] { 0, 1, 2, 3, 4 }) + { + using var handler = new RetryUpstream { Failures = failures }; + using var http = new HttpClient(handler); var delays = new List(); + using var result = await GatewayUpstreamRetry.SendAsync(http, () => RetryUpstream.Request("https://chatgpt.com/backend-api/codex/responses"), true, + (retry, delay, token) => { delays.Add(retry); return Task.CompletedTask; }, CancellationToken.None); + check("Explicit 507 sequence is bounded and stops on success: " + failures, handler.Requests.Count == Math.Min(failures + 1, 4) && + delays.SequenceEqual(Enumerable.Range(1, Math.Min(failures, 3))) && (int)result.StatusCode == (failures < 4 ? 200 : 507)); + check("Rejected responses are disposed before another attempt: " + failures, handler.Errors.All(content => content.Disposed) || failures == 4 && handler.Errors.Count(content => content.Disposed) == 3); + } + foreach (var status in new[] { 400, 401, 403, 413, 429, 500, 502, 503 }) + { + using var handler = new RetryUpstream { OtherStatus = status }; + using var http = new HttpClient(handler); + using var result = await GatewayUpstreamRetry.SendAsync(http, () => RetryUpstream.Request("https://chatgpt.com/backend-api/codex/responses"), true, + (_, _, _) => throw new Exception("Unexpected retry"), CancellationToken.None); + check("507 recovery never expands to another error code: " + status, (int)result.StatusCode == status && handler.Requests.Count == 1); + } + using (var handler = new RetryUpstream { ThrowTransport = true }) + using (var http = new HttpClient(handler)) + { + var failed = false; + try { using var result = await GatewayUpstreamRetry.SendAsync(http, () => RetryUpstream.Request("https://chatgpt.com/backend-api/codex/responses"), true, + (_, _, _) => throw new Exception("Unexpected retry"), CancellationToken.None); } + catch (HttpRequestException) { failed = true; } + check("An ambiguous transport failure is never retried", failed && handler.Requests.Count == 1); + } + using (var handler = new RetryUpstream { Failures = 4 }) + using (var http = new HttpClient(handler)) + using (var stop = new CancellationTokenSource()) + { + var cancelled = false; + try { using var result = await GatewayUpstreamRetry.SendAsync(http, () => RetryUpstream.Request("https://chatgpt.com/backend-api/codex/responses"), true, + async (_, _, token) => { stop.Cancel(); await Task.Delay(TimeSpan.FromHours(1), token); }, stop.Token); } + catch (OperationCanceledException) { cancelled = true; } + check("Cancellation during backoff cannot submit another inference", cancelled && handler.Requests.Count == 1); + } + + // Full production listener: retain all bytes, selected destination and original + // auth; comments are keep-alives rather than assistant or tool events. + foreach (var custom in new[] { false, true }) + { + using var handler = new RetryUpstream { Failures = 3 }; + await using var fixture = new Fixture(root, handler, custom); + var beforeRoutes = JsonSerializer.Serialize(fixture.Store.Read().ConversationRoutes); + using var response = await fixture.Send(); + var body = await response.Content.ReadAsStringAsync(); + var requests = handler.Requests.ToArray(); + check("Fourth attempt completes through native gateway after three 507s: " + custom, response.IsSuccessStatusCode && body.Contains("response.completed") && !body.Contains("response.failed") && requests.Length == 4); + check("Every retry preserves the complete body and history: " + custom, requests.Select(item => item.Body).Distinct().Count() == 1 && requests.All(item => item.Body.Contains("fixture-call") && item.Body.Contains("original-tool-result") && item.Body.Contains("unchanged image"))); + check("Every retry preserves original account, model and destination: " + custom, requests.All(item => item.Host == (custom ? "retry.example.invalid" : "chatgpt.com") && item.Token == (custom ? "fictional-custom-retry-key" : "fictional-openai-retry-token") && item.Account == (custom ? null : "fictional-retry-account") && item.Session == (custom ? null : "retry-conversation"))); + check("HTTP 507 retries expose no upstream error body or extra response events: " + custom, !body.Contains("fictional-secret-in-error") && body.Split("response.completed").Length == 2 && body.Contains("recovery 3/3")); + check("Only the recovered success commits conversation provenance: " + custom, JsonSerializer.Serialize(fixture.Store.Read().ConversationRoutes) != beforeRoutes && fixture.Gateway.LastError == null); + } + using (var handler = new RetryUpstream { Failures = 10 }) + { + await using var fixture = new Fixture(root, handler); + using var response = await fixture.Send(); + var body = await response.Content.ReadAsStringAsync(); + check("Persistent HTTP 507 stops after the original plus three requests", handler.Requests.Count == 4 && body.Contains("response.failed") && body.Contains("3 of 3 retries") && !body.Contains("response.completed")); + check("Exhaustion never commits a route or leaks an upstream error", fixture.Store.Read().ConversationRoutes.Count == 0 && !body.Contains("fictional-secret-in-error") && fixture.Gateway.LastError!.Contains("507")); + } + using (var handler = new RetryUpstream { Failures = 1, OtherAfterFailures = 429 }) + { + await using var fixture = new Fixture(root, handler); + using var response = await fixture.Send(); var body = await response.Content.ReadAsStringAsync(); + check("A different failure after recovery is terminal without extra attempts", handler.Requests.Count == 2 && body.Contains("response.failed") && body.Contains("HTTP 429") && !body.Contains("fictional-secret-in-error")); + } + foreach (var path in new[] { "/responses/compact", "/images/generations", "/images/edits", "/responses?nonstream" }) + { + using var handler = new RetryUpstream { Failures = 10 }; + await using var fixture = new Fixture(root, handler); + using var response = await fixture.Send(path); + check("Compaction, image and nonstream requests retain single-send policy: " + path, (int)response.StatusCode == 507 && handler.Requests.Count == 1); + } + using (var handler = new RetryUpstream { Failures = 10 }) + { + await using var fixture = new Fixture(root, handler); + // A raw reset avoids HttpClient's background response draining. + var endpoint = new Uri(fixture.Gateway.BaseUrl); + using var caller = new TcpClient(); await caller.ConnectAsync(endpoint.Host, endpoint.Port); + caller.Client.LingerState = new LingerOption(true, 0); + var input = "{\"model\":\"fixture-openai\",\"stream\":true,\"input\":\"disconnect fixture\"}"; + var bytes = Encoding.UTF8.GetBytes(input); + var headers = Encoding.ASCII.GetBytes("POST " + endpoint.AbsolutePath + "/responses HTTP/1.1\r\nHost: " + endpoint.Authority + "\r\nAuthorization: Bearer fictional-openai-retry-token\r\nsession_id: retry-conversation\r\nversion: 0.163.0\r\nContent-Type: application/json\r\nContent-Length: " + bytes.Length + "\r\n\r\n"); + await caller.GetStream().WriteAsync(headers); await caller.GetStream().WriteAsync(bytes); + using var read = new StreamReader(caller.GetStream(), Encoding.UTF8, leaveOpen: true); + string? line; + do { line = await read.ReadLineAsync().WaitAsync(TimeSpan.FromSeconds(5)); } while (line != null && !line.Contains("Switch HTTP 507")); + check("The first 507 starts only a comment keep-alive", line != null && handler.Requests.Count == 1); + caller.Dispose(); + await Task.Delay(3000); + check("A disconnected caller cannot trigger another upstream attempt", handler.Requests.Count == 1); + } + using (var handler = new RetryUpstream { Failures = 10 }) + { + await using var fixture = new Fixture(root, handler); + using var response = await fixture.Send(); + fixture.SignedIn = false; + try { await response.Content.ReadAsStringAsync(); } + catch (HttpRequestException) { } + catch (IOException) { } + check("A sign-in change during backoff stops recovery on the original request", handler.Requests.Count == 1 && fixture.Store.Read().ConversationRoutes.Count == 0); + } + using (var handler = new RetryUpstream { BrokenStream = true }) + { + await using var fixture = new Fixture(root, handler); + using var response = await fixture.Send(); + var interrupted = false; + try { var body = await response.Content.ReadAsStringAsync(); interrupted = !body.Contains("response.completed"); } + catch (HttpRequestException) { interrupted = true; } + catch (IOException) { interrupted = true; } + check("Failure after successful response headers is never replayed", interrupted && handler.Requests.Count == 1); + } + using (var handler = new RetryUpstream { Failures = 10 }) + { + await using var fixture = new Fixture(root, handler); + using var response = await fixture.Send(); + fixture.GatewayDisposed = true; + await fixture.Gateway.DisposeAsync(); + await Task.Delay(250); + check("Gateway shutdown cancels pending backoff before another attempt", handler.Requests.Count == 1); + } + } + + private sealed class Fixture : IAsyncDisposable + { + internal readonly ModelProviderStore Store; + internal readonly ModelGateway Gateway; + private readonly HttpClient _upstream; + private readonly HttpClient _client = new() { Timeout = TimeSpan.FromSeconds(50) }; + private readonly string _model; + internal bool SignedIn = true; + internal bool GatewayDisposed; + internal Fixture(string root, RetryUpstream handler, bool custom = false) + { + Store = new ModelProviderStore(Path.Combine(root, "retry-" + Guid.NewGuid().ToString("N"))); + var model = new ProviderModel { UpstreamId = "fixture-openai" }; + var provider = new ModelProvider { Name = "Retry fixture", BaseUrl = "https://retry.example.invalid/v1", ApiKey = "fictional-custom-retry-key", Models = new() { model } }; + if (custom) Store.Update(next => next.Providers.Add(provider)); + _model = custom ? model.Alias(provider.Id) : model.UpstreamId; + _upstream = new HttpClient(handler) { Timeout = Timeout.InfiniteTimeSpan }; + Gateway = new ModelGateway(Store, (_, _) => SignedIn ? new GatewayIdentity("fictional-openai-retry-token", "fictional-retry-account", false) : null, _upstream); + Gateway.Start(); + _client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Bearer", "fictional-openai-retry-token"); + _client.DefaultRequestHeaders.Add("session_id", "retry-conversation"); + _client.DefaultRequestHeaders.Add("version", "0.163.0"); + } + internal async Task Send(string path = "/responses") + { + var nonstream = path.EndsWith("?nonstream", StringComparison.Ordinal); + var body = JsonSerializer.Serialize(new { model = _model, stream = !nonstream, input = new object[] { + new { type = "function_call", call_id = "fixture-call", name = "fixture_tool", arguments = "{}" }, + new { type = "function_call_output", call_id = "fixture-call", output = "original-tool-result" }, + new { role = "user", content = "unchanged image / Korean 원문" } } }); + using var request = new HttpRequestMessage(HttpMethod.Post, Gateway.BaseUrl + path.Split('?')[0]) { Content = new StringContent(body, Encoding.UTF8, "application/json") }; + return await _client.SendAsync(request, HttpCompletionOption.ResponseHeadersRead); + } + public async ValueTask DisposeAsync() { _client.Dispose(); if (!GatewayDisposed) await Gateway.DisposeAsync(); _upstream.Dispose(); } + } + private sealed record Captured(string Body, string Host, string? Token, string? Account, string? Session); + private sealed class RetryUpstream : HttpMessageHandler + { + internal int Failures; + internal int? OtherStatus; + internal int? OtherAfterFailures; + internal bool ThrowTransport; + internal bool BrokenStream; + internal readonly ConcurrentQueue Requests = new(); + internal readonly ConcurrentQueue Errors = new(); + internal static HttpRequestMessage Request(string url) => new(HttpMethod.Post, url) { Content = new StringContent("{\"model\":\"fixture-openai\",\"stream\":true}") }; + protected override async Task SendAsync(HttpRequestMessage request, CancellationToken cancellation) + { + if (request.Method == HttpMethod.Get) return new(HttpStatusCode.OK) { Content = new StringContent(new JsonObject { ["models"] = new JsonArray(ProviderCatalog.Generic("fixture-openai")) }.ToJsonString(), Encoding.UTF8, "application/json") }; + var body = await request.Content!.ReadAsStringAsync(cancellation); + Requests.Enqueue(new(body, request.RequestUri!.Host, request.Headers.Authorization?.Parameter, + request.Headers.TryGetValues("ChatGPT-Account-ID", out var account) ? account.Single() : null, + request.Headers.TryGetValues("session_id", out var session) ? session.Single() : null)); + if (ThrowTransport) throw new HttpRequestException("Owned ambiguous transport fixture"); + if (BrokenStream) return new(HttpStatusCode.OK) { Content = new StreamContent(new BrokenResponse()) { Headers = { ContentType = new MediaTypeHeaderValue("text/event-stream") } } }; + var code = OtherStatus ?? (Requests.Count <= Failures ? 507 : OtherAfterFailures ?? 200); + if (code != 200) + { + var content = new DisposedContent(); Errors.Enqueue(content); + return new((HttpStatusCode)code) { Content = content }; + } + return new(HttpStatusCode.OK) { Content = new StringContent("data: {\"type\":\"response.completed\",\"response\":{\"id\":\"recovered-fixture\",\"status\":\"completed\",\"output\":[]}}\n\n", Encoding.UTF8, "text/event-stream") }; + } + } + private sealed class DisposedContent : StringContent + { + internal bool Disposed; + internal DisposedContent() : base("fictional-secret-in-error") { } + protected override void Dispose(bool disposing) { Disposed = true; base.Dispose(disposing); } + } + private sealed class BrokenResponse : Stream + { + private bool _read; + public override bool CanRead => true; + public override bool CanSeek => false; + public override bool CanWrite => false; + public override long Length => throw new NotSupportedException(); + public override long Position { get => throw new NotSupportedException(); set => throw new NotSupportedException(); } + public override ValueTask ReadAsync(Memory buffer, CancellationToken cancellationToken = default) + { + if (_read) throw new IOException("Owned partial-stream interruption"); + _read = true; + var bytes = Encoding.UTF8.GetBytes("data: {\"type\":\"response.created\"}\n\n"); bytes.CopyTo(buffer); return ValueTask.FromResult(bytes.Length); + } + public override int Read(byte[] buffer, int offset, int count) => throw new NotSupportedException(); + public override long Seek(long offset, SeekOrigin origin) => throw new NotSupportedException(); + public override void SetLength(long value) => throw new NotSupportedException(); + public override void Write(byte[] buffer, int offset, int count) => throw new NotSupportedException(); + public override void Flush() { } + } +} diff --git a/tests/model-providers/Program.cs b/tests/model-providers/Program.cs index 696d5f9..a101464 100644 --- a/tests/model-providers/Program.cs +++ b/tests/model-providers/Program.cs @@ -40,6 +40,16 @@ async Task Refuses(string name, Func action) { try { await action(); } catch (Exception e) when (e is ArgumentException or InvalidOperationException or IOException or InvalidDataException) { Check(name, true); return; } throw new Exception(name); } var root = Path.Combine(Environment.GetFolderPath(Environment.SpecialFolder.LocalApplicationData), "Temp", "CodexSwitchModelProviders", Guid.NewGuid().ToString("N")); Directory.CreateDirectory(root); +if (args.Contains("--retry-only")) +{ + try + { + await GatewayRetryContracts.Run(root, Check); + Console.WriteLine(JsonSerializer.Serialize(new { passed = checks.Count, checks, scope = "Isolated HTTP 507 recovery; no external inference or owner home." })); + } + finally { Directory.Delete(root, true); } + return; +} if (args.Contains("--stress")) { try @@ -332,6 +342,7 @@ async Task Send(string model, string session, JsonArray? in try { await saving; } catch (OperationCanceledException) { cancelled = true; } Check("Shutdown cancels and drains discovery before disposing state without saving a partial key", cancelled && !File.Exists(Path.Combine(cancelPath, "model-providers.vault"))); await GatewayLoadContracts.Run(root, Check, args.Contains("--stress")); + await GatewayRetryContracts.Run(root, Check); Console.WriteLine(JsonSerializer.Serialize(new { passed = checks.Count, checks, nativeClient = args.Contains("--codex"), framework = System.Runtime.InteropServices.RuntimeInformation.FrameworkDescription, scope = "Disposable encrypted storage, loopback gateway and fictional upstreams; no owner home, account or desktop." }, new JsonSerializerOptions { WriteIndented = true })); } finally @@ -349,6 +360,8 @@ internal sealed class Upstreams : HttpMessageHandler internal bool BlockDiscovery; internal bool MatchingReasoning; internal bool ImageFailure; + internal int StorageFailures; + internal int StreamFailures; internal TaskCompletionSource DiscoveryEntered { get; } = new(TaskCreationOptions.RunContinuationsAsynchronously); internal GatedStream? Slow; internal string? BlockedKey; @@ -382,6 +395,12 @@ protected override async Task SendAsync(HttpRequestMessage if (BlockNext) { BlockNext = false; BlockedKey = request.Headers.Authorization?.Parameter; Entered.TrySetResult(); await Release.Task.WaitAsync(cancellation); } if (FailNext) { FailNext = false; return new(HttpStatusCode.TooManyRequests) { Content = new StringContent("fictional-key-a-new") }; } + if (StorageFailures > 0) { StorageFailures--; return new((HttpStatusCode)507) { Content = new StringContent("fictional-key-must-not-be-echoed") }; } + if (StreamFailures > 0) + { + StreamFailures--; + return new(HttpStatusCode.OK) { Content = new StringContent("data: {\"type\":\"response.failed\",\"response\":{\"id\":\"fixture-stream-error\",\"status\":\"failed\",\"error\":{\"code\":\"fixture_stream_failure\",\"message\":\"An error occurred while processing your request. Request ID fixture-stream.\"}}}\n\n", Encoding.UTF8, "text/event-stream") }; + } if (StreamNext) { StreamNext = false; var content = new StreamContent(Slow!); content.Headers.ContentType = new MediaTypeHeaderValue("text/event-stream"); @@ -419,7 +438,7 @@ internal static async Task Run(string executable, string home, ModelProviderMana await using var rpc = new CodexRpc(executable, home, preserveProvider: true); var diagnostics = new ConcurrentQueue(); rpc.DiagnosticMessage += line => { if (line.Contains("models", StringComparison.OrdinalIgnoreCase) && diagnostics.Count < 20) diagnostics.Enqueue(line); }; - using var timeout = new CancellationTokenSource(TimeSpan.FromSeconds(90)); var cancellation = timeout.Token; + using var timeout = new CancellationTokenSource(TimeSpan.FromSeconds(150)); var cancellation = timeout.Token; await rpc.InitializeAsync(cancellation); var listed = await rpc.RequestAsync("model/list", new { includeHidden = true }, cancellation); var models = listed.GetProperty("data").EnumerateArray().Select(m => m.GetProperty("model").GetString()).ToArray(); @@ -435,7 +454,7 @@ internal static async Task Run(string executable, string home, ModelProviderMana check("Actual Codex retains the original OpenAI display name", listed.GetProperty("data").EnumerateArray().Single(m => m.GetProperty("model").GetString() == "fixture-openai").GetProperty("displayName").GetString() == "fixture-openai"); var thread = await rpc.RequestAsync("thread/start", new { model = "fixture-openai", cwd = home, ephemeral = true, approvalPolicy = "never", sandbox = "read-only", config = new Dictionary { ["mcp_servers"] = new Dictionary() } }, cancellation); var threadId = thread.GetProperty("thread").GetProperty("id").GetString()!; - async Task Turn(string target, string? model, string name, string? effort = null) + async Task Turn(string target, string? model, string name, string? effort = null, bool expectedFailure = false) { var complete = new TaskCompletionSource(TaskCreationOptions.RunContinuationsAsynchronously); void Notification(string method, JsonElement value) { if (method == "turn/completed" && value.GetProperty("threadId").GetString() == target) complete.TrySetResult(value.Clone()); } @@ -444,8 +463,10 @@ async Task Turn(string target, string? model, string name, string? effort = null { await rpc.RequestAsync("turn/start", new { threadId = target, model, effort, input = new[] { new { type = "text", text = "Reply with fixture-ok." } } }, cancellation); var result = await complete.Task.WaitAsync(cancellation); - if (result.GetProperty("turn").GetProperty("status").GetString() != "completed") Console.WriteLine(JsonSerializer.Serialize(new { nativeFixtureTurn = result, gatewayError = manager.LastError, lastRoute = upstreams.Requests.Last().Host })); - check(name, result.GetProperty("turn").GetProperty("status").GetString() == "completed"); + var expected = expectedFailure ? "failed" : "completed"; + if (result.GetProperty("turn").GetProperty("status").GetString() != expected) Console.WriteLine(JsonSerializer.Serialize(new { nativeFixtureTurn = result, gatewayError = manager.LastError, lastRoute = upstreams.Requests.Last().Host })); + check(name, result.GetProperty("turn").GetProperty("status").GetString() == expected); + return result; } finally { rpc.Notification -= Notification; } } @@ -463,6 +484,17 @@ async Task Turn(string target, string? model, string name, string? effort = null check("Upstream response model names cannot redirect the next turn to OpenAI", upstreams.Requests.Last().Host == "a.example.invalid"); await Turn(threadId, "fixture-openai", "Changing another chat leaves the original chat on OpenAI"); check("Actual native follow-up requests reach all three upstream routes", upstreams.Requests.Any(r => r.Method == "POST" && r.Host == "a.example.invalid") && upstreams.Requests.Any(r => r.Method == "POST" && r.Host == "b.example.invalid")); + upstreams.StorageFailures = 3; var beforeRecovery = upstreams.Requests.Count(r => r.Method == "POST"); + await Turn(threadId, "fixture-openai", "Actual Codex completes the same turn after three gateway HTTP 507 retries"); + check("Actual Codex does not create another turn or duplicate gateway requests after recovery", upstreams.Requests.Count(r => r.Method == "POST") - beforeRecovery == 4); + upstreams.StorageFailures = 10; var beforeExhaustion = upstreams.Requests.Count(r => r.Method == "POST"); + await Turn(threadId, "fixture-openai", "Actual Codex receives terminal failure after gateway recovery is exhausted", expectedFailure: true); + check("Actual Codex cannot multiply the exhausted three-retry budget", upstreams.Requests.Count(r => r.Method == "POST") - beforeExhaustion == 4); + upstreams.StorageFailures = 0; + upstreams.StreamFailures = 1; + var streamFailed = await Turn(threadId, "fixture-openai", "Actual Codex persists an explicit post-response stream failure", expectedFailure: true); + check("Actual post-response failure is eligible for guarded same-chat continuation", StreamRecoveryPolicy.Failed(CodexAccountDesk.JsonValue.String(streamFailed.GetProperty("turn"), "status")!)); + await Turn(threadId, null, "Actual Codex completes a following turn in the same chat after stream failure"); var config = await File.ReadAllTextAsync(Path.Combine(home, "config.toml")); check("Connection preserves unrelated TOML comments and settings", config.Contains("# Fictional isolated native client") && config.Contains("shell_tool = false")); var outsideEdit = config.Replace("name = \"Codex Switch\"", "name = \"Owner edited provider\""); diff --git a/tests/protocol/Program.cs b/tests/protocol/Program.cs index 5f7bbc9..a303e89 100644 --- a/tests/protocol/Program.cs +++ b/tests/protocol/Program.cs @@ -24,6 +24,47 @@ void Rejected(Action action) where T : Exception Assert(assembly.GetName().Name == "CodexSwitch.Sqlite" && !assembly.GetReferencedAssemblies().Any(a => a.Name is "CodexSwitch" or "CodexSwitch.Core" or "CodexSwitch.Protocol" or "PresentationFramework" or "Avalonia" or "Avalonia.Controls")); }); +JsonElement Session(string status, string? turn = null, bool child = false, long revision = 1) => JsonSerializer.SerializeToElement(new { + type = "snapshot", revision, conversationState = new { parentThreadId = child ? "main" : null, sideConversation = false, + threadRuntimeStatus = new { type = status, activeFlags = Array.Empty() }, threadGoal = (object?)null, + turns = turn == null ? Array.Empty() : new object[] { new { turnId = "fixture-turn", status = turn } } } }); +Check("Error count stays unknown until all discovered chats are observed", () => +{ + var ledger = new SessionActivityLedger(); Assert(ledger.Read().Errors == null); + ledger.Discover("unread", "owner"); ledger.CompleteDiscovery(); Assert(ledger.Read().Errors == null); + ledger.Apply("unread", "owner", Session("idle")); Assert(ledger.Read().Errors == 0); + ledger.Reset(); Assert(ledger.Read().Errors == null); + ledger.CompleteDiscovery(); Assert(ledger.Read().Errors == 0); +}); +Check("Only currently failed main chats count, without duplicates or interruptions", () => +{ + var ledger = new SessionActivityLedger(); + foreach (var id in new[] { "system", "failed", "active", "cancelled", "completed", "child", "unloaded" }) ledger.Discover(id, "owner"); + ledger.Apply("system", "owner", Session("systemError", "failed")); + ledger.Apply("system", "owner", Session("systemError", "failed")); + ledger.Apply("failed", "owner", Session("idle", "failed")); + ledger.Apply("active", "owner", Session("active", "failed")); + ledger.Apply("cancelled", "owner", Session("idle", "interrupted")); + ledger.Apply("completed", "owner", Session("idle", "completed")); + ledger.Apply("child", "owner", Session("systemError", "failed", true)); + ledger.Apply("unloaded", "owner", Session("notLoaded", "failed")); + ledger.CompleteDiscovery(); Assert(ledger.Read().Errors == 2); + ledger.Apply("failed", "other-owner", Session("idle", "completed", revision: 2)); Assert(ledger.Read().Errors == 2); + ledger.Apply("system", "owner", Session("active", "inProgress", revision: 2)); Assert(ledger.Read().Errors == 1); + ledger.Apply("failed", "owner", Session("idle", "completed", revision: 2)); Assert(ledger.Read().Errors == 0); +}); +Check("Latest failure and recovery patches update the count without rescanning history", () => +{ + var ledger = new SessionActivityLedger(); ledger.Discover("main", "owner"); ledger.Apply("main", "owner", Session("active", "inProgress")); ledger.CompleteDiscovery(); + ledger.Apply("main", "owner", JsonSerializer.SerializeToElement(new { type = "patches", baseRevision = 1, revision = 2, patches = new object[] { + new { op = "replace", path = new object[] { "threadRuntimeStatus", "type" }, value = "idle" }, + new { op = "replace", path = new object[] { "turns", 0, "status" }, value = "failed" } } })); + Assert(ledger.Read().Errors == 1); + ledger.Apply("main", "owner", JsonSerializer.SerializeToElement(new { type = "patches", baseRevision = 2, revision = 3, patches = new object[] { + new { op = "replace", path = new object[] { "threadRuntimeStatus", "type" }, value = "active" } } })); + Assert(ledger.Read().Errors == 0); + ledger.Remove("main"); Assert(ledger.Read().Errors == 0); +}); var absent = Path.Combine(Path.GetTempPath(), "codex-protocol-contract-" + Guid.NewGuid().ToString("N")); var monitor = new DesktopSessionMonitor(absent); try