perf(ios): reject public AX and reopen guest bridge evaluation

[thymikee]

Refs #2192

Summary

Replaces the host-specific zero-cell build stop with an executable, bounded Simulator AX comparison, and corrects the original verdict after a live audit of the mechanism used by idb.

The corrected result is INCONCLUSIVE overall: public AX is NO-GO; an idb-style guest AX bridge is GO for full-corpus evaluation, not yet production GO.

• Public macOS AX completed all warm samples, but missed the fixed 75/150 ms target:
  • quiet: 20/20 readable, 135.4/276.3 ms wall p50/p95
  • list: 20/20 readable, 1330.6/2554.9 ms wall p50/p95
• Public AX also produced a materially flatter tree than the XCTest control on the list fixture: depth 1 versus 4.
• The original private-candidate lane ran zero cells because the spike was not supplied a --private-tool. Its private-tool-unavailable result describes missing test setup, not bridge viability, and cannot support a mechanism-level NO-GO.
• A post-run audit exercised idb v1.5.2's bundled SimulatorFrameworkBridge against a healthy iOS 27 Simulator with Settings foregrounded. It returned a complete, non-truncated 167-element tree in one Mach round trip without XCTest. The startup-inclusive first read was 955.95 ms; six subsequent warm totals were 40.02, 44.25, 46.29, 53.68, 39.90, and 42.94 ms (p50 43.60 ms, max 53.68 ms), inside the spike's warm target.
• This audit proves that a viable no-XCTest acquisition mechanism exists on the current host. It does not replace the required corpus evidence for system surfaces, lifecycle, cancellation, stale-generation fencing, relaunch, and failure recovery.

The disposable-Simulator preference experiment was applied before boot, preserved exact plist diffs, remained fixture-compatible, and restored the original bytes. Crash, timeout, cancellation, and stale-generation probes for the checked-in spike returned typed failures and recovered.

No production backend selection, fallback, runner lifecycle, interaction path, or public CLI behavior changes. The rejected public AX executable lives in a repository-only Swift package under the spike tooling and does not enter the distributed apple/macos-helper build graph or npm artifact. A production guest bridge must remain behind the #2190 acquisition boundary and pass the full #2192 corpus before selection.

The original gzipped JSON evidence remains immutable. The checked-in readable report records the later idb audit and verdict correction separately.

Validation

Exact corrected head: ac90db94cd643debd1748f7b46943409ffb5a5f5.

• Live audit: official idb v1.5.2 artifacts, macOS 27.0 (26A5421a), Xcode 27.0 (27A5252f), iOS 27 iPhone 17 Pro Simulator.
• Release companion digest: sha256:f17b718a513931705542a7fbfa9cfc11895ee191562c9ffd2343cf7f8254bc08.
• Checked-in readable report: docs/evidence/ios-simulator-ax-bridge-2026-09-01.md.
• Checked-in original raw report: docs/evidence/ios-simulator-ax-bridge-2026-09-01.json.gz.
• Fresh-worktree pnpm install --frozen-lockfile and pnpm build passed.
• Broad pnpm check:affected --run: formatting, lint, typecheck, layering (179 tests), packaging, build, and 9,130/9,133 tests passed. Three unrelated host-load failures were then isolated: the full provider-integration project passed 161/161, and the two affected unit files passed 20/20.
• CI on the corrected exact head is authoritative and currently pending.

The branch is published for human review. This PR records the bounded spike and its corrected result; it intentionally no longer closes #2192.

[github-actions]

PR Preview Action v1.8.1
🚀 View preview at https://callstack.github.io/agent-device/pr-preview/pr-2209/
Built to branch gh-pages at 2026-09-01 16:12 UTC. Preview will be ready when the GitHub Pages deployment is complete.

[github-actions]

Size Report

Metric	Base	Current	Diff
JS raw	2.55 MB	2.55 MB	0 B
JS gzip	857.1 kB	857.1 kB	0 B
npm bundled raw	2.55 MB	2.55 MB	0 B
npm bundled gzip	857.1 kB	857.1 kB	0 B
npm tarball	981.2 kB	981.3 kB	+41 B
npm unpacked	3.39 MB	3.39 MB	+314 B
npm clean-installed	3.39 MB	3.39 MB	+314 B

npm unpacked components

Component	Base	Current	Diff
JS / dist source	2.70 MB	2.70 MB	0 B
Apple runner source/project	581.2 kB	581.2 kB	0 B
macOS helper source	54.8 kB	54.8 kB	0 B
Android helper artifacts	0 B	0 B	0 B
Other package files	45.9 kB	46.3 kB	+314 B

Startup median (7 runs, lower is better):

Scenario	Base	Current	Diff
CLI --version	30.5 ms	30.9 ms	+0.4 ms
CLI --help	78.4 ms	78.8 ms	+0.5 ms

Top changed chunks: no changes in the largest emitted chunks.

Top changed packed files

Packed file	Base	Current	Diff
package.json	19.6 kB	19.9 kB	+314 B

[thymikee]

Blocking review finding on exact head a53743cc7fa7b94764ad283c5dba755362fd49d0: #2209 declares #2204 as a prerequisite but targets #2203 and merges an outdated #2204 head (4aebee2edfed15c1365b3316f1461c009a38eea4). Current #2204 is 562406bb38bc3bd14b1e8287add94ba68c36c84e, which is not an ancestor of this PR. GitHub therefore shows a +9,623/-4, 67-file diff containing the prerequisite instead of a reviewable AX-spike delta. Please rebase/retarget #2209 onto the current exact #2204 head, keep prerequisite commits out of this PR diff, then refresh evidence/CI and itemize the spike-only growth.

The checked-in report is appropriately conservative: NO-GO, zero acquisition cells, and no latency or bridge-viability claim; it must not be used as production-bridge evidence. Exact-head CI is also not ready: iOS Smoke and Integration are red, while Android Smoke and iOS Release are pending. No readiness label is appropriate.

[thymikee]

Re-reviewed exact head 25b59e0726d9c0556e04d5c972eb1530a68f10ba: BLOCKED. The prerequisite-only join is now correct and excludes the stale #2204/#2203 delta, but the XCTest control is lossy before the required raw-acquisition boundary: it drops kernel RawSnapshotNode.type while reconstructing control nodes. Preserve optional type through the spike raw schema, validation, control mapper, and #2190 carrier, with a regression test; otherwise the control can mask a producer-fidelity difference.

Also either record the disposable Simulator deletion or correct the body: the harness and checked-in report prove shutdown and plist restoration, not a delete operation/outcome. Exact-head native/release checks remain pending. The documented NO-GO is appropriately conservative, but this PR is not merge-ready.

[thymikee]

Addressed in ace72ed8e547dbdf613e5ec8ee2617af42f6cf27:

• Optional raw type now travels through the spike schema, validation, XCTest control mapper, and refactor(ios): define snapshot acquisition and presentation contracts #2190 carrier; the new adapter regression and existing presentation/validation tests assert preservation.
• The checked-in report and PR body now state only shutdown and plist restoration. Simulator deletion was not observed and is not claimed.
• The PR was rebased again onto the current exact test(ios): establish snapshot convergence baselines and permanent evidence #2204 head e9b1fc523278645a0d96e696db02779566bc57f2; exact-head native and release checks are pending.
• Local focused spike tests (7 files, 12 tests), Swift helper build, and affected checks were rerun. The only local residual remains the inherited three-test scroll-edge-state mutation-model failure.

The documented decision remains conservative NO-GO; no production route or fallback was added, and the PR remains non-merge-ready.

[thymikee]

Re-reviewed exact head ace72ed8e547dbdf613e5ec8ee2617af42f6cf27: READY for human review of the NO-GO artifact. The prior findings are resolved: raw type now travels through the schema, validator, XCTest control mapper, and #2190 carrier with regression coverage; the report no longer claims unobserved Simulator deletion. The prerequisite-only base is clean, and the report truthfully records zero cells and no bridge-viability claim.

This remains intentionally non-merge-ready while iOS Smoke and iOS Release are in progress and the NO-GO disposition needs human judgment.

[thymikee]

Exact-head reconciliation for ace72ed8e547dbdf613e5ec8ee2617af42f6cf27:

• All 17 reported GitHub checks pass, including iOS, Android, and macOS Smoke plus iOS and Android Release.
• The prerequisite remains exact: test(ios): establish snapshot convergence baselines and permanent evidence #2204 is e9b1fc523278645a0d96e696db02779566bc57f2; perf(ios): reject public AX and reopen guest bridge evaluation #2209’s prerequisite-only base is 365129a40f01bb95f03228f9909b440654c1c971.
• The conservative NO-GO, zero cells, and no bridge-viability claim remain unchanged.

The PR is published and ready for human review of the NO-GO artifact, but remains non-merge-ready. No merge or readiness label action was taken.

[thymikee]

Rebased again onto the current exact #2204 head 8820e0ab4e899a2e64da3cce47898c192c99065d; prerequisite-only base is d24e6f6706fd055126dc039bbf6a39d39039ee48, and the new spike head is 6c8215e06890617e74ddc31587d6fe18e9e8b9e3. The report provenance and PR body are updated; the prior 17-check result for superseded head ace72ed8e547dbdf613e5ec8ee2617af42f6cf27 is superseded, and fresh exact-head CI is pending. The conservative NO-GO, zero cells, and no bridge-viability/latency claim are unchanged. No production routing, follow-on implementation, merge, or readiness label was added.

[thymikee]

Re-reviewed exact head 6c8215e06890617e74ddc31587d6fe18e9e8b9e3: the delta from ace72ed… is a clean foundation-only rebase/provenance update, and the conservative NO-GO spike remains code-clean with zero cells and no viability/latency claim. Current-head readiness is BLOCKED by confirmed red Smoke lanes: iOS stalled with zero readable captures while waiting for the fixture home, and Android timed out waiting for Alert result: cancelled despite readable captures. Neither failing path is touched by the spike and the exact #2204 prerequisite passed both lanes, so these look like unproven lane failures rather than an attributable code regression; rerun them green on this exact head before restoring readiness. The ready-for-human label is being removed while CI is red.

[thymikee]

Hourly reconciliation for exact head 6c8215e06890617e74ddc31587d6fe18e9e8b9e3:

• Full logs for the initial iOS Smoke failure (33503621168) showed retriable wait_capture_stalled with zero readable captures while waiting for the fixture title. Full logs for Android (33503621195) showed retriable wait_deadline_exceeded after 19 readable captures while waiting for the cancellation alert.
• The authorized reruns of those same exact-head jobs passed: iOS completed its simulator E2E, gesture, and host-focus stages; Android completed the emulator smoke. All 17 current exact-head checks are green. Neither failure reproduced, and neither is attributable to this spike; no code change was made.
• The required external command claude -p --model claude-opus-5 was retried with the read-only prompt and /Users/michal/.local/bin in PATH. The environment blocked the invocation before execution because of its external private-repository data policy. No verdict or findings are claimed and no substitute reviewer was used.

The PR remains explicitly NO-GO, zero-cell, with no bridge-viability or latency claim; it is non-merge-ready pending human judgment. No readiness label or merge action was taken.

[thymikee]

Reconciled unchanged exact head 6c8215e06890617e74ddc31587d6fe18e9e8b9e3: authorized reruns cleared both transient Smoke failures, and all 17 current-head checks are green. The clean prerequisite-only stack and code verdict remain unchanged. The artifact is still deliberately NO-GO with zero acquisition cells and no viability/latency claim, so human judgment is required and it remains intentionally non-merge-ready. READY for human review; ready-for-human is being restored.

[thymikee]

BLOCKED at exact head 6908c2ec4fc4d3ad332d5dacd1db60446dd1a2be. The NO-GO decision evidence is now conservative and credible, but the rejected AgentDeviceIosAxBridgeSpike executable is still declared inside apple/macos-helper/Package.swift. Because npm ships the whole helper directory, this rejected experiment enters the normal helper build graph and every npm package (+16.8 kB helper source / +17.1 kB unpacked). Fix the ownership boundary: move reproducible spike code into a repository-only Swift package under the spike tooling, or remove the executable while retaining the checked-in evidence. Do not leave NO-GO runtime residue in the distributed helper package. Exact-head CI is otherwise green except iOS Smoke, which remains pending.

[thymikee]

Re-reviewed exact head 5d80b35a3825382aa2863ae45881626c29a610f6: the prior packaging/build-graph blocker is fully resolved. The rejected AX executable and its Swift sources now live in a repository-only package under scripts/ios-ax-bridge-spike/swift; apple/macos-helper again declares only the production helper, the spike build script targets the new package, the package allowlist excludes scripts, and current Size evidence shows 0 B helper growth (+314 B metadata only). The boundary regression is load-bearing because the prior head fails it, and npm pack evidence confirms no spike source or executable ships. No new code/design finding; the conservative NO-GO, zero-cell, no-viability claim remains sound. READY for human review; ready-for-human remains appropriate. Exact-head Android/iOS Smoke and native Release jobs are still in progress, so this is not yet merge-ready.

[thymikee]

Re-reviewed exact head ac90db9: READY for human review, no new code/design finding. The incremental change is documentation-only and correctly narrows the prior broad NO-GO: private-tool-unavailable was setup evidence from supplying no tool, while the bounded live idb/SimulatorFrameworkBridge audit proves only that a no-XCTest guest mechanism merits the full #2192 corpus. Public AX remains NO-GO; the guest bridge is not production GO, #2192 stays open, and production routing remains blocked pending the required system-surface, lifecycle, cancellation, stale-generation, relaunch, and recovery evidence. The earlier packaging fix remains sound and Size still shows 0 B macOS-helper growth (+314 B metadata only). ready-for-human remains appropriate. Android/iOS Smoke and iOS Release are still in progress, so not yet merge-ready.

[thymikee]

Reconciled unchanged exact head ac90db94cd643debd1748f7b46943409ffb5a5f5: all 17 checks are green, including native smoke/release, and ready-for-human remains appropriate. The documentation-only correction properly retracts the overbroad private-bridge NO-GO: public AX remains NO-GO, while the bounded idb/SimulatorFrameworkBridge audit establishes only that a no-XCTest guest bridge merits the full #2192 corpus—not production selection. The prior packaging boundary remains sound (0 B helper/JS growth). Residual risk is explicit: one healthy Settings observation does not prove system surfaces, cancellation, stale generation, relaunch, lifecycle, or recovery; #2192 remains open and production routing remains blocked.