Skip to content

test(runtime-host): assert liveness deadlines on a virtual clock - #4939

Merged
Astro-Han merged 2 commits into
apache:mainfrom
orangeCatDeveloper:test/runtime-host-virtual-clocks
Sep 7, 2026
Merged

test(runtime-host): assert liveness deadlines on a virtual clock#4939
Astro-Han merged 2 commits into
apache:mainfrom
orangeCatDeveloper:test/runtime-host-virtual-clocks

Conversation

@orangeCatDeveloper

@orangeCatDeveloper orangeCatDeveloper commented Sep 6, 2026

Copy link
Copy Markdown
Contributor

Summary

Two @maka/runtime-host tests slept through real deadlines — the 5s proactive
upgrade interval and the 8s liveness probe deadline — costing 18s per run while
only observing the end state. Because they never checked the boundary, they also
passed against a broken threshold. Driving the mocked clock to each side of the
deadline pins the threshold and removes the waiting.

Separately, kache report wrote only to $GITHUB_STEP_SUMMARY, so Rust build
cache hit rates were visible in the run page panel but absent from the raw log —
not greppable in a downloaded log, not comparable across runs. All six workflows
that run kache now tee the report into both, unified rather than fixed one at a
time.

The tests keep real socket I/O and every original assertion; only the clock is
mocked, at the seams production already uses (performance.now, the heartbeat
setInterval, and the probe deadline setTimeout). No production logic, no
threshold, no runner and no cache key changes.

Verification

node --test on the two suites, the five CI policy scripts, and biome on the
touched files — all pass. The targeted cases ran 10 consecutive times with no
failure or cancellation.

Wall clock, same machine and build:

before:
resumable-peer-stream.test.js        12s   10 pass
session-subscription-client.test.js  11s   27 pass
failed proactive upgrade                     duration_ms: 10034.729667
closes an unresponsive request path          duration_ms:  8061.532459

after:
resumable-peer-stream.test.js         2s   10 pass
session-subscription-client.test.js   3s   27 pass
failed proactive upgrade                     duration_ms:     5.623125
closes an unresponsive request path          duration_ms:   160.268125

The new boundary assertions were mutation-tested. Lowering the upgrade interval
in transport/resumable-peer-stream.ts from 5_000 to 3_000:

old assertions, mutated production code:
ok 1 - failed proactive upgrade preserves transit; a later direct attachment keeps the logical stream
  duration_ms: 6028.081083
# pass 1

new assertions, same mutated production code:
not ok 1 - failed proactive upgrade preserves transit; a later direct attachment keeps the logical stream
  expected: 0
  actual: 1
  operator: 'strictEqual'
# fail 1

Lowering DEFAULT_LIVENESS_TIMEOUT_MS in client/connection.ts from 8_000 to
6_000 fails the probe-deadline assertion the same way. Reverting one workflow
to >> fails the policy check:

all six workflows unified:
ℹ pass 40   ℹ fail 0

one workflow reverted to >>:
ℹ pass 39   ℹ fail 1
    actual:   'kache report --format github >> "$GITHUB_STEP_SUMMARY"'
    expected: 'kache report --format github | tee -a "$GITHUB_STEP_SUMMARY"'

Not run: Linux and Node 24. These suites interleave mocked timers with real
socket I/O, so CI is the first run on the platform that matters.

Review focus

set -o pipefail makes a failing kache report fail its step instead of being
swallowed by the pipe. That is the intent, but a runner with a flaky kache exit
code would surface here for the first time.

AI use

Select exactly one:

  • No generative tool made a substantive contribution
  • Generative tooling made a substantive contribution

Tool(s) and scope: Claude Code — wrote the virtual-clock test changes and the
workflow/policy edits, and ran the timing and mutation verification above. The
approach, the assertion placement and the final review are the author's.

Checklist

  • Tests cover the change and fail without it
  • Lint, format, typecheck and the affected suites pass locally

Does this PR entail a change in behavior?

  • Yes — described under Summary above
  • No

Both tests slept through the real 5s upgrade interval and the real 8s
probe deadline, so they only observed the end state and passed even when
the threshold moved: cutting the upgrade interval to 3s left the old
assertions green. Driving the mocked clock to each boundary pins the
deadline itself and drops 18s of waiting.

Generated-by: Claude Code
The Rust build cache report only reached the rendered step summary, so
cache hit rates could not be grepped from downloaded logs or compared
across runs. Piping through tee keeps the summary and adds the log;
pipefail preserves the step's failure on a broken report.

Generated-by: Claude Code
@github-actions github-actions Bot added the effort/S Under 100 readable lines label Sep 6, 2026

@hqhq1025 hqhq1025 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

复核 f22cbc928ec51a1b52f6bc7fed4fba3a29d1b7b6,未发现有充分证据支持的 P0–P3 问题。

实际改动是把两个保留真实 socket I/O 的测试改为受控时钟,并让六份 workflow 的 kache 报告同时进入原始日志与 step summary。生产超时常量和协议没有修改。

resumable-peer-stream.test.ts:472 使用生产读取的 performance.now 和心跳 interval,:532:539 检查两次升级前后的边界及旧连接存活。session-subscription-client.test.ts:1421 等真实对端收到探测后推进 7999ms,在收到最后一个通知后验证连接仍开,再推进 1ms 验证关闭。scripts/ci-workflow-policy.test.mjs:294 覆盖六份报告配置;Windows 发布 workflow 的 Bash 默认值在 .github/workflows/release-windows-check.yml:123,管道保留了失败退出码。

Linux / Node 24.18.1 的干净安装、build:test、77 项相关测试、变更脚本 lint/format 和 diff 检查通过。独立执行六个实际报告管道的成功/失败分支,共 12 项通过。四个生产计时常量的提前/延后 mutation 均被测试拒绝,恢复后两项控制测试通过。

局限是延后的 liveness mutation 在测试超时后仍留下通知 interval,需要外层守护结束进程,不能把它描述成干净的断言失败;250ms 轮询也限制了可观察的阈值精度。这些测试不证明真实操作系统调度延迟。当前 head 的 hosted 检查已成功,Eval 检查为 SKIPPED;与 main b06eb02e6 合并树无冲突,未在合并树上重跑测试。

Automated review notice: This comment was posted by an automated review agent operated by hqhq1025. It is not an independent human review and does not replace one.

@Astro-Han Astro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the change. Approving exact head f22cbc928ec51a1b52f6bc7fed4fba3a29d1b7b6 following the completed review and Astro-Han’s explicit acceptance of this merge batch. Current checks pass and no review threads remain unresolved.

Liveness tests waited through real deadlines without checking their boundaries. Drive existing clock seams before and after the deadlines while retaining socket I/O. Also write kache reports to both job summaries and raw logs with failure status preserved. No production timeout or cache-key changes.

AI assistance: Codex performed the review and final-state verification; Astro-Han authorized approval and merge.

中文

感谢改动。基于已完成的审查和 Astro-Han 对本批次的明确认可,批准当前精确 head;检查通过,讨论已结清。此前说明的验证边界与后续事项保持不变。本次由 Codex 执行审查和状态核对,Astro-Han 授权批准与合并。

@Astro-Han
Astro-Han merged commit 99ad181 into apache:main Sep 7, 2026
19 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

effort/S Under 100 readable lines

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants