Skip to content

Add a replay control API for embedded browser viewers - #102

Open
JBoggsy wants to merge 1 commit into
mainfrom
replay-control
Open

JBoggsy wants to merge 1 commit into
mainfrom
replay-control

Conversation

@JBoggsy

@JBoggsy JBoggsy commented Oct 11, 2026

Copy link
Copy Markdown

Add a replay control API for embedded browser viewers

What and why

Pages that embed the WASM replay viewer can currently only load a replay and listen
for loading, ready and error. Coaching and analysis tools, such as Softmax
Observatory coaching and PlayerADE, need to drive the viewer. They need to seek to
the moment being discussed, pause, change speed, choose the camera, and draw
overlays that stay on top of units. That last one needs screen-to-world projection
and picking.

This PR adds that control surface. Gods of the Arena is hooked up. Other games
load as before and report that control is not available.

  • src/polyworld/replaycontrols.nim (new, game-agnostic): C exports over the
    shared Player transport (seek, cancel, play, pause, speed, loop, seek slice),
    plus projection math. Game-specific queries go through a small
    ReplayControlHooks object that the game installs: ground height, pick, unit
    lookup and camera mode. After presenting each frame, the game calls
    publishReplayControlFrame. That call reports the tick, play and seek state, a
    seek id and the view-projection matrix the frame was drawn with.
  • src/polyworld/replay.html:
    • window.polyworldReplay for same-origin hosts. It is synchronous and works in
      CSS pixels.
    • A postMessage bridge for cross-origin hosts. Commands are sent as
      {dst: 'coworld-replay', type, ...}, and queries are answered through a
      requestId.
    • Opt-in tick events (throttled to 10 Hz) and frame events.
    • Three URL options:
      • embed=1 hides the in-canvas HUD, blocks direct mouse, wheel, touch and key
        input, and forwards key presses to the host.
      • loop=0|1 sets looping.
      • seekSlice=MS sets the seek time slice.
    • The existing outbound messages are unchanged. New outbound messages are sent
      only to hosts that subscribe or use embed=1.
  • src/polyworld/player.nim:
    • Paused seeks now yield after a per-transport time budget, catchUpSeconds.
      Before, they ran to completion inside one frame, so a cold seek to the end of
      a match froze the page for about 7.5 s.
    • New cancelSeek() clears both the seek target and any pending restore.
    • Playing seeks keep the old 100 ms budget. Embedded viewers default to 33 ms.
  • GotA (examples/gods_of_the_arena/graphics.nim):
    • Installs the hooks.
    • pickEntity now takes an optional screen position, which defaults to the
      mouse.
    • Unit queries report alive and seat.
    • Camera modes: director, fixed, follow, and freeze, which holds the
      current pose.
    • The HUD is skipped when embedded.

The API is documented in docs/replay-control.md, which coworld/integration.md
links to.

How to use it

// Same origin
const replay = iframe.contentWindow.polyworldReplay;
replay.pause();
replay.seek(12000, 1);
replay.onFrame(f => { if (f.seekId === 1 && !f.seeking) draw(f.viewProjection); });
replay.pick({x: 640, y: 300});           // unit id or 0
replay.unit(104);                        // {x, y, z, seat, alive}

// Cross origin
iframe.contentWindow.postMessage({dst: 'coworld-replay', type: 'subscribe', tick: true}, '*');
iframe.contentWindow.postMessage({dst: 'coworld-replay', type: 'seek', tick: 12000, seekId: 1}, '*');

Viewer URL: gota.html#replay=URL&embed=1&loop=0.

What to review

  • The paused-seek change in Player.shouldTick affects every game's transport.
    Paused seeks used to finish in one frame. They now yield after 100 ms by
    default, the same budget playing seeks already had.

    • Step back and step forward are still exact.
    • A long paused seek now shows intermediate frames while seeking is true.
    • tests/test_player.nim now passes epochTime() instead of 0 as the frame
      start. With 0 the budget is always exceeded, and the old assertions only
      passed because paused seeks ignored the budget.
  • The seek slice is a trade-off between responsiveness and seek time. These
    numbers are for a cold seek over a 28,909-tick match, measured on an M-series
    Mac in headless Chromium using Metal:

    Slice Seek time 90th-percentile timer gap
    12 ms 28 s 78 ms
    33 ms (embed default) 17 s 123 ms
    100 ms 12 s 285 ms

    The worst single gap was about 300 to 460 ms at every slice. Something other
    than simulation causes those gaps, and I did not trace it. Hosts can pick
    their own slice with seekSlice.

  • embed=1 blocks input in JavaScript, not in Nim. A capture-phase listener
    on window stops events before Emscripten or windy see them. Resize is not
    blocked. This keeps the Nim input code untouched.

  • Exports pass vector results through polyworld_replay_result(i). This
    avoids relying on Emscripten heap views being exported, which the 6.x builds
    in testing did not need.

  • The GotA end tick in the browser test changed from 18680 to 28909. A
    recording made today with base.bas for all 10 seats runs to the time limit,
    1204.54 s at 24 ticks/s. The old constant no longer matches what the
    documented recording procedure produces.

Tests run

All results are from macOS arm64, Nim 2.2.10 and Emscripten 6.0.12.

  • nim r tests/test_player.nim: pass. Includes new cases for the
    paused-seek budget and for cancelSeek.
  • nim r tests/test_replaycontrols.nim: pass. This is a new test. It covers
    the projection round trip and the exports driving the transport and the hooks.
  • nim r tests/tests.nim: all tests passed.
  • nim r tests/test_directors.nim: pass.
  • nim check:
    • GotA, LvD and CTA pass in desktop, -d:headless and -d:coworld builds.
    • Crewrift and Heartleaf pass.
  • nim r coworld/tools/test_runtime.nim gota: pass, using a freshly built
    -d:coworld binary.
  • Headless recording and replay: I recorded tmp/coworld/gota.replay with
    the headless binary and --record. Replaying it with --replay verified
    8902/8902 actions.
  • test_browser_with_playwright.py --control-only --gpu: every check passes.
    • Cold seek to the last tick: 28 s, worst timer gap 371 ms, 814 progress frames.
    • Exact seeks in both directions, including the last tick.
    • Cancel lands on the last simulated tick.
    • Pause holds.
    • Speed mapping is correct, and 2x measured 48 ticks/s.
    • The frozen camera stays fixed while playing.
    • Unproject-then-project round trip: p90 error 0.013 px.
    • Pick: 10 of 10 live hero torsos.
    • postMessage bridge: commands, tick events, and requestId replies.
    • Embed: the wheel, clicks and Space do not change the camera or play state,
      and keys are forwarded.
  • test_browser_with_playwright.py --games gota on SwiftShader:
    • The existing GotA checks pass: full replay, seek, speed, resize, and looping
      still on by default.
    • The run then stops at the CTA error-fixture step, because no CTA viewer was
      built locally. That step always loads CTA, even with --games gota. This
      behaviour already existed and is not caused by this change.
  • --control-only on SwiftShader: every check passes.
    • Seeks use one-second slices here, because a software-rendered frame takes
      seconds late in a match.
    • The cold seek took 11.6 s over 9 progress frames.
    • The worst timer gap was 2.3 s. It is recorded only; responsiveness is not
      asserted without a GPU.
    • Pick: 9 of 9 live heroes.
  • Cherry-picked onto 02eda3d:
    • The same --control-only --gpu run passes.
    • A public 2026.10.8.1 replay (format 69, 23,642 ticks) seeks to its final tick
      with no divergence.

Follow-ups (not in this PR)

  • Faster forward seeks. A forward seek never restores a later checkpoint that
    already exists, so seeking back to the end re-simulates the whole match. Two
    cheaper options: restore the nearest checkpoint at or below the target in both
    directions, or pre-simulate the tape once in the background.
  • The 300–460 ms worst-case frame gap during seeks needs profiling. Likely
    candidates are checkpoint capture and memory growth.
  • Hook up LvD, CTA and AWM. Each needs its own ReplayControlHooks.
  • Arbitrary or fractional speeds. Only 1, 2, 4 and 16 are supported, through
    PlayerSpeeds.

🤖 Generated with Claude Code

Host pages (Observatory coaching, PlayerADE) need to drive the GotA WASM
replay viewer: exact seek, play/pause/speed, camera modes, and screen<->world
queries for overlays.

- src/polyworld/replaycontrols.nim: game-agnostic C exports over the shared
  Player transport plus ReplayControlHooks (groundY, pick, unit, camera) that
  a game installs; per-frame callback with tick, seek id and view-projection.
- replay.html: window.polyworldReplay for same-origin hosts, a postMessage
  bridge ({dst:'coworld-replay'}) with opt-in tick/frame events, and URL
  options embed=1 (hide HUD, block input, forward keys), loop=0|1, seekSlice.
  Existing outbound messages are unchanged.
- player.nim: paused seeks now yield after a per-transport catchUpSeconds
  budget instead of blocking until they land; add cancelSeek.
- GotA installs the hooks; unit queries report alive and seat.
- Tests: test_replaycontrols, test_player cases, browser control checks
  (--control-only, --gpu); GotA browser end tick updated to 28909.
- Docs: docs/replay-control.md, coworld/integration.md.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant