Skip to content

[Docs] Every OneJev-27B run behind #36's deadline table, the three interrupted ones included - #56

Open
chaimaerachdi wants to merge 2 commits into
ThinkFlowLab:mainfrom
chaimaerachdi:decision-timeout-records
Open

chaimaerachdi wants to merge 2 commits into
ThinkFlowLab:mainfrom
chaimaerachdi:decision-timeout-records

Conversation

@chaimaerachdi

Copy link
Copy Markdown
Contributor

Follow-up to #36 and #53: the review asked to retain the three interrupted runs alongside the seven completed ones. docs/results/decision-timeout/2026-10-01-a100-qev/ now holds every OneJev-27B run (qev serve, one A100 80 GB) with its log, profile and final screenshot, and a README mapping each run to the table:

  • 5 s (default): 5 runs, 0 / 5, each BLOCKED at decision 8 (the date on the calendar page, "decisions connection failed").
  • 30 s: 10 runs started; 7 completed and counted (7 / 7); 3 interrupted by hand (one KeyboardInterrupt, two that failed at their first request while the batch was being stopped). The recomputed medians match the description (61.2 s model time, 3.7 s per decision), except the browser median, 9.9 s rather than 9.7 s.
  • Disclosed: these runs predate [Feat] Make the decision deadline configurable for slower local System One servers #36's code; the 30 s deadline was set by a test-only launcher (slow_ok.py, included) that does what S1A_DECISION_TIMEOUT_S=30 does now.
  • Two later batches not in the [Feat] Make the decision deadline configurable for slower local System One servers #36 table are kept too: 30 s with a compact page state (4 / 5, one Google error page) and 30 s after installing flash-linear-attention (5 / 5).

Docs and records only; no code change. ruff check clean.

…'s table, the three interrupted ones included

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@Birfy Birfy left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Focused evidence review of head abe2a88d658025db6d23e1601fbab74c48fab4a4 against 3a2c2c66354c8ce587797d3e9e831db83abdaeb4.

One major evidence gap is attached inline: the raw run records described by this PR are absent from its complete Git tree and have no archive link here. Please restore reviewer access before treating #36's evidence request as fulfilled.

Scope: all seven changed-file names and the complete head tree, README claims, CSV summaries, and #36's linked discussion. The diff adds 287 lines, including 204 in the launchers/batch script. This was an artifact-availability check; runtime/model tests were not run. Observed core/full CI passed; browser CI was skipped.

Comment on lines +47 to +49

- `runs/<prefix>.log`: each run's stdout and stderr (the final JSON with status, answer and actions, or the
traceback). `runs/<prefix>.json`: the `--profile-out` record with every decision's time, tokens and choice (absent

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Major: make the claimed raw runs available to reviewers. At abe2a88, the complete, non-truncated Git tree contains only this README, three CSVs, bench.sh, slow_ok.py and compact_ok.py under this directory. There is no runs/ directory, so none of the promised logs, profiles or final screenshots can be inspected, including the three interrupted attempts that motivated this follow-up. #36's latest reply points here without an alternative archive.

Please upload the sanitized full records to a durable, reviewer-accessible archive or PR artifact and link it here, with each listed run mapped to its available files and genuinely unrecorded files identified. Preserve the interrupted attempts. This also follows the repository's artifact-hygiene guidance to keep one-off raw outputs outside the source tree.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rechecked head cfc83d2e1a33dfd9170ccdb34d223cd5119fd9e4: the availability and external-packaging requests above are addressed. I downloaded the linked ZIP; its SHA256 matches the README (372ff9d828250c2b146f341578a2c7bfbaac00fcf443bcb6aa6291986896fe57), and all 69 members are byte-identical to the files in 2dc6675 (25 logs, 19 profiles, 25 screenshots), with none missing. The current Git tree no longer contains the raw runs/ copies, consistent with the artifact-hygiene guidance.

The six absent profiles match the documented five 5-second runs and 140127. The interrupted/failure records are preserved: 140127 retains KeyboardInterrupt, and 140231/140246 retain the first-request connection failures. This verifies archive availability, mapping and byte preservation only; I did not independently reproduce model/browser outcomes or timing measurements. No overall approval is implied; thread state is unchanged.

@chaimaerachdi

Copy link
Copy Markdown
Contributor Author

Raw run records as an archive, per the review: 69 files (25 logs, 19 --profile-out records, 25 final screenshots), sha256 372ff9d828250c2b146f341578a2c7bfbaac00fcf443bcb6aa6291986896fe57. I am taking them back out of the source tree so the branch only carries the README, the CSVs and the scripts.

decision-timeout-2026-10-01-a100-qev-runs.zip

The 69 raw files (logs, profiles, final screenshots) stay outside the source
tree, per the repository's artifact hygiene; the README links the archive
attached to the pull request, with its sha256.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@chaimaerachdi
chaimaerachdi force-pushed the decision-timeout-records branch from 2dc6675 to cfc83d2 Compare October 12, 2026 15:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants