Skip to content

docs: qualify the FPGA LLM's 63 tok/s, 92 MHz and ~1W where they still read as results - #1159

Merged
dmitrii-f-t27 merged 2 commits into
gHashTag:mainfrom
dmitrii-f-t27:docs/stale-figures-2026-09-24
Sep 24, 2026
Merged

dmitrii-f-t27 merged 2 commits into
gHashTag:mainfrom
dmitrii-f-t27:docs/stale-figures-2026-09-24

Conversation

@dmitrii-f-t27

Copy link
Copy Markdown
Collaborator

The FPGA table in the root README.md (since ce54c79, #1040) already says ~34 tok/s was measured (16 tokens in ~467 ms at 50 MHz). It also says 63 tok/s is a projection at a 92 MHz Fmax estimate, and that power was not measured (no instrument, rail or method is on record), so ~1W and every tok/s/W derived from it are withdrawn. These files still stated the old figures as results. In docs and notes, the old number is struck through or qualified where it stands, with a pointer to that README table. In email texts (templates.zig and the three outreach drafts) the figures are replaced outright with the measured one, with no pointer.

File Was Now Source
docs/CLI_DEMOS_README.md:71,181 63 tok/s struck; tri benchmark prints no tok/s, the figure was the FPGA LLM projection README.md:703
docs/docs/research/fpga-autoregressive-llm-report.md:29,34,35,36 ~1W; 92 MHz; 15.9 ms/token and ~63 tok/s @ 92 MHz power not measured; 92 MHz an estimate; ~34 tok/s measured at 50 MHz, 63 a projection README.md:703-704
docs/docs/research/fpga-autoregressive-llm-report.md:91,96 ~63 tok/s/W, and a note that the ratio shows an advantage both withdrawn README.md:704 ("every tok/s/W derived from it")
docs/lab/papers/patent-strategy/full-analysis.md:16 63 tok/s @ 92 MHz, ~1W kept as the Zenodo record's wording, marked not current README.md:703-704
docs/reddit/t27ai/SIDEBAR.md:27,37 63 tok/s @ 1W; 1.2W struck, measured figure given; 1.2W struck (power not measured) README.md:703-704
docs/reddit/t27ai/EXAMPLE_POSTS.md:38 63 tok/s @ 1W struck, measured figure given README.md:703-704
docs/reddit/t27ai/EXAMPLE_POSTS.md:439,441 header "tok/s @ 1W"; HSLM 63 header marked power not measured; 63 withdrawn README.md:703-704
docs/proposals/BAEZ_OUTREACH.md:29, FRENKEL_OUTREACH.md:36,48,72,104, HOSSENFELDER_OUTREACH.md:29 63 tok/s, ~1W (FRENKEL:36 "concrete, verifiable") ~34 tok/s measured at 50 MHz, power not measured; "verifiable" dropped README.md:703-704
docs/research/sacred_formats_fpga.md:243 63 tok/s ~34 tok/s measured at 50 MHz, 63 a projection README.md:703
fpga/openxc7-synth/BENCH-005_RESULTS.md:34, BENCH-005_CORRECTED.md:44, gf16_synthesis_metrics.md:12,56 ≥92 MHz, "✅ Measured" ≥92 MHz labelled an Fmax estimate; status "Measured (LUT/FF/DSP; Fmax not measured)"; LUT/FF untouched README.md:703
src/tri/outreach/templates.zig:176,206,352,359 63 tok/s @ 1W replaced: ~34 tok/s at 50 MHz, power not measured README.md:703-704
src/tri/tri_patent.zig:145,347 63 tok/s @ 92 MHz, ~1W the Zenodo record's figures qualified in the printed text README.md:703-704
tapes/tri-benchmark.tape:2 comment quoting 63 tok/s quote withdrawn (comment only) README.md:703

The three outreach drafts are dated 2026-03-28 but are unsent (status "Draft"), so they are corrected like the email templates rather than kept as dated history.

Related wording ("silicon-proven", "~322 MHz") in apps/website/public/t27/files/** is a byte-identical mirror of files in gHashTag/t27, gHashTag/trinity-fpga and gHashTag/tri-net, so it is left to those repositories.

No golden file or snapshot compares the changed strings. templates.zig's "template word counts" test counts spaces in each body; the edits add four spaces to three templates: kanerva (81→85) and bitnet (71→75) stay inside its 70–130 bounds, rabaey was already below them on main (56→60), and the test does not compile on main (shadowed std). zig fmt --check passes on both .zig files (Zig 0.15.2), and tri_patent.zig passes ast-check.

{
  "version": 1,
  "head_sha": "38abb53b6b5afadd242723b71b4cd5c0064115fb",
  "summary": "Docs, outreach drafts and two CLI outputs stop stating the FPGA language model's 63 tok/s, ~1W and 92 MHz as results. Each line now follows the FPGA table in the root README.md: ~34 tok/s measured at 50 MHz, 63 tok/s a projection at a 92 MHz Fmax estimate, power not measured.",
  "changes": [
    "docs/docs/research/fpga-autoregressive-llm-report.md: Power, Fmax, Latency and Throughput rows qualified; the ~63 tok/s/W row and the efficiency note under it withdrawn",
    "docs/CLI_DEMOS_README.md: 63 tok/s struck; says tri benchmark prints no tok/s and the figure was the FPGA LLM projection",
    "docs/reddit/t27ai SIDEBAR.md and EXAMPLE_POSTS.md: 63 tok/s @ 1W and 1.2W struck, the tok/s @ 1W table header marked power not measured",
    "docs/proposals BAEZ, FRENKEL and HOSSENFELDER outreach drafts: 63 tok/s, ~1W replaced with ~34 tok/s measured at 50 MHz, power not measured",
    "docs/lab/papers/patent-strategy/full-analysis.md and docs/research/sacred_formats_fpga.md: the old figures marked as projection and withdrawn",
    "fpga/openxc7-synth BENCH-005_RESULTS.md, BENCH-005_CORRECTED.md and gf16_synthesis_metrics.md: the hslm row's >=92 MHz labelled an Fmax estimate, status says Fmax not measured",
    "src/tri/outreach/templates.zig email templates and src/tri/tri_patent.zig output: the figures replaced or qualified in the printed text only",
    "tapes/tri-benchmark.tape: the header comment no longer quotes 63 tok/s; tape commands unchanged"
  ],
  "tests": [
    {
      "command": "zig fmt --check src/tri/tri_patent.zig src/tri/outreach/templates.zig",
      "result": "Both files are formatted, no output",
      "status": "passed",
      "evidence": "Zig 0.15.2 on the branch blobs 2908854 and b132a72, run locally on 2026-09-24"
    },
    {
      "command": "zig ast-check src/tri/tri_patent.zig",
      "result": "No errors reported for the patent command",
      "status": "passed",
      "evidence": "Zig 0.15.2 on blob 2908854, run locally on 2026-09-24"
    },
    {
      "command": "zig ast-check src/tri/outreach/templates.zig",
      "result": "Fails with local constant shadows declaration of std at lines 428 and 444, the same errors as on main",
      "status": "failed",
      "evidence": "Same command on main blob of templates.zig at 18dafd7 gives identical errors; this PR does not touch those lines"
    },
    {
      "command": "python3 scripts/pr_blog_report.py validate --event event.json --output out",
      "result": "This PR body validates against the work report contract",
      "status": "passed",
      "evidence": "Local run of the script from main 18dafd7 with a synthetic event whose head sha is 38abb53"
    },
    {
      "command": "markdown link check over docs (docs-check.yml)",
      "result": "Not run locally; this PR adds or removes no links",
      "status": "not_run",
      "evidence": "The job checks the whole docs folder, so any red there predates this PR"
    }
  ],
  "limitations": [
    "apps/website/public/t27/files mirrors of t27, trinity-fpga and tri-net are left to those repositories",
    "The 1.2W and 19.6% LUT figures in the CLARA proposals and zenodo code are a separate cluster and are not edited here",
    "LUT ~7,400 (5.8%) in the FPGA report disagrees with 4,267 (6.7%) in README.md and is left for a follow-up",
    "The BitNet-3B 12 tok/s row in EXAMPLE_POSTS.md has no source found in the repository and is left for a follow-up",
    "The outreach texts still call deterministic-pattern weights a ternary LLM; README.md says the tokens do not reflect a trained model"
  ],
  "tags": [
    "fpga",
    "measurement",
    "docs",
    "honesty"
  ],
  "blog": {
    "title": "Retiring 63 tok/s at 1 W: the FPGA LLM numbers now match what was measured",
    "summary": "The root README already said the FPGA language model ran at about 34 tok/s at 50 MHz, that 63 tok/s was a projection and that no power measurement is on record. This PR carries that correction into the docs, outreach drafts, benchmark notes and two CLI outputs that still quoted the old figures.",
    "outline": [
      "The README's FPGA table records ~34 tok/s measured from 16 tokens in about 467 ms at 50 MHz, and calls 63 tok/s a projection at a 92 MHz Fmax estimate.",
      "Power was not measured on this board: no instrument, rail or method is on record, so ~1W and every tok/s per watt figure built on it are withdrawn.",
      "Docs, subreddit drafts and benchmark notes still quoted 63 tok/s at 1 W, so each line is struck or qualified in place with a pointer to the README table.",
      "Email templates and the three unsent outreach drafts replace the figures with the measured one, and the patent command's output qualifies the Zenodo record's figures in its printed text.",
      "Mirrored spec files under apps/website are byte-identical copies of other repositories, so their wording is fixed at the source rather than in the copy."
    ]
  }
}

🤖 Generated with Claude Code

dmitrii-f-t27 and others added 2 commits September 24, 2026 13:26
…l read as results

The FPGA table in the root README.md (since ce54c79, gHashTag#1040) records
~34 tok/s measured (16 tokens in ~467 ms at 50 MHz), 63 tok/s as a
projection at a 92 MHz Fmax estimate, and power as not measured: the
~1W figure and every tok/s/W derived from it are withdrawn. These files
still stated the old figures as results:

- docs/CLI_DEMOS_README.md (lines 71, 181): 63 tok/s struck, marked a projection
- docs/docs/research/fpga-autoregressive-llm-report.md: Power, Fmax,
  Latency and Throughput rows qualified; the ~63 tok/s/W row withdrawn
- docs/lab/papers/patent-strategy/full-analysis.md: the Zenodo record's
  figures marked not current
- docs/reddit/t27ai/EXAMPLE_POSTS.md (lines 38, 441) and
  docs/reddit/t27ai/SIDEBAR.md: 63 tok/s @ 1W struck
- docs/research/sacred_formats_fpga.md: hslm_full_top note corrected
- fpga/openxc7-synth/BENCH-005_RESULTS.md, BENCH-005_CORRECTED.md:
  >=92 MHz labelled an Fmax estimate
- src/tri/outreach/templates.zig: email templates say ~34 tok/s at
  50 MHz, power not measured
- src/tri/tri_patent.zig: `tri patent` output qualifies the record's figures
- tapes/tri-benchmark.tape: comment no longer quotes 63 tok/s

Source for every edit: the FPGA table in README.md (ce54c79, gHashTag#1040).
Files under apps/website/public/t27/files/ mirror t27 and trinity-fpga
byte for byte and are left to their source repositories.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Follow-up to 4b1f62a after review. Source for every edit, as before:
the FPGA table in the root README.md (ce54c79, gHashTag#1040), which gives
~34 tok/s measured at 50 MHz, calls 63 tok/s a projection at a 92 MHz
Fmax estimate, and says power was not measured (no instrument, rail or
method on record), withdrawing ~1W and every tok/s/W derived from it.

- docs/docs/research/fpga-autoregressive-llm-report.md: the note under
  the tok/s/W table no longer claims an efficiency advantage; "never
  measured" softened to "not measured (none is on record)"
- docs/reddit/t27ai/SIDEBAR.md: 1.2W struck (no power measurement is on
  record for this board); "never measured" softened as above
- docs/reddit/t27ai/EXAMPLE_POSTS.md: "tok/s @ 1W" header marked power
  not measured; "never measured" softened as above
- docs/proposals/BAEZ_OUTREACH.md, FRENKEL_OUTREACH.md,
  HOSSENFELDER_OUTREACH.md: unsent email drafts now say ~34 tok/s
  measured at 50 MHz, power not measured; FRENKEL drops "verifiable"
- docs/CLI_DEMOS_README.md: says `tri benchmark` prints no tok/s and
  that the withdrawn 63 tok/s was the FPGA LLM projection
- fpga/openxc7-synth/gf16_synthesis_metrics.md: the hslm row's >=92 MHz
  labelled an Fmax estimate, status says Fmax not measured
- fpga/openxc7-synth/BENCH-005_RESULTS.md, BENCH-005_CORRECTED.md:
  status cell says LUT/FF/DSP measured, Fmax not measured

Correction to 4b1f62a's message: the files under
apps/website/public/t27/files/ mirror gHashTag/t27, gHashTag/trinity-fpga
and gHashTag/tri-net (not only the first two) and stay untouched.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@dmitrii-f-t27
dmitrii-f-t27 merged commit 1be6b95 into gHashTag:main Sep 24, 2026
30 of 43 checks passed
github-actions Bot added a commit that referenced this pull request Sep 24, 2026
docs: qualify the FPGA LLM's 63 tok/s, 92 MHz and ~1W where they still read as results (#1159)

The root README's FPGA table (#1040) already says ~34 tok/s was measured at 50 MHz, 63 tok/s is a projection at a 92 MHz Fmax estimate, and power was not measured. Docs, outreach drafts, email templates and two CLI outputs now follow it: old figures struck or qualified in notes, replaced in text that generates new emails.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant