grade(mcp): first evidence-graded review — B, citations one spec revision behind - #24
grade(mcp): first evidence-graded review — B, citations one spec revision behind#24john-the-dev wants to merge 1 commit into
Conversation
First evidence-graded review of entries/mcp.md, following the invitation in #22. Verified against the primary source the entry itself cites (modelcontextprotocol.io/docs/learn/architecture, checked 2026-08-31): three participants, two layers, JSON-RPC 2.0 data layer, stdio + Streamable HTTP. All four hold, so nothing here is wrong. B rather than A, because the citations have aged past the content: - the entry links specification/2025-06-18 twice; the live spec is 2026-07-28 - its own last_verified said 2026-07-29 — one day AFTER that revision — so the date asserted a freshness the body did not have - 2026-07-28 makes MCP stateless with a mandatory server/discover and deprecates sampling and logging; this entry frames the data layer around lifecycle messages and mentions neither server/discover nor elicitation That is the rubric's "accurate-but-aging, note the caveat in evidence", not "partially superseded" — the structure is current, the references are not. The revision itself is already covered by the sibling entry mcp-stateless, so this entry does not need to absorb it; it needs its citations refreshed, which is a separate change from grading it. validate.mjs: 59 entries valid, warnings 48 -> 47. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This branch graded mcp A on source-quality signals: official docs, HTTP 200, "no superseding protocol doc found at the same origin". A separate pass read the entry against the live spec instead and contradicted that last clause — the entry cites specification/2025-06-18 in two places while the current revision is 2026-07-28, which also makes MCP stateless with a mandatory server/discover and deprecates sampling and logging. The entry mentions none of that. GRADING.md defines A as "verified accurate within the last 90 days", so an accurate-source argument does not reach A when the entry's own citations are a revision behind. B with the caveat stated in evidence is the rubric's answer. Folds in the grading from #24, which is closed as superseded by this branch: both were open at once and assigned different grades to the same entry.
|
Closing as superseded by #21 — and the overlap was my error, not a coincidence. #21 (opened last night) grades 58 of 59 entries. I opened this PR and #22 this morning against a remembered figure of "48 unrated", which #21 had already obsoleted. Two of my own open PRs ended up assigning different grades to the same entry: A here on #21, B here. The disagreement is real and this PR was right. #21 graded mcp A on source-quality signals — official docs, HTTP 200, "no superseding protocol doc found at the same origin". This PR read the entry against the live spec and contradicted that last clause: it cites So the grading here is folded into #21 verbatim ( This PR's CI was also red for a reason worth naming — |
Taking my own suggestion from #22 — "One entry, graded with dated evidence, is a complete and useful PR." This is that PR, so the invitation has a worked example attached rather than just a claim.
What I verified
Against the primary source the entry itself cites (
modelcontextprotocol.io/docs/learn/architecture, checked 2026-08-31):Nothing in the entry is wrong.
Why B and not A
The citations have aged past the content:
specification/2025-06-18in two places; the live spec is 2026-07-28last_verifiedread 2026-07-29 — one day after that revision — so the date asserted a freshness the body did not have2026-07-28makes MCP stateless with a mandatoryserver/discover, and deprecates sampling and logging. This entry frames the data layer around lifecycle messages and mentions neitherserver/discovernorelicitationPer
GRADING.mdthat is B — "accurate-but-aging, note the caveat inevidence", not C. C is partially superseded; here the structure is current and only the references trail it.Deliberately not in this PR
entries/mcp-stateless.mdalready covers it as its own entry.Result
47 remain. The evidence items are written as dated, checkable statements per the rubric's own examples, so a later reader can re-run them rather than trust them.