john-the-dev · john-the-dev · Aug 30, 2026 · Aug 30, 2026 · Aug 31, 2026 · Aug 31, 2026
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
@@ -8,7 +8,10 @@ Thanks for helping keep AI-agent engineering knowledge accurate and fresh.
 2. Fill the front-matter per [`SCHEMA.md`](SCHEMA.md). Grade it per [`GRADING.md`](GRADING.md) and back the grade with dated `evidence`.
 3. Rebuild the compiled output: `node scripts/build.mjs` (commit the updated `dist/kb.json`).
 4. Validate: `node scripts/validate.mjs` (CI runs this on every PR).
-5. Open a PR.
+5. If you set or changed a `grade`: `node scripts/check-sources.mjs` — it probes each
+   source against a same-origin control, because a 200 can be a catch-all challenge
+   page rather than the document. See GRADING.md for why this is not optional.
+6. Open a PR.
 
 ## Principles
 

diff --git a/GRADING.md b/GRADING.md
@@ -26,6 +26,38 @@ Each `evidence` item should be a dated, checkable statement, e.g.:
 `"last release 2026-06, 40k stars, MCP-native (checked 2026-07-14)"` or
 `"CVE-2026-55255 in CISA KEV, actively exploited (2026-07-11) → grade D"`.
 
+## Before you write "fetched live" — run the source check
+
+```
+node scripts/check-sources.mjs          # report
+node scripts/check-sources.mjs --strict # exit 1 if a GRADED entry is unverifiable
+```
+
+It fetches every source — front-matter `url` **and** body links — plus one
+nonsense-path control per origin, and classifies each `ok` / `catch-all` /
+`blocked` / `dead` / `error`.
+
+**Why the control, and not just a status check.** On 2026-08-31 two entries were
+graded on regulator pages that were never read. `www.ecfr.gov` answers **200**
+with the title `Federal Register :: Request Access` for *every* path, so a bare
+200 from that host means nothing at all. Only a path that should not exist can
+tell a real page from a catch-all.
+
+Three rules that came out of that, and out of the fixes to it:
+
+1. **A 200 is not a read.** Compare the title against a control before claiming
+   the source says anything.
+2. **A failure describes your CLIENT, not the page.** `www.ftc.gov` returned 403
+   to `curl` and 200 to `node`'s `fetch` on the same URL, minutes apart. If one
+   client is refused, try another before concluding anything about the source.
+3. **When a source genuinely cannot be verified, say that** — "HTTP 403 from
+   here, not read" — rather than grading it or writing "fetched live". Leaving an
+   entry `unrated` on an unreadable source is a correct outcome, not a gap.
+
+Not wired into CI on purpose: it makes live third-party requests, so per-push
+runs would be flaky and impolite to the sites being cited. Run it when grading,
+and when re-verifying a stale entry.
+
 ## Lifecycle
 
 - `status: current` → in-use, verified.

diff --git a/README.md b/README.md
@@ -32,23 +32,26 @@ Each entry is a markdown file with YAML front-matter (schema in [`SCHEMA.md`](SC
 
 ```markdown
 ---
-id: token-budgets
-title: "Token budgets change agent capability"
-url: https://www.aisi.gov.uk/
-category: evaluation
-source_type: research
+id: agent-delegated-auth
+title: "How do you authorize an AI agent to act on a user's behalf?"
+url: https://www.rfc-editor.org/rfc/rfc9728.html
+category: security
+source_type: docs
 status: current
 grade: A
 added: 2026-07-14
-last_verified: 2026-07-14
+last_verified: 2026-08-30
 superseded_by: null
 evidence:
-  - "AISI eval, 1M→10M tokens = +25% SWE success (verified 2026-07-14)"
-tags: [tokens, budget, evaluation]
+  - "RFC 9728 'OAuth 2.0 Protected Resource Metadata' fetched live 2026-08-30: HTTP 200"
+tags: [oauth, delegation, security]
 ---
-Increasing an agent's token budget from 1M to 10M improved SWE-task success by ~25% …
+An agent acting for a user needs delegated authority that is scoped, auditable and revocable …
 ```
 
+The `evidence` line is the load-bearing part: it names what was checked and when, so
+a reader can re-run the check rather than trust the grade.
+
 ## Quality grades
 
 Entries are graded **A/B/C/D** on an evidence-based rubric (see [`GRADING.md`](GRADING.md)) — not opinion. Deprecated or superseded tech is labeled, not deleted, so readers learn *why* something fell out of favor. Newly-seeded entries are `grade: unrated` until their first evidence-graded review.
@@ -59,7 +62,14 @@ PRs welcome — see [`CONTRIBUTING.md`](CONTRIBUTING.md). CI validates every ent
 
 ## Status
 
-**MVP / seed.** Seeded from 36 curated entries. Automated maintainer loop and site-ingestion wiring are landing next (see the repo issues / project notes).
+**59 entries, 58 evidence-graded** (25 A, 33 B). One is deliberately `unrated`: its
+source cannot be fetched from an automated client, and grading evidence nobody could
+read would be worse than leaving the gap visible.
+
+Sources are re-checked with [`scripts/check-sources.mjs`](scripts/check-sources.mjs),
+which probes every citation against a nonsense-path control on the same origin —
+some hosts answer `200` with a challenge page for *any* path, so a bare status code
+is not proof the page was read.
 
 ## License
 

diff --git a/dist/kb.json b/dist/kb.json
diff --git a/entries/a2a-protocol.md b/entries/a2a-protocol.md
@@ -5,11 +5,14 @@ url: https://a2a-protocol.org/
 category: protocols
 source_type: docs
 status: current
-grade: unrated   # seeded baseline — pending first evidence-graded review (see GRADING.md)
+grade: B
 added: 2026-08-16
-last_verified: 2026-08-16
+last_verified: 2026-08-30
 superseded_by: null
-evidence: []
+evidence:
+  - "the cited url https://a2a-protocol.org/ is a REDIRECT STUB, not documentation: title 'Redirecting', and curl -L does not move because the redirect is a client-side meta refresh (content='1; url=latest/'). The real docs are https://a2a-protocol.org/latest/, HTTP 200, title 'A2A Protocol' - that is where the url should point"
+  - "verified ON that page 2026-08-30: 'Linux Foundation' x3, 'Agent Card' x1, 'SSE' x11 - so the entry's Linux-Foundation, Agent-Cards-for-discovery and streaming claims are corroborated at the source"
+  - "CAVEAT driving B not A: the same scan found ZERO occurrences of json-rpc, webhook, or task lifecycle, which are three more things the entry asserts. They may sit on deeper spec pages, but the page this entry effectively points to does not evidence them. NEGATIVE CONTROL: /latest/not-a-real-a2a-page-xyz returns 404, so these are real absences and not a catch-all route"
 tags: [a2a, agent-to-agent, interoperability, protocols, agent-cards, json-rpc, linux-foundation, mcp]
 ---
 A2A (Agent-to-Agent) is the Linux Foundation open protocol for cross-vendor agent interop: JSON-RPC over HTTP with Agent Cards for discovery, task lifecycles for long-running delegation, and SSE/webhooks for streaming. Use it between agents from different teams or products; use MCP for an agent's own tools. v1.0 support now ships in major runtimes (e.g. Hermes Agent, Aug 2026).

diff --git a/entries/agent-cost-control.md b/entries/agent-cost-control.md
@@ -5,11 +5,15 @@ url: https://www.anthropic.com/engineering/building-effective-agents
 category: evaluation
 source_type: blog
 status: current
-grade: unrated   # seeded baseline — pending first evidence-graded review (see GRADING.md)
+grade: B
 added: 2026-07-14
-last_verified: 2026-07-29
+last_verified: 2026-08-30
 superseded_by: null
-evidence: []
+evidence:
+  - "source fetched live 2026-08-30: HTTP 200, 'Building Effective AI Agents \ Anthropic' — a primary vendor engineering essay, and a real one"
+  - "CAVEAT driving B not A: the essay is about agent design generally and does not address cost control — token budgeting, model tiering, caching economics — which is what this entry is about. It underpins the entry's framing, not its subject"
+  - "SHARED-SOURCE CAVEAT: this exact url backs FOUR entries in this KB — agent-loop, agent-cost-control, customer-support-agents and multi-agent-orchestration. agent-loop reaches it via /research/building-effective-agents, which 301s to the /engineering/ path, so it is one essay under two urls. A single general essay cannot be the sole evidence for four different applied claims; measured 2026-08-30"
+  - "NEGATIVE CONTROL: /engineering/not-a-real-post-xyz returns 404, so the 200s here are real pages rather than a catch-all route"
 tags: [cost, control, token, economics, measure]
 ---
 

diff --git a/entries/agent-delegated-auth.md b/entries/agent-delegated-auth.md
@@ -5,11 +5,14 @@ url: https://www.rfc-editor.org/rfc/rfc9728.html
 category: security
 source_type: docs
 status: current
-grade: unrated   # seeded baseline — pending first evidence-graded review (see GRADING.md)
+grade: A
 added: 2026-08-23
-last_verified: 2026-08-23
+last_verified: 2026-08-30
 superseded_by: null
-evidence: []
+evidence:
+  - "RFC 9728 'OAuth 2.0 Protected Resource Metadata' fetched live 2026-08-30: HTTP 200, Category Standards Track — a ratified IETF standard, not a draft"
+  - "entry's central claim (MCP servers implement RFC 9728 Protected Resource Metadata) names the exact RFC the source title states, verified 2026-08-30"
+  - "primary standards document, the top source-quality tier in GRADING.md; no superseding RFC found for 9728 as of 2026-08-30"
 tags: [identity, oauth, delegation, scoped-tokens, mcp, rfc9728, rfc8707, spiffe, authorization]
 ---
 Give an AI agent its own identity rather than the user's credentials: issue short-lived tokens whose sub names the human, act names the agent, aud binds a single resource server, and scope grants the narrowest verb that finishes the job. MCP servers are OAuth 2.0 resource servers and must implement RFC 9728 Protected Resource Metadata, returning 401 with a WWW-Authenticate resource_metadata pointer and binding token audience via RFC 8707. Subagents get narrowed derived tokens, never copies of the parent's credentials. Anti-patterns: inherited user tokens, unscoped API keys, standing credentials.

diff --git a/entries/agent-deployment.md b/entries/agent-deployment.md
@@ -5,11 +5,14 @@ url: https://docs.aws.amazon.com/lambda/latest/dg/best-practices.html
 category: memory
 source_type: docs
 status: current
-grade: unrated   # seeded baseline — pending first evidence-graded review (see GRADING.md)
+grade: A
 added: 2026-07-14
-last_verified: 2026-07-14
+last_verified: 2026-08-30
 superseded_by: null
-evidence: []
+evidence:
+  - "source is the official AWS Lambda best-practices guide, fetched live 2026-08-30: HTTP 200, title 'Best practices for working with AWS Lambda functions - AWS Lambda'"
+  - "unusually tight source-to-claim fit: the entry is specifically about serverless agent workers, and this is the primary vendor doc for exactly that runtime — not a general doc being stretched to cover it"
+  - "the entry's core instruction (treat workers as stateless, persist state in durable services because instances can vanish or be retried) is the same guidance the cited page gives for Lambda, checked 2026-08-30"
 tags: [deploying, serverless, infrastructure, treat, workers]
 ---
 

diff --git a/entries/agent-evals.md b/entries/agent-evals.md
@@ -5,11 +5,15 @@ url: https://www.anthropic.com/engineering/demystifying-evals-for-ai-agents
 category: evaluation
 source_type: blog
 status: current
-grade: unrated   # seeded baseline — pending first evidence-graded review (see GRADING.md)
+grade: B
 added: 2026-07-14
-last_verified: 2026-07-28
+last_verified: 2026-08-30
 superseded_by: null
-evidence: []
+evidence:
+  - "source fetched live 2026-08-30: HTTP 200, 'Demystifying evals for AI agents \ Anthropic' — a primary vendor engineering essay"
+  - "CAVEAT driving B not A: the essay is squarely about eval pitfalls, so the topical fit is good — but this entry's specific claims (single-run pass rates hiding nondeterminism, short time/token limits misclassifying capable agents) are the sharp, checkable ones and rest entirely on one vendor essay with no second source"
+  - "SHARED-SOURCE CAVEAT: this url backs TWO entries — agent-evals and evaluation-strategy. A redirect-normalised audit of all 59 entries on 2026-08-30 found only 2 shared sources covering 6 entries (55 distinct sources for 59 entries), so reuse is bounded and this is one of the two clusters, not a general pattern"
+  - "NEGATIVE CONTROL: anthropic.com/engineering/not-a-real-post-xyz returns 404, so the 200 is a real page"
 tags: [evaluation, pitfalls, evaluations, should, separate]
 ---
 

diff --git a/entries/agent-identity-secrets.md b/entries/agent-identity-secrets.md
@@ -3,13 +3,16 @@ id: agent-identity-secrets
 title: "Agent identity and secret management"
 url: https://csrc.nist.gov/pubs/sp/800/207/final
 category: memory
-source_type: blog
+source_type: docs
 status: current
-grade: unrated   # seeded baseline — pending first evidence-graded review (see GRADING.md)
+grade: B
 added: 2026-07-14
-last_verified: 2026-07-14
+last_verified: 2026-08-30
 superseded_by: null
-evidence: []
+evidence:
+  - "source is NIST SP 800-207 'Zero Trust Architecture' (csrc.nist.gov, final), fetched live 2026-08-30: HTTP 200 — a final NIST Special Publication"
+  - "CAVEAT driving B not A: SP 800-207 is a general zero-trust architecture standard that predates LLM agents; its per-workload identity and least-privilege principles transfer, but it makes no agent-specific claim (checked 2026-08-30)"
+  - "source_type corrected blog -> docs: a final NIST SP is a primary standards document, and GRADING.md ranks primary docs above secondhand blog"
 tags: [identity, secret, management, give, each]
 ---
 

diff --git a/entries/agent-loop.md b/entries/agent-loop.md
@@ -5,11 +5,14 @@ url: https://www.anthropic.com/research/building-effective-agents
 category: tools
 source_type: research
 status: current
-grade: unrated   # seeded baseline — pending first evidence-graded review (see GRADING.md)
+grade: B
 added: 2026-07-14
-last_verified: 2026-07-14
+last_verified: 2026-08-30
 superseded_by: null
-evidence: []
+evidence:
+  - "source fetched live 2026-08-30: HTTP 200, 'Building Effective AI Agents \ Anthropic' (the entry cites /research/, which 301s to /engineering/)"
+  - "of the four entries sharing this essay, agent-loop is the CLOSEST fit — the essay's subject is the loop itself, so this entry has the best claim on it of the four"
+  - "SHARED-SOURCE CAVEAT: one essay backs four entries here (agent-loop, agent-cost-control, customer-support-agents, multi-agent-orchestration). A redirect-normalised audit of all 59 entries on 2026-08-30 found 3 shared sources covering 8 entries, 54 distinct sources overall — bounded reuse, and this is the largest cluster"
 tags: [what, loop, repeatedly, reads, current]
 ---
 

diff --git a/entries/agent-memory-tiers.md b/entries/agent-memory-tiers.md
@@ -3,13 +3,16 @@ id: agent-memory-tiers
 title: "Short-term and long-term agent memory"
 url: https://arxiv.org/abs/2310.08560
 category: memory
-source_type: blog
+source_type: paper
 status: current
-grade: unrated   # seeded baseline — pending first evidence-graded review (see GRADING.md)
+grade: B
 added: 2026-07-14
-last_verified: 2026-07-29
+last_verified: 2026-08-30
 superseded_by: null
-evidence: []
+evidence:
+  - "source is arXiv 2310.08560 'MemGPT: Towards LLMs as Operating Systems', citation_date 2023-10-12, at least v2, fetched live 2026-08-30: HTTP 200"
+  - "CAVEAT driving B not A: foundational and widely cited, but a 2023 preprint — the tiered-memory idea it introduced is current while its specific system details have been overtaken (checked 2026-08-30)"
+  - "source_type corrected blog -> paper: arXiv preprint, which GRADING.md ranks above secondhand blog"
 tags: [short, term, long, memory, working]
 ---
 

diff --git a/entries/agent-observability.md b/entries/agent-observability.md
@@ -5,11 +5,13 @@ url: https://opentelemetry.io/docs/concepts/signals/traces/
 category: orchestration
 source_type: docs
 status: current
-grade: unrated   # seeded baseline — pending first evidence-graded review (see GRADING.md)
+grade: B
 added: 2026-07-14
-last_verified: 2026-07-28
+last_verified: 2026-08-30
 superseded_by: null
-evidence: []
+evidence:
+  - "source fetched live 2026-08-30: HTTP 200, 'Traces | OpenTelemetry' — the primary spec-level doc for distributed tracing concepts"
+  - "CAVEAT driving B not A: OTel Traces defines spans, context propagation and sampling generically; the entry is about AGENT observability — tool-call attribution, token and cost accounting, trajectory replay — which the page does not address. The tracing substrate is evidenced, the agent-specific layer is not"
 tags: [tracing, replay, represent, each, trace]
 ---
 

diff --git a/entries/agent-security.md b/entries/agent-security.md
@@ -5,11 +5,15 @@ url: https://agent-loop.xyz/library/agent-security
 category: security
 source_type: docs
 status: current
-grade: unrated   # seeded baseline — pending first evidence-graded review (see GRADING.md)
+grade: B
 added: 2026-08-16
-last_verified: 2026-08-16
+last_verified: 2026-08-30
 superseded_by: null
-evidence: []
+evidence:
+  - "source fetched live 2026-08-30: HTTP 200, 'AI agent security: the complete map - The Agent Loop'"
+  - "CIRCULAR CITATION — the real finding here. agent-loop.xyz is the site this KB powers (see the repo description), so citing one of its library pages cites a DERIVATIVE of this KB rather than independent evidence. The entry's four-pillar map may well be right; this url cannot be what establishes it"
+  - "partial corroboration that the page is KB-derived rather than independent: of 4 eight-word shingles taken from the entry text, 1 appears verbatim on the page. Weak evidence — the page is 38KB and partly client-rendered — so recorded as suggestive, not proof"
+  - "ACTIONABLE: repoint the url at the primary sources the pillar page itself rests on (prompt-injection, sandboxing and approval-gate references), or drop the url and let the pillar stand on the per-surface entries it links"
 tags: [security, hub, prompt-injection, memory-poisoning, sandboxing, guardrails, pillar]
 ---
 Pillar hub mapping the four agent attack surfaces — input (prompt injection), memory (poisoning), execution (sandboxing), authority (guardrails/approval gates) — with links to the deep-dive guide for each, plus real incident case studies (Langflow KEV, Therac-25, Knight Capital).

diff --git a/entries/agent-skills-vs-mcp.md b/entries/agent-skills-vs-mcp.md
@@ -5,11 +5,14 @@ url: https://agentskills.io/
 category: protocols
 source_type: docs
 status: current
-grade: unrated   # new entry — pending first evidence-graded review (see GRADING.md)
+grade: B
 added: 2026-08-25
-last_verified: 2026-08-25
+last_verified: 2026-08-30
 superseded_by: null
-evidence: []
+evidence:
+  - "source fetched live 2026-08-30: HTTP 200, 'Agent Skills Overview - Agent Skills' (negative control on the same host 404s, so the 200 is real)"
+  - "the url is a site ROOT but that is NOT a defect here — agentskills.io serves its documentation overview at the root, so the cited resource does address the subject"
+  - "CAVEAT driving B not A: the entry is a COMPARISON (Skills vs MCP) and this source documents only the Skills side. A comparison sourced from one of the two things compared cannot settle where the boundary falls"
 tags: [agent-skills, mcp, progressive-disclosure, tool-search, context-budget, tool-selection, capability-boundary, supply-chain]
 ---
 

diff --git a/entries/coding-agents.md b/entries/coding-agents.md
@@ -5,11 +5,14 @@ url: https://arxiv.org/abs/2310.06770
 category: security
 source_type: paper
 status: current
-grade: unrated   # seeded baseline — pending first evidence-graded review (see GRADING.md)
+grade: B
 added: 2026-07-14
-last_verified: 2026-07-14
+last_verified: 2026-08-30
 superseded_by: null
-evidence: []
+evidence:
+  - "source is arXiv 2310.06770 'SWE-bench: Can Language Models Resolve Real-World GitHub Issues?', citation_date 2023-10-10, fetched live 2026-08-30: HTTP 200"
+  - "CAVEAT driving B not A: SWE-bench remains the reference benchmark and the paper is heavily cited, but it is a 2023 preprint and the headline resolve-rates in it are far below what current coding agents achieve — cite it for the benchmark design, not for its numbers"
+  - "source_type corrected to paper: an arXiv preprint, which GRADING.md ranks above secondhand blog"
 tags: [production, coding, should, work, isolated]
 ---
 

diff --git a/entries/computer-use-agents.md b/entries/computer-use-agents.md
@@ -5,11 +5,13 @@ url: https://docs.claude.com/en/docs/agents-and-tools/computer-use
 category: tools
 source_type: docs
 status: current
-grade: unrated   # new entry — pending first evidence-graded review (see GRADING.md)
+grade: B
 added: 2026-08-24
-last_verified: 2026-08-24
+last_verified: 2026-08-30
 superseded_by: null
-evidence: []
+evidence:
+  - "source fetched live 2026-08-30: HTTP 200, 'Computer use tool - Claude Platform Docs'"
+  - "CAVEAT driving B not A: this is one vendor's tool reference. It is primary and current for that tool, but the entry generalises to computer-use agents as a class — screen-grounding accuracy, action-space design, failure recovery — which a single tool page does not establish"
 tags: [computer-use, browser-agents, gui-automation, screenshots, accessibility-tree, prompt-injection, approval, idempotency]
 ---