Skip to content

feat(web): add read-only Rate Shopping MCP server and /llms.txt - #1870

Merged
JOY (JOY) merged 3 commits into
devfrom
feat/1866-rate-shopping-mcp
Nov 22, 2026
Merged

JOY (JOY) merged 3 commits into
devfrom
feat/1866-rate-shopping-mcp

Conversation

@JOY

@JOY JOY (JOY) commented Nov 22, 2026 •

Copy link
Copy Markdown
Contributor

Closes #1866
Claimant: antigravity:8ad3645f-874c-4469-b6c5-68df6e156303

Summary

Implements Package 8A Feature F4: a stateless, read-only Streamable HTTP Model Context Protocol (MCP) server (POST /api/mcp), self-hosted AI discovery endpoints (GET /llms.txt, GET /.well-known/mcp.json), non-locale routing/crawler rules, and trilingual developer documentation across vi, en, and cs with zero database migrations and zero external LLM/infrastructure costs.

Key Changes

  • Stateless Streamable HTTP MCP Server (apps/web/src/server/rate/mcp.ts, apps/web/src/app/api/mcp/route.ts):
    • Implements JSON-RPC 2.0 over POST /api/mcp (initialize, notifications/initialized, ping, tools/list, tools/call) with backward-compatible protocol negotiation (2025-06-18, 2025-03-26, 2024-11-05), GET/DELETE 405 Method Not Allowed (Allow: POST, OPTIONS), and OPTIONS 204 CORS preflight.
    • Enforces strict anti-DoS and security guards: Content-Type: application/json (415), RFC 9110 Accept quality-weight validation (406), bounded stream reader <= 16384 bytes (413 / -32600 with early stream cancellation), batch array rejection, prototype-pollution key rejection (__proto__, constructor, prototype), safe-integer JSON-RPC id checks (including pre-parse > 53-bit integer rejection), .toWellFormed() UTF-16 sanitization, per-IP BoundedSlidingWindowRateLimiter (60 req/min general + 30 req/min URL audits, 429 + Retry-After), and sanitized error messages that never leak internal PostgREST/Supabase details.
    • Exposes 5 Zod-4 validated read-only tools with exported draft-2020-12 JSON Schemas (tools/list), backed exclusively by existing withRateDataCache BFF readers (limit <= 10):
      1. search_verified_catalog: Cross-category search with minPriceVnd/maxPriceVnd and price_asc/price_desc evaluated against verified in-stock plausible offer floors (excluding bait-price and sold-out traps).
      2. get_22_store_offers_and_price_truth: In-stock offers across 22 Vietnamese stores, historical floor/median truth, Deal Radar signal, Bayesian rating, 6-axis Review Trust Model v2 summary, non-synthetic crawl timestamps, and #1376 / #1465 risk-score masking (null when unanalysed or dosafe_risk_known: false).
      3. compare_devices_by_use_case: Side-by-side comparison of 2 to 4 same-category slugs (including Item/variant resolution and deduplication), 10-group hardware specs, and 5-criterion JReviews scores normalized over shared rankable criteria.
      4. audit_listing_url: Canonicalizes tracking/affiliate query parameters before cache lookup, validates public URLs across the 22 supported Vietnamese platforms or rate.vn, and returns catalog match, 5-level fake-discount verdict, accessory gatekeeper check, and seller risk scores.
      5. get_genuine_deals: Verified below-median and multi-store floor deals (limit <= 10) with category, platform, official-mall, and budget filters.
    • Includes canonical_citation_url (https://rate.vn/...) and ISO-8601 freshness timestamps (observed_at / price_observed_at) on every returned tool payload and item.
  • Self-Hosted AI Discovery & Routing (apps/web/src/app/llms.txt/route.ts, apps/web/src/app/.well-known/mcp.json/route.ts, apps/web/src/proxy.ts, apps/web/src/app/robots.ts):
    • Prerenders GET /llms.txt (text/plain; charset=utf-8) and GET /.well-known/mcp.json (application/json) as static assets with Cache-Control: public, s-maxage=3600, stale-while-revalidate=86400 and MCP discovery CORS headers.
    • Bypasses locale redirects in src/proxy.ts without breaking the next.config.mjs CSP header complement (csp.test.ts), and explicitly allows /llms.txt, /.well-known/mcp.json, and /api/mcp in production robots.ts.
  • Developer Portal Documentation (apps/web/src/app/[locale]/developers/page.tsx, apps/web/locales/{vi,en,cs}.json):
    • Documents /api/mcp, /.well-known/mcp.json, /llms.txt, all 5 MCP tools, and copy-ready .mcp.json / .cursor/mcp.json / DOSClaw + claude_desktop_config.json (mcp-remote) and JSON-RPC curl snippets with 100% vi, en, cs parity and WCAG 2.2 AA dark-mode contrast (btn-outline-dark).

Fresh-Context Review

  • Note: Local Codex CLI is currently out of quota until Nov 25, 2026, so Round-1 fresh-context review was executed via 2 independent read-only research subagents with disjoint review scopes:
    1. MCP Protocol, Security & Anti-DoS Reviewer: Identified 6 P1 and 3 P2 edge cases (pre-cache URL canonicalization in audit_listing_url, stream reader abort handling, MCP-Protocol-Version negotiation on initialize, Accept: application/json;q=0 handling, safe-integer / > 53-bit JSON-RPC id checks, and price_asc / minPriceVnd / maxPriceVnd post-plausibility filtering). All fixed and covered by regression tests.
    2. Data Truth, Citations, A11y & i18n Reviewer: Identified 5 P1 and 3 P2 edge cases (ignoring synthetic live_listing / sold-out points in price_observed_at, Item-level compare deduplication and shared-criteria weight normalization, rate.vn dotted-slug validation, btn-outline-dark dark-mode AA contrast on /developers, exact 10 ProductSpecifications group keys in /llms.txt, and natural Vietnamese "AI agent" phrasing). All fixed and covered by regression tests.

Verification

  • pnpm --filter @repo/web test (all 56 test suites passed, including new mcp.test.ts, developers-copy.test.ts, proxy.test.ts, csp.test.ts, and core-rate-architecture.test.ts)
  • pnpm typecheck (6/6 packages passed)
  • pnpm lint (4/4 packages passed, 0 warnings)
  • pnpm build (5/5 packages passed; /.well-known/mcp.json and /llms.txt statically prerendered, /api/mcp dynamic)

📌 TL;DR

This PR introduces a stateless, read-only Model Context Protocol (MCP) server and /llms.txt endpoint to expose verified price data from 22 Vietnamese retail platforms to AI agents. It includes the core API routes, middleware updates to bypass locale routing for these discovery paths, comprehensive localization, and a dedicated documentation section on the developer page.

🎯 Type of Change

  • 🚀 New feature
  • 🐛 Bugfix
  • 🧹 Refactor
  • ⚡ Performance
  • 📚 Documentation
  • ⚙️ CI / Configuration

🔍 Changes Walkthrough

File Summary of Changes
apps/web/src/server/rate/mcp.ts Implied by imports/tests: Core logic for the MCP server, including JSON-RPC handling, tool definitions, rate limiting, and response builders.
apps/web/src/app/api/mcp/route.ts New API route handling POST (JSON-RPC), OPTIONS (CORS), and rejecting GET/DELETE for the MCP endpoint.
apps/web/src/app/.well-known/mcp.json/route.ts New static route serving the MCP discovery descriptor with appropriate CORS and caching headers.
apps/web/src/app/llms.txt/route.ts New static route serving a Markdown context guide for LLMs with appropriate headers.
apps/web/src/proxy.ts Adds isDirectNonLocalePath helper to bypass locale rewriting and session refresh for /api/mcp, /llms.txt, and /.well-known paths.
apps/web/src/app/robots.ts Updates production robots.txt to explicitly allow crawling of the new MCP and LLM discovery endpoints.
apps/web/src/app/[locale]/developers/page.tsx Adds a new "MCP & /llms.txt" section to the developer docs, including tool descriptions, client config snippets, and endpoint references.
apps/web/locales/{en,vi,cs}.json Adds localization keys for the new MCP documentation section, tool descriptions, and endpoint summaries.
apps/web/src/server/rate/__tests__/mcp.test.ts New comprehensive test suite validating MCP tool logic, rate limiting, response formats, and edge cases (bait pricing, sold-out items).
apps/web/src/lib/__tests__/proxy.test.ts Extends proxy tests to verify that MCP/LLM paths skip middleware, do not redirect, and are correctly allowed in robots.txt.
apps/web/src/lib/__tests__/developers-copy.test.ts Extends copy tests to ensure MCP documentation keys are present, localized, and free of em-dashes/en-dashes.
apps/web/package.json Adds the new mcp.test.ts to the test execution pipeline.

📊 Architectural Flow

sequenceDiagram
    participant AI as AI Agent / LLM
    participant Proxy as Next.js Proxy (Middleware)
    participant MCP as /api/mcp (POST)
    participant LLM as /llms.txt (GET)
    participant WK as /.well-known/mcp.json (GET)
    participant DB as Data Layer (Supabase/Cache)

    Note over AI, WK: Discovery Phase
    AI->>WK: GET /.well-known/mcp.json
    Proxy->>WK: Pass-through (No locale rewrite)
    WK-->>AI: MCP Descriptor (JSON)
    
    AI->>LLM: GET /llms.txt
    Proxy->>LLM: Pass-through (No locale rewrite)
    LLM-->>AI: Markdown Context Guide

    Note over AI, DB: Interaction Phase
    AI->>MCP: POST JSON-RPC (tools/call)
    Proxy->>MCP: Pass-through (No session refresh)
    MCP->>MCP: Validate Rate Limit & Body Size
    MCP->>DB: Query Verified Offers/Prices
    DB-->>MCP: Raw Data
    MCP->>MCP: Filter (Exclude bait/sold-out) & Format
    MCP-->>AI: JSON-RPC Response (with citations)
Loading

Add stateless Streamable HTTP JSON-RPC 2.0 MCP endpoint (/api/mcp) with 5
Zod-4 validated read-only tools across 22 Vietnamese retail platforms,
self-hosted discovery routes (/llms.txt and /.well-known/mcp.json),
proxy/robots bypass rules, trilingual /developers docs (vi, en, cs), and
comprehensive security and data-truth regression tests.

Closes #1866

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Code review skipped — your organization has no extra usage available to pay for this review.

If your organization's extra usage balance is empty, an organization admin can add extra usage credits at claude.ai/admin-settings/usage. If its monthly spend limit was reached, an admin can raise it on the same page. If neither applies, contact Anthropic support.

Once extra usage is available, someone with write access to this repository can comment @claude review on this pull request to trigger a review.

@vercel

vercel Bot commented Nov 22, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
rate Ready Ready Preview Nov 22, 2026 3:49pm UTC

Request Review

@coderabbitai

coderabbitai Bot commented Nov 22, 2026 •

Copy link
Copy Markdown

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 30 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: a7550132-fba1-42c1-90b1-e232ab6266e1
Commits

Reviewing files that changed from the base of the PR and between 1c75b42 and 6ccc307.

Files selected for processing: 15

  • .gitignore
  • apps/web/locales/cs.json
  • apps/web/locales/en.json
  • apps/web/locales/vi.json
  • apps/web/package.json
  • apps/web/src/app/.well-known/mcp.json/route.ts
  • apps/web/src/app/[locale]/developers/page.tsx
  • apps/web/src/app/api/mcp/route.ts
  • apps/web/src/app/llms.txt/route.ts
  • apps/web/src/app/robots.ts
  • apps/web/src/lib/__tests__/developers-copy.test.ts
  • apps/web/src/lib/__tests__/proxy.test.ts
  • apps/web/src/proxy.ts
  • apps/web/src/server/rate/__tests__/mcp.test.ts
  • apps/web/src/server/rate/mcp.ts
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@github-actions github-actions Bot added the area:frontend Frontend / Next.js / UI label Nov 22, 2026
dos[bot]
dos Bot approved these changes Nov 22, 2026

@dos dos Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⏱️ Code Review completed (14 eligible files · 211,893 chars · 1 model call(s) · 8 PR unit(s))

Whole diff with bounded context at base d896a99 / head 43bf56f.

✅ CLEAN_PASS: Complete diff reviewed

dos[bot]
dos Bot approved these changes Nov 22, 2026

@dos dos Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⏱️ Code Review completed (15 eligible files · 212,180 chars · 3 model call(s) · 8 PR unit(s))

Whole diff with bounded context at base 1c75b42 / head 6ccc307.

✅ CLEAN_PASS: Complete diff reviewed

Context investigation was bounded to the explicitly requested snapshot paths.

@JOY

Copy link
Copy Markdown
Contributor Author

PO acceptance: merge at 6ccc307

@JOY
JOY (JOY) merged commit 31572ce into dev Nov 22, 2026
15 checks passed
@JOY
JOY (JOY) deleted the feat/1866-rate-shopping-mcp branch November 22, 2026 16:00

This branch was successfully deployed

1 active deployment
Preview — 6ccc3078 Deployed Nov 22, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:frontend Frontend / Next.js / UI

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(web): add read-only Rate Shopping MCP server and /llms.txt

1 participant