ClauseGuard

Multimodal RAG compliance analyst for SEC filings, built as an inspectable AI engineering portfolio project.

ClauseGuard ingests real EDGAR filings, retrieves text and visual evidence, answers compliance and financial questions, verifies every generated claim against cited source chunks, flags risk clauses, and tracks regression metrics over time.

What This Demonstrates

AI engineering requirement	Where it appears
RAG over real data	SEC EDGAR ingestion, section-aware chunks, Qdrant hybrid retrieval
Multimodal retrieval	ColQwen2-style page multivectors for rendered filing pages
Multi-agent orchestration	Explicit LangGraph state machine with typed nodes and edges
LLM abstraction	All model calls go through ModelGateway typed contracts
Citation verification	Entailment-style verifier strips unsupported generated claims
Evaluation infrastructure	159-case real-filing golden set, deterministic metrics, RAGAS adapter
LLMOps / observability	Node traces, API p50/p95 latency, token-cost fields, LangSmith payloads
Production-aware serving	FastAPI, basic auth, rate limiting, async jobs, metrics endpoint
Product surface	Streamlit review UI for queries, sources, risks, evals, and ops metrics

Architecture

flowchart LR
    sec["SEC EDGAR"] --> ingest["Ingestion"]
    ingest --> chunks["Parsed blocks + source chunks"]
    chunks --> qdrant["Qdrant hybrid index"]
    user["Streamlit / API user"] --> api["FastAPI"]
    api --> graph["LangGraph agents"]
    graph --> qdrant
    graph --> verifier["Citation verifier"]
    verifier --> answer["Cited answer + risk flags"]
    evals["Golden eval harness"] --> graph
    evals --> reports["Regression reports"]

Loading

Agent graph:

graph TD
    router["Router"] --> retrieval["Retrieval Agent"]
    retrieval --> extraction["Extraction Agent"]
    extraction -->|"comparison"| comparison["Cross-Filing Comparison"]
    extraction -->|"single/risk"| verifier["Citation Verifier"]
    comparison --> verifier
    verifier -->|"flagged/risk"| risk["Risk Scorer"]
    verifier -->|"supported"| final["Finalize Answer"]
    risk --> final

Loading

Current Status

• Phase 0: Python project scaffold, uv, Makefile, pre-commit config
• Phase 1: SEC EDGAR ingestion with resumable local artifacts
• Phase 2: Hybrid text/visual retrieval and retrieval-only eval
• Phase 3: Typed LangGraph multi-agent pipeline
• Phase 4: Golden-set evaluation harness and regression reports
• Phase 5: FastAPI serving layer with async jobs, auth, rate limits, metrics
• Phase 6: Streamlit frontend for query review, citations, risks, evals, ops
• Phase 7: Architecture and portfolio writeup

Quickstart

uv sync --all-extras
cp .env.example .env

Before live ingestion, set a real SEC user agent:

SEC_USER_AGENT="ClauseGuard/0.1 your-email@example.com"

Run tests:

make test

Run ingestion:

make ingest

Index retrieval artifacts:

make retrieval-index
make retrieval-visual

Run the API:

make serve

Run the frontend:

make frontend

Run the golden-set eval:

make eval

API

All non-health endpoints use HTTP Basic auth. Replace CLAUSEGUARD_API_PASSWORD before exposing the service.

Method	Path	Purpose
GET	/healthz	Liveness
POST	/query	Single or multi-filing question
POST	/ingest	Async filing ingestion
GET	/jobs/{id}	Long-running job status/result
GET	/metrics	Route latency and token-cost metrics

Evaluation Results

These are actual local baseline runs from .clauseguard/eval/reports, using the 159-case real-filing golden set. No numbers are fabricated.

Run	Cases	Faithfulness	Answer relevancy	Context precision	Context recall	Citation accuracy
phase4-iteration-1	159	0.7672	0.2629	0.1627	0.6384	0.2453
phase4-iteration-2	159	0.7672	0.2629	0.1627	0.6384	0.2453
phase4-iteration-3	159	0.7672	0.2629	0.1627	0.6384	0.2453
phase5-serving-check	159	0.7672	0.2629	0.1627	0.6384	0.2453

The flat table is intentional: those runs used the same local-heuristic baseline. The low citation accuracy is a measured improvement target, not something hidden behind a polished demo.

Repository Map

clauseguard/
  ingestion/       EDGAR fetch, parsing, chunking
  retrieval/       embeddings, Qdrant indexing, reranking
  agents/          typed LangGraph nodes and graph definition
  eval/            golden set, metrics, RAGAS adapter, reports
  api/             FastAPI app, jobs, auth, metrics
  frontend/        Streamlit app and frontend helpers
  observability/   JSONL tracing primitives
tests/             offline unit tests
docs/architecture.md

Known Limitations

• Local demo mode uses deterministic graph fallbacks unless a model gateway is configured.
• The built-in job queue is local and should be replaced by Celery/Redis or a managed queue for multi-worker deployment.
• HTML filings do not always provide page/bounding-box anchors; scanned PDFs are stronger for page-image citation review.
• RAGAS and LangSmith integrations are wired but require external credentials and judge-model configuration.

Next Improvements

• Add provider-backed model clients behind ModelGateway.
• Improve retrieval recall and citation accuracy before prompt tuning.
• Expand the golden set by sector, filing form, and adversarial pattern.
• Add CI eval gates and publish trend reports as build artifacts.
• Replace local queue/metrics with production infrastructure for deployment.