FinanceBench evaluation of Mafin 2.5 (Powered by PageIndex)
-
Updated
Oct 20, 2025 - Python
FinanceBench evaluation of Mafin 2.5 (Powered by PageIndex)
Advanced RAG pipelines for medical (HealthBench, MedCaseReasoning, MetaMedQA, PubMedQA) and financial (FinanceBench, Earnings Calls) QA. LangGraph orchestration + BAML structructed generation, Milvus Hybrid search (Dense + BM25 + RRF), three-layer Metadata Enrichment, Contextual AI instruction-following reranker, and DeepEval evaluation.
Multi-agent LangGraph RAG for financial Q&A — 72.7% on FinanceBench under κ=0.932 calibrated judge. RBAC at the vector layer, multi-party HITL on high-stakes answers, self-hosted LLM observability. pip install financebench-rag-agent
Finance-grade tool-calling agent over SEC filings for any US public company: exact XBRL numbers (never LLM arithmetic), every claim cited, abstains instead of fabricating. Validated on FinanceBench (44%→87% coverage, near-zero hallucination) with a 3-layer eval and a domain judge calibrated to human labels (κ=0.76). LangGraph + pgvector.
🔍 Empower efficient retrieval with PageIndex, a reasoning-based system that eliminates the need for vector databases and chunking for human-like results.
Measuring RAG Groundedness on Financial Filings
FinanceBench financial-report RAG with FAISS, Flask, Ragas, and financial numeric accuracy evaluation
Grounded Q&A over financial filings (10-Ks/10-Qs): routes to the right document, decomposes the question, retrieves, and answers with page citations — or abstains when unsure. Simple-agentic RAG; FinanceBench hardest subset: 0.71 corpus / ~0.79 per-doc vs 0.19 naive.
A filings analyst that cites the page behind every figure, scored on FinanceBench. Retrieval measured; model runs next.
a deliberately challenging benchmark built around real financial filings, where retrieving the right evidence is just as important as generating the answer.
Production RAG over SEC filings, built and tuned through evaluation on FinanceBench
Production RAG pipeline on FinanceBench: hybrid BM25+dense+RRF+Cohere Rerank, Anthropic Contextual Retrieval, conformal abstention (Yadkori 2024), Citations API grounding, triple eval (RAGAS+HHEM+DeepEval). Every eval number is reproducible and CI-gated.
RAG pipeline for FinanceBench with retrieval, evaluation, improvement cycles, and chunk-size experiments.
An end-to-end RAG pipeline that answers questions about SEC 10-K filings with grounded citations instead of hallucinated numbers. Built on FinanceBench: indexing with FAISS + BGE embeddings, generation with Llama-3.3-70B, three-axis evaluation (correctness / faithfulness / page-hit@k), improvement cycles.
Research framework for long-document financial QA using MapReduce and benchmark-driven evaluation, with a multi-document web app.
RAG over SEC 10-K/10-Q filings evaluated on FinanceBench: retrieval ablation, cited answers, LLM judge, citation check, MCP server
RAG benchmark lab on FinanceBench filings: chunking, hybrid retrieval, reranking and HyDE measured on evidence-page recall
Rigorous evaluation of contextual retrieval techniques on FinanceBench: comparing 5 embedders × 4 chunking strategies with bootstrapped confidence intervals on FinMTEB and FinanceBench.
Five RAG architectures built from scratch in plain Python — Naive, Hybrid, Agentic, Graph & production-grade Advanced — over real 10-K filings, evaluated on FinanceBench. No LangChain, no API keys, 100% local.
SEC dosyaları üzerinde ölçülmüş finans RAG hattı: e5 + belge yönlendirme + reranker + Gemini. Ön kayıtlı deneyler, kilitli test, olumsuz sonuçlar dahil.
To associate your repository with the financebench topic, visit your repo's landing page and select "manage topics."