evalops

Agent PM: OpenAI Agents-powered product management orchestrator with automated PRDs, tickets, and comms.

Towards Evaluation Engineering: An Empirical Study of ML Evaluation Harnesses in the Wild

Reliable infrastructure for agentic knowledge-work evals: traces, artifact previews, model adapters, red-team scanners, and eval-quality audits.

AI Agent EvalOps dashboard for regression testing, drift detection, run comparison, and quality analysis.

CI-ready LLM regression harness for document-grounded RAG assistants: hard checks, pairwise judging, bootstrap CIs, and artifact reports.

Evidence-aware benchmark and PR advisory prototype for AI coding-agent context safety.

RAG pipeline evaluation for .NET — LLM-as-judge scoring for Faithfulness, Answer Relevance, Context Precision, and Context Recall. The evaluation layer your .NET RAG pipeline is missing.

EvalOps platform for continuous AI-quality evaluation

LLM gateway with explainable routing, streaming failover boundaries, budget reservations, and evaluation regression gates. Zero-dependency Python demo.

本地可复现的 AI 输出质量评测与发布门禁：固定任务集、确定性检查、四维裁判（仅建议）、版本回归与证据包

Hermes plugin: route LLM calls through the EvalOps gateway and report agent registration + tool spans to the EvalOps platform.