ml-observability

Free Open-source ML observability course for data scientists and ML engineers. Learn how to monitor and debug your ML models in production.

A comprehensive solution for monitoring your AI models in production

A Python client to interact with Arize API

A Java client to interact with Arize API

Example projects for Arthur Model Monitoring Platform

Autonomous multi-agent system for ML infrastructure reliability. Detects, diagnoses, and remediates production ML incidents through a coordinated Sentry → Sleuth → Medic → Scribe agent pipeline. Built with Google ADK + Gemini, FastAPI, and a live failure-injection demo harness.

Enterprise-grade machine learning observability platform that detects data drift, concept drift, and performance degradation in production models. Features statistical drift detection (KS test, PSI), real-time alerting, Redis caching, and FastAPI backend.

Autonomous ML supply chain guardian on the DataHub metadata graph, finds every model in the blast radius of an upstream change, files evidence-backed incidents, blocks bad deploys, and remembers through the graph.

Autonomous AI agents on DataHub that catch silent data changes breaking production ML models — trace lineage, measure the AUC hit, open a fix PR, and write the warning back via the MCP Server.

Tamper-evident governance evidence bundles for AI systems: signed, hash-chained attestations an external auditor can verify offline.

Local-first ML observability — track runs, metrics, and environments without a cloud backend.

Your model didn't break. Your data did — three hops upstream, three days ago. An agent that root-causes silent ML failures through DataHub's lineage, and knows when not to blame the pipeline.

Detect and explain ML model regression: metric degradation, drift, and segment analysis with bootstrap statistics. Python + Streamlit, fully offline.

Top Model Monitoring (Opensource) 🌟 Star if you like it! 🌟

Scheduled RAG eval-drift harness: re-runs eval suites on a cadence, flags quality decay with evidence

CPU-first framework to compare ML decision-system versions, identify changed decisions, and attribute shifts to features, models, thresholds, rules, and interactions.

Sentinel — Production ML Observability Platform MLOps · Model Drift Detection · Real-time Anomaly Monitoring · Full-stack A production-grade platform that monitors deployed ML models in real time — detecting data drift, concept drift, and prediction anomalies — visualised through a live dashboard with alerting. No hardware needed.

Fork of Evidently AI (monitoring/observability for ML): fixes for open issues in Charan's AI-governance fleet

Detects ML features whose upstream warehouse columns were renamed, dropped, or changed type — using DataHub's schema and ML lineage metadata via the MCP Server.

Evidence-based ML release-gate agent that catches cross-pipeline target and temporal leakage using DataHub column-level lineage