Skip to content
@probityai

Probity

Open source tools for checking AI agent actions: evidence formats, tests, observation and deployment policy.

Probity

Evidence about AI agent actions that anyone can check again from the bytes.

agent-evidence-vectors on PyPI probity-verify on PyPI jcs-admit on crates.io dsse on crates.io

An intact archive verifies, a changed byte fails, a missing artifact is not established, and a changed verifier keeps both outcomes

Replay a public conformance corpus through the bundled reference verifier, with nothing to clone. Python 3.11+ and uv are enough:

uvx agent-evidence-vectors==0.18.0

Add --verifier "<your command>" to test your own verifier against the same records.

Check any verifier of signed AI-agent and supply-chain records against a public corpus. Choose a tool by task.

Explore retained runs or contribute one in the Open Evidence Lab. Browse the task catalog in HTML or Markdown. Read the agent guide for commands and pinned sources. Use the replay procedure to run a published example and keep its evidence.

Choose a tool

Task Project
Choose a tool or inspect recorded claims and retained runs Agent Evidence Conformance Board
Sign and verify DSSE envelopes in Rust dsse
Reject duplicate JSON keys and canonicalize in Rust jcs-admit
Record file actions or check native delegation evidence Probity Observer
Test a verifier against conformance cases Agent Evidence Vectors
Install a pinned verifier and check an evidence claim Probity Verify
Use shared definitions for agent evidence claims Probity Vocabulary
Run a checkout composition and replay its retained claims Checkout composition reference
Check artifact digests or evaluate admission policies Probity Admission
Check an agent guard against replay and regression cases Probity Ratchet

Prove an agent harness change

Ratchet turns each observed agent failure into an executable guard. It proves the guard by replay against recorded history and fresh held-out runs before admitting it, and signs every admission decision with the agent change admission predicate. Install it with pip install probity-ratchet==0.1.0; its README gives the commands and the recorded results.

Current releases

Find release states, source revisions and pinned install commands in the Atlas project catalog.

Try a worked task:

Each project explains its supported commands and examples in its README. Verify links to source-pinned installation and checked task recipes. Admission includes policy-engine rules and a pre-call budget API and CLI for bounded local workloads. Retained runs keep source revisions, readers, operators and rerun instructions with the result.

Pinned Loading

  1. agent-evidence-vectors agent-evidence-vectors Public

    Conformance corpora and reference verifiers for AI-agent and supply-chain evidence formats

    Python 6 3

  2. agent-evidence-vocabulary agent-evidence-vocabulary Public

    Versioned terms and crosswalks for the claims carried in signed AI-agent and supply-chain records.

    Python 3 2

  3. agent-evidence-admission agent-evidence-admission Public

    OPA, Kyverno and Sigstore policy-controller rules that admit signed AI-agent execution evidence against a consumer-pinned policy.

    Python 2

  4. probity-verify probity-verify Public

    Checks supplied evidence bytes for signed AI-agent records offline, through claim-specific adapters.

    Python 3

  5. agent-evidence-atlas agent-evidence-atlas Public

    Maps assurance claims about AI agents to the evidence that can check them, with an open run register for verifier results.

    Python 3

  6. agent-evidence-observer agent-evidence-observer Public

    Records brokered file requests from AI agents and includes an offline reader for selected native delegation runs.

    Python 2 1

Repositories

Showing 10 of 15 repositories

People

This organization has no public members. You must be a member to see who’s a part of this organization.

Top languages

Loading…

Most used topics

Loading…