Skip to content
tstaplerPublic

About

Advisory, diff-aware code/doc quality checks for AI agents, CI, and local dev — PostToolUse checks with git-baseline and grace-period false-positive suppression

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Repository files navigation

kibitzer logo

kibitzer

Advisory, diff-aware code and doc checks built for how AI agents actually edit — locally, in CI, or wired into an agent's hooks. Runs a set of built-in checks (e.g. Go primitive-obsession, Markdown link integrity) against only the files that actually changed, with a caching daemon to keep repeat checks fast. Ships as a CLI, an MCP server, and a Claude Code PostToolUse hook — Claude Code is the first integration, not the ceiling.

Install

# Homebrew (once the tap is set up — see dist-workspace.toml)
brew install tstapler/tap/kibitzer

# From source
cargo install --path .

Then wire up the PostToolUse hook — kibitzer install merges it into an existing settings.json rather than overwriting it, and is idempotent (safe to re-run):

kibitzer install             # this project only: <cwd>/.claude/settings.json
kibitzer install --global    # every project: ~/.claude/settings.json
kibitzer install --dry-run   # preview the resulting file without writing it

Usage

kibitzer run [dir] --trigger batch   # batch mode: check every file under dir
kibitzer install [--global] [--dry-run]  # wire the PostToolUse hook into settings.json
kibitzer hook                        # Claude Code PostToolUse hook, reads event off stdin
kibitzer mcp                         # serve as an MCP server over stdio
kibitzer mcp --socket [path]         # one shared MCP server on a Unix socket (see below)
kibitzer mcp --connect [path]        # stdio MCP bridge to that shared server
kibitzer lsp                         # serve as an LSP server over stdio (diagnostics) (see docs/lsp.md)
kibitzer daemon start|stop|status    # background daemon that caches check results
kibitzer check native primitive-obsession <file>   # run a single built-in check directly
kibitzer check native go-blank-imports <file>      # flag unjustified `import _ "pkg"`
kibitzer check native go-ignored-error <file>      # flag `result, _ := f()` discards
kibitzer check native go-error-context <file>      # flag bare error passthroughs (advisory)
kibitzer check native syntax-rules <file>          # run the native syntactic rule catalog (see docs/syntax-rules.md)
kibitzer check native markdown-link-integrity <file> # flag broken markdown reference links/anchors
kibitzer check native file-complexity <file>       # flag a Go file with several functions over a cyclomatic-complexity threshold
kibitzer check duplicates <dir>      # cross-file duplicate-code detection, repo-wide (batch)
kibitzer check list                  # list all natively implemented checkers
kibitzer schema [--out <file>]       # emit .kibitzer/inspect.json's JSON Schema

New to kibitzer? See docs/getting-started.md for a walkthrough of wiring it into a repo end to end. See docs/checking-invocations.md for how checks are wired up.

One shared MCP server

By default each MCP client spawns its own kibitzer mcp over stdio. To serve many clients from one long-lived process instead, run a shared server on a Unix socket and register the stdio bridge with each client:

kibitzer mcp --socket          # $XDG_RUNTIME_DIR/kibitzer/mcp.sock by default
claude mcp add kibitzer -- kibitzer mcp --connect

Both flags take an optional path (--socket <PATH>, --connect <PATH>). The server speaks MCP streamable HTTP at /mcp over the socket, but Claude Code's HTTP transport takes a URL and can't reach a Unix socket. That's what --connect is for: a stdio MCP server that forwards each message to the socket and loads no checks itself. When the server has dropped the bridge's session (idle for 2 hours, or restarted), the bridge opens a new one on its own. If the server stops responding (stopped or wedged), the bridge doesn't hang: each request returns a JSON-RPC error after a timeout (5s to connect, 60s for response headers, 60s with no data on a reply stream; the server pings every 15s), and the bridge's exit waits at most 3s for the session close.

  • Who can connect: only processes running as your user. The socket is 0600, it sits in a directory only you can write (created 0700), and the server drops any connection whose peer uid isn't its own. In the other direction, --connect refuses a server that isn't running as your user, so nobody squatting the socket path can feed it tool results. There's no other authentication. Any process running as you can use every tool, so it can make kibitzer run a command check from an inspect.json it controls. That's no more access than such a process already has, since it could run kibitzer mcp (or the command) itself.
  • Without a path, the socket is $XDG_RUNTIME_DIR/kibitzer/mcp.sock. If XDG_RUNTIME_DIR is unset, relative or untrusted, it's kibitzer/mcp/mcp.sock under $XDG_CACHE_HOME, or under ~/.cache if that's unset. Relative values are skipped. If no absolute candidate exists, both commands exit with an error and you have to pass a path. The server won't start if another server owns the path or answers on it. It replaces a stale socket, but never a file that isn't a socket.
  • Each MCP session gets its own server instance, so sessions don't share an architecture-model cache.
  • Ctrl-C or SIGTERM stops it. It waits up to 5 seconds for open streams, then removes the socket.
  • Unlike stdio mode, the server's working directory doesn't follow each client's project. The one cwd-sensitive behavior is test-file detection: it ignores test/, spec/, … components only in the part of a path above the server's cwd. A repo checked out under such a directory (say ~/test/myrepo) has every file treated as a test file unless the server was started from inside that directory.

inspect.json schema

kibitzer schema [--out <file>] prints the JSON Schema for .kibitzer/inspect.json, generated from config::Config's own type definitions (so it can't drift from what kibitzer actually parses) — each field's doc comment becomes its schema description, covering scope glob syntax, triggers, {file}/{changed_lines} substitution, and everything else in Check/ArchitectureConfig. A generated copy is checked in at schema/inspect.schema.json; reference it for editor autocomplete/validation:

{
  "$schema": "https://raw.githubusercontent.com/tstapler/kibitzer/master/schema/inspect.schema.json"
}

Migrating off markdownlint-cli2 / scripts/doc_report.py

If your .kibitzer/inspect.json currently shells out to markdownlint-cli2 and/or a project-local scripts/doc_report.py for reference-link/anchor checking (see docs/markdown-link-integrity-false-positives.md for the whole-file false-positive issues that setup has), replace those command entries with a single native checker entry — no npm install, no bespoke Python script:

// Before
{
  "checks": [
    {
      "name": "markdown-link-integrity",
      "command": "markdownlint-cli2 {file}",
      "severity": "blocking",
      "scope": ["**/*.md"],
      "triggers": ["PostToolUse", "batch"]
    },
    {
      "name": "doc-structure-report",
      "command": "python3 scripts/doc_report.py",
      "severity": "blocking",
      "scope": ["**/*.md"],
      "triggers": ["PostToolUse", "batch"]
    }
  ]
}
// After
{
  "checks": [
    {
      "name": "markdown-link-integrity",
      "checker": "markdown-link-integrity",
      "severity": "blocking",
      "scope": ["**/*.md"],
      "triggers": ["PostToolUse", "batch"],
      "message": "broken markdown link/anchor"
    }
  ]
}

The native checker replaces both — it covers what doc_report.py checked (dangling reference-style uses, unused reference definitions) plus dead heading anchors, and it runs in-process rather than shelling out. You can drop markdownlint-cli2 and scripts/doc_report.py entirely once this is wired up. Reference-use-before-definition across a multi-step edit is handled by a grace period, not by disabling the check — see the module doc comment on MarkdownLinkIntegrityChecker in src/markdown_link_integrity.rs.

Diff-aware scoping

When kibitzer knows which lines an edit touched (the PostToolUse hook, not batch mode), a check in .kibitzer/inspect.json can opt into scoping its results to just those lines, two ways:

  • {changed_lines} in command: substituted with a comma-separated list of 1-indexed, inclusive start-end ranges (e.g. 12-15,40-40), for linters that support scoping their own scan.
  • Automatic output filtering: if a check's command emits {file}:{line}: message-style lines (most linters do), kibitzer filters them down to the changed ranges and recomputes pass/fail from what survives — no command changes needed.

A failure kibitzer determines predates the current edit is reported as advisory rather than blocking, even for a check configured as blocking. For a per-file check (one with {file} in command), this means the same violation is also present in git show HEAD:<file>. For a whole-repo check (no {file} placeholder — it scans the whole tree), kibitzer snapshots the tree at HEAD (via git archive) and reruns the check against that snapshot to make the same determination.

Structured output formats

A command check's output_format field tells kibitzer to parse its stdout as a known structured shape (currently SARIF) instead of relying solely on the exit code, so severity and finding counts survive into the reported output. See docs/output-formats.md.

Architecture checks

An architecture_checker entry runs a native, whole-repo, cross-file check against an import graph built once per invocation (Go and TS/JS import extraction), instead of a command or a per-file checker:

{
  "checks": [
    {
      "name": "import-cycles",
      "architecture_checker": "import-cycles",
      "severity": "advisory"
    }
  ]
}

architecture_checker is mutually exclusive with command/checker. Because building the import graph means walking the whole repo, architecture_checker checks may only declare triggers: ["batch"] (or omit triggers — batch and the MCP tool's implicit invocation are the only callers) — kibitzer rejects config that lists PostToolUse or any other per-edit trigger for one of these checks, to avoid rescanning the whole import graph on every file edit. kibitzer check list doesn't cover these (that command lists per-file Checkers only); the registered architecture checkers are in src/architecture_checks.rs::registry().

For Go, import-cycles models each package's test build separately: package X_test files are left out of the graph, and a cycle through _test.go imports is reported only when all of those imports come from one package's own test build (Go's "import cycle not allowed in test"), never by combining test imports of different packages.

Two more architecture checkers ship alongside import-cycles:

  • layering flags an import edge that runs from a later-declared layer back into an earlier-declared one. Declare the layer order, highest-level first, in a top-level architecture section of .kibitzer/inspect.json:
    {
      "architecture": {
        "layers": ["handlers", "domain", "infra"]
      },
      "checks": [
        { "name": "layering", "architecture_checker": "layering", "severity": "advisory" }
      ]
    }
    A package/module belongs to the first declared layer whose name exactly matches one of its /-separated path segments; packages matching no declared layer are ignored. With the layers above, infra is expected to depend on domain/handlers, but an import from infra back into domain is flagged. If architecture.layers is empty (the default), layering finds nothing.
  • coupling flags a package/module whose fan-out (distinct packages it imports) or fan-in (distinct packages that import it) exceeds a fixed threshold of 10, mirroring the fixed-then-configurable threshold pattern long-function already uses (see docs/syntax-rules.md) — no separate config field yet.

kibitzer mcp also exposes an architecture_assessment tool: given a repo path and an optional scope glob, it runs every configured architecture_checker check plus the native syntax-rules* complexity checkers across every in-scope file, and returns a combined findings report (one [level] {file}:{line}: {message} line per finding) with canned recommendations for import-cycles/layering findings appended under a ## Recommendations heading. When include_diagram is true (the default), a ## Dependency graph heading follows with a Mermaid graph TD diagram of the import graph — nodes are packages/modules, edges are import relationships, and any node/edge that's part of an import cycle is highlighted in red. Repos with more than 150 nodes fall back to a text note instead of an unreadably large diagram; pass a narrower scope to render a subgraph.

cargo test --workspace
cargo clippy --workspace --all-targets -- -D warnings
cargo fmt --all --check

CI (.github/workflows/ci.yml) runs the same three commands on every push and pull request. Releases are tag-triggered and handled by .github/workflows/release.yml (generated by cargo dist) plus cliff.toml for changelog generation.

License

MIT — see LICENSE.

About

Advisory, diff-aware code/doc quality checks for AI agents, CI, and local dev — PostToolUse checks with git-baseline and grace-period false-positive suppression

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages