diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 254a42a..05c6409 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -36,10 +36,9 @@ a documentation update covers a code change, and it does not rewrite content. Unresolved publication placeholders warn during normal checks. Before a release, run `python scripts/check_docs.py --release` to fail if any remain in reader-facing content. Version-tag builds and the workflow's release-readiness option run the -same strict check. The current missing hosted URL and two screenshots must be -resolved before that check can pass. +same strict check. Read the [contributing guide](docs/development/contributing.mdx) for CI behavior, or [add a trace regression case](docs/guides/regression-checks.mdx) to document -expected healthy and stuck behavior. See [the maintainer handoff](docs/README.md) -for local preview and later publishing. +expected healthy and stuck behavior. See [the documentation guide](docs/README.md) +for local preview and publishing. diff --git a/README.md b/README.md index 6ea7708..08bf854 100644 --- a/README.md +++ b/README.md @@ -4,7 +4,7 @@ [![Python versions](https://img.shields.io/pypi/pyversions/loopguard-runtime)](https://pypi.org/project/loopguard-runtime/) [![PyPI monthly downloads](https://img.shields.io/pypi/dm/loopguard-runtime)](https://pypi.org/project/loopguard-runtime/) -[![Docs](https://img.shields.io/badge/Docs-2563EB?style=for-the-badge)](docs/index.mdx) +[![Docs](https://img.shields.io/badge/Docs-2563EB?style=for-the-badge)](https://loopguard.mintlify.site/) [![PyPI](https://img.shields.io/badge/PyPI-3775A9?style=for-the-badge)](https://pypi.org/project/loopguard-runtime/) [![Demo](https://img.shields.io/badge/Demo-FF0000?style=for-the-badge)](https://youtu.be/i5XRrvFoPA0) [![GitHub](https://img.shields.io/badge/GitHub-181717?style=for-the-badge)](https://github.com/mahimathacker/loopguard) @@ -33,8 +33,7 @@ the actual decision. [![LoopGuard catching a real search agent in a semantic loop and interrupting the run](ui/public/realagentloop.png)](https://youtu.be/i5XRrvFoPA0) -An agent keeps searching for an impossible confirmation. LoopGuard flags repeated -intent and interrupts the demo. This repository capture shows an earlier UI layout. +Screenshot from a LoopGuard demo run showing repeated intent and an interrupted run. The [four visual examples](docs/demo/scenarios.mdx) pair detection explanations with Python SDK code so you can inspect the same behavior in your own program. @@ -132,7 +131,7 @@ harness. [Evaluation and its limits →](docs/development/evaluation.mdx) ## Documentation -**Hosted documentation:** [DOCS URL TO ADD AFTER DEPLOYMENT] +**Hosted documentation:** [Read the LoopGuard docs](https://loopguard.mintlify.site/). The [Mintlify source](docs/index.mdx) covers the full developer journey. The [concept guides](docs/concepts/agents-and-loops.mdx) explain agent loops, diff --git a/docs/README.md b/docs/README.md index a75fbc7..17ffe3a 100644 --- a/docs/README.md +++ b/docs/README.md @@ -1,306 +1,90 @@ -# Maintaining the LoopGuard docs +# LoopGuard documentation -The Mintlify site lives in this directory. Start reading at [Introduction](index.mdx). -This file is a maintainer handoff, not a published docs page; `.mintignore` excludes it. +The published site is [loopguard.mintlify.site](https://loopguard.mintlify.site/). +Mintlify builds from this repository's `docs/` folder on `master`. Update the SDK +and its documentation in the same pull request when behavior changes. ## Preview -Use Node.js 20.17+ and run the current official `mint` CLI from `docs/`: +Use Node.js 24 and the Mintlify version pinned in CI. From the repository root: ```bash cd docs -npx --yes mint dev --port 3001 --no-open +npx --yes mint@4.2.1178 dev --port 3001 --no-open ``` -Open http://localhost:3001. Port 3001 leaves port 3000 free for the LoopGuard demo UI. -The preview stays local. Mintlify may require a login to activate local search; -reading the preview does not require deploying it. +Open http://localhost:3001. Port 3000 remains available for the demo UI. ## Validate -From `docs/`: - -```bash -npx --yes mint validate -npx --yes mint broken-links --check-anchors -npx --yes mint a11y -``` - -Use the `mint` package, not the legacy `mintlify` npm package. The CLI follows -[Mintlify's current command reference](https://www.mintlify.com/docs/cli/commands). -Its generated cache and installed tooling are not source files to commit. - -From the repository root, also run: +With the framework dependencies installed, run from the repository root: ```bash python -m unittest discover -v -python scripts/check_docs.py +python scripts/check_docs.py --release python scripts/verify_doc_examples.py -python examples/first_run.py -python -m loopguard.ingest examples/sample_trace.json --json -python -m loopguard.check examples/sample_trace.json --json -``` - -Install the framework extra to run `first_run.py`. The sample trace check should -exit `1`; it deliberately contains a loop. Provider-backed scenarios need separate -credentials and model access. Do not interpret offline fixture scores as production -accuracy. - -## Navigation - -```text -Start here - Introduction - Getting started - How LoopGuard works -Concepts - AI agents and loops - Tests, traces, and evals - Agent trajectories and convergence -Use LoopGuard - Detecting stuck agents - LangGraph and LangChain - Configure LoopGuard - Analyze saved traces - Add a trace regression check -Explore the demo - Demo scenarios - Run the demo UI -Test and troubleshoot - Evaluation and tests - Troubleshooting - Project status - Contributing and docs maintenance -Reference - Python API - CLI reference - Configuration reference +cd docs +npx --yes mint@4.2.1178 validate +npx --yes mint@4.2.1178 broken-links --check-anchors ``` -Every public `.mdx` page belongs in `docs.json`. Use root-relative site links such -as `/guides/configuration`. README links at the repository root use real file paths -so they work on GitHub before hosting exists. - -## Automated maintenance - -See [the contributing guide](development/contributing.mdx) for the CI behavior -and local equivalents. The three workflows live in `.github/workflows/`: +See [Contributing](../CONTRIBUTING.md) for dependency installation and CI details. +The strict documentation check rejects unresolved publication placeholders. -- `documentation.yml`: tests, offline docs checks, README example execution, - Mintlify build, and internal link checks on PRs and pushes. -- `docs-drift.yml`: a file-based PR warning and summary for developer-facing changes - without Markdown updates. It does not rewrite docs or prove coverage. -- `external-links.yml`: a weekly/manual Lychee check of public documentation URLs. - Its schedule takes effect after the workflow reaches the default branch. +## Documentation maintenance -`python scripts/check_docs.py --release` fails on the current hosted-URL placeholder -and two screenshot placeholders. Normal checks warn so in-progress docs can be -reviewed. Version tags and manual release-readiness runs use the strict check. -These checks do not publish the docs. Require the relevant status checks in branch -protection if they should block merging. +- Pull requests and pushes run Python tests, documentation validation, internal + link and image checks, and executable callback examples. +- The drift workflow flags developer-facing changes without related documentation + changes for human review. It does not rewrite documentation. +- The weekly external-link workflow checks public HTTP URLs with retries and + reports failures in the job summary. It can also run manually. -## Agent discovery +The workflows are in `.github/workflows/`. Required-check settings in GitHub +control whether a failing check blocks merging. See +[documentation maintenance](development/contributing.mdx) for the commands, +workflow behavior, and failure investigation. -Mintlify generates `/llms.txt`, `/llms-full.txt`, and `.md` page exports from the -site content. We keep these generated so links and page contents follow the -navigation. SDK guidance in `docs.json` under `markdown.instructions` is included -in the agent-facing exports. See [Mintlify's export documentation](https://www.mintlify.com/docs/ai/markdown-export). +## Publishing and discovery -The checked-in `robots.txt` allows crawling. `seo.metatags.robots` permits page -indexing. Mintlify generates `/sitemap.xml`; after confirming the public domain, -add its absolute URL as a `Sitemap:` directive to `robots.txt`. +Mintlify's Git source must be `mahimathacker/loopguard`, branch `master`, content +directory `docs`. After changes merge, verify the deployment and public pages. +Use local preview and CI for review; hosted PR previews depend on the Mintlify plan. -After deployment, verify `/robots.txt`, `/sitemap.xml`, `/llms.txt`, -`/llms-full.txt`, and `/getting-started.md` on the actual docs domain. Confirm that -the index covers all navigation pages and that Markdown exports include the SDK -guidance. These generated routes return 404 in the local preview used here. -The dashboard's **Don't index project** setting must be off for public discovery. -Discovery files help tools find content; they do not guarantee indexing or citation. +The Docs button and hosted link in the root README, the canonical URL in +`docs.json`, and the sitemap directive in `robots.txt` use +`https://loopguard.mintlify.site`. Keep them aligned if the domain changes. -## Ask in an external AI tool +Mintlify generates `llms.txt`, `llms-full.txt`, Markdown page exports, and the +sitemap from the published content. Keep agent guidance in +`docs.json` under `markdown.instructions` rather than maintaining separate copies. +After deploying, verify: -`docs.json` enables Mintlify's built-in `chatgpt`, `claude`, `aistudio`, `perplexity`, -`grok`, `copy`, and `view` context-menu actions in the page header. Google AI Studio -is the supported entry point for Gemini. The first action is the primary button; -the others appear in its dropdown. This setup uses external AI tools rather than -Mintlify's paid embedded assistant. No model key or chat backend is added here. +- [Documentation index](https://loopguard.mintlify.site/llms.txt). +- [Full documentation export](https://loopguard.mintlify.site/llms-full.txt). +- [Getting-started Markdown](https://loopguard.mintlify.site/getting-started.md). +- [Sitemap](https://loopguard.mintlify.site/sitemap.xml). +- [Crawler rules](https://loopguard.mintlify.site/robots.txt). -These actions provide the current page as context, not an automatically indexed -copy of every repository file. The introduction explains how to include the -published `llms.txt` or `llms-full.txt` for broader questions and how to copy a page -when fetching is unavailable. The user's external AI account and its limits apply. +The exported documentation links must use the current public domain. -After deployment, check the links on the real docs URL and confirm that each -provider can retrieve the page. Local preview links can be inspected, but external -tools cannot fetch `localhost`. See -[Mintlify's contextual menu documentation](https://www.mintlify.com/docs/ai/contextual-menu) -for the supported actions. +## Screenshots -## Screenshot provenance +The screenshots are real demo captures from `ui/public/`: -All four images were copied unchanged from `ui/public/`; no screenshots were generated. - -| Original | Docs image | Used on | +| Source | Documentation asset | Example | | --- | --- | --- | -| `realagentloop.png` | `images/stuck-agent.png` | Introduction and demo scenarios; the root README uses the original. | -| `scriptedtool.png` | `images/exact-loop.png` | Exact scenario and local demo setup. | -| `scriptedopenai.png` | `images/semantic-loop.png` | Semantic scenario. | -| `realagentmath.png` | `images/healthy-calculation.png` | Healthy calculation scenario. | - -These captures show an older UI layout. Captions disclose that; they do not claim -that current thresholds always stop at the counts displayed in those screenshots. -Each of the four visual scenarios includes a runnable Python SDK example and an -explanation of the messages behind the UI. The screenshots illustrate SDK behavior; -the UI is one way to inspect it. - -## Screenshot capture checklist - -Two new captures complete the healthy-recovery versus permanent-failure comparison. -Both placeholders are on `demo/scenarios.mdx`. - -First follow [local demo setup](demo/running-locally.mdx). With the Python environment -active, run the backend from the repository root: - -```bash -uvicorn server:app --reload --port 8000 -``` - -In a second terminal, from the repository root: - -```bash -cd ui -npm install -npm run dev -``` - -Configure one chat provider in the root `.env`. Neither capture needs embeddings. -Open http://localhost:3000 and use the scenario selector; CLI commands alone do not -populate the browser UI. - -1. **503 recovery:** choose **Controlled ReAct: 503 -> success**, select **Run**, - and wait for completion. Capture the scenario name, decision/verdict, graph, - and event log containing both `503 temporarily unavailable` results and the - later success. Save the real capture as `images/controlled-503-recovery.png`. -2. **401 failure:** choose **Controlled ReAct: auth failure**, select **Run**, - and wait for `STOP`. Capture the selected scenario, first 401 result, stop - reason/decision, and red graph together. The demo stops on the first observed - permanent failure, so do not force additional failures. Save the real capture - as `images/controlled-401-failure.png`. - -Replace each visible screenshot placeholder with a `Frame` and an image with clear -alt text after the file exists. Use a viewport wide enough to show the event log; -avoid clipping the later success out of the recovery capture. - -## Items to verify before release - -- Provider model access: defaults come from source; no credential-backed run is - implied by the offline checks. Gemini chat and embedding defaults are separate. -- Your agent's error handling must propagate `LoopGuardInterrupt`; a recorded - `STOP` alone does not prove execution stopped. -- The published 0.2.1 wheel was inspected: Gemini support, controlled demo scenarios, - and additional serialized decision display fields require the source checkout. -- Add the two captures and verify local search after Mintlify login if needed. -- Replace `[DOCS URL TO ADD AFTER DEPLOYMENT]` in the root README only when a real - hosted URL exists; point the Docs button to it at the same time. - -## Verification completed - -- All 72 runtime and documentation automation tests passed with framework dependencies installed. -- The first-run example passed against both the source checkout and the actual - published 0.2.1 wheel. -- A real LangGraph `ToolNode` run propagated the stop exception when configured - to let tool errors escape. -- Mintlify full validation/build and internal link checks passed. -- All 20 pages rendered successfully in Chrome with no broken images. The - introduction, getting-started page, demo scenarios, config reference, and all - three concept pages, plus the two new practical guides, were checked at a 390px - viewport without document overflow. -- Mintlify found alt text on every image. Brand colors meet the check's AA/minimum - contrast thresholds; its optional AAA suggestion remains a warning. -- Python code blocks parsed, JSON/YAML samples loaded, and nine trace/example - command variants returned the expected exit codes. -- The convergence example printed `0.58`; the equal-score and nonpositive-count - limitations described in the concept guide were checked against the helper. -- The local backend's runtime and graph routes and the exact WebSocket run were - checked without making provider calls. -- Workflow syntax passed actionlint. The external-link checker passed against - the actual public links. Strict release checking correctly failed on the three - remaining reader-facing publication placeholders. -- ChatGPT, Claude, Google AI Studio, Perplexity, and Grok context-menu actions - were checked at desktop and mobile widths, including the generated page URLs - and copy/Markdown alternatives. - External chat navigation was intercepted during verification; retrieval of a - published page still needs a deployment check. - -Provider-backed scenarios were inspected in source, not run against paid APIs. -Local search requires Mintlify login and was not activated. - -## Files in this change - -```text -README.md -CONTRIBUTING.md -.lychee.toml -.github/workflows/ - documentation.yml - docs-drift.yml - external-links.yml -scripts/ - check_docs.py - check_docs_drift.py - verify_doc_examples.py -tests/ - test_docs_automation.py -examples/ - first_run.py - langgraph_callback_demo.py -docs/ - .mintignore - README.md - docs.json - robots.txt - index.mdx - getting-started.mdx - how-it-works.mdx - detection.mdx - troubleshooting.mdx - concepts/ - agents-and-loops.mdx - tests-traces-and-evals.mdx - trajectory-and-convergence.mdx - guides/ - integration.mdx - configuration.mdx - saved-traces.mdx - regression-checks.mdx - demo/ - scenarios.mdx - running-locally.mdx - development/ - evaluation.mdx - status.mdx - contributing.mdx - reference/ - python.mdx - cli.mdx - configuration.mdx - images/ - stuck-agent.png - exact-loop.png - semantic-loop.png - healthy-calculation.png -``` - -## Publish later +| `scriptedtool.png` | `images/exact-loop.png` | Repeated identical calls with empty results. | +| `scriptedopenai.png` | `images/semantic-loop.png` | Repeated intent across different search phrases. | +| `realagentmath.png` | `images/healthy-calculation.png` | A successful calculation. | +| `realagentloop.png` | `images/stuck-agent.png` | An interrupted search for an impossible goal. | -No hosting setup or deployment is part of this change. After content review and -approval to publish: +Keep captions tied to the behavior shown. Add descriptive alt text when replacing +an image, and verify it in local preview. -1. Connect `mahimathacker/loopguard` to your Mintlify project. -2. Choose the reviewed branch and set the documentation/content path to `/docs`. -3. Preview the build, then publish using the Mintlify dashboard. -4. Add the resulting public URL to the root README. +## Editing pages -See [Mintlify monorepo setup](https://www.mintlify.com/docs/deploy/monorepo) for the -content-path setting. Do not invent a `loopguard.mintlify.app` address before the -project has a confirmed domain. +Every public page belongs in `docs.json`. Use root-relative links such as +`/guides/integration` in MDX. Keep source links in repository Markdown relative +to the file containing them. `.mintignore` excludes this tooling guide from the +published navigation. diff --git a/docs/demo/running-locally.mdx b/docs/demo/running-locally.mdx index bee6f85..81b4e7e 100644 --- a/docs/demo/running-locally.mdx +++ b/docs/demo/running-locally.mdx @@ -122,13 +122,10 @@ Open [http://localhost:3000](http://localhost:3000). Choose **Scripted: identica tool loop**, then select **Run**. Watch repeated empty results build evidence until the run stops. - + LoopGuard demo main interface with an agent-tool graph, repeated tool observations, and metrics -This capture predates the current decision panel. Fresh capture instructions -are in the repository's `docs/README.md` maintainer checklist. - ## Change the backend address The frontend defaults to `http://localhost:8000` and derives the WebSocket address diff --git a/docs/demo/scenarios.mdx b/docs/demo/scenarios.mdx index 9ad3dde..1f3aee3 100644 --- a/docs/demo/scenarios.mdx +++ b/docs/demo/scenarios.mdx @@ -3,9 +3,10 @@ title: Demo scenarios description: Run the Python SDK behind the UI examples and compare healthy and stuck behavior. --- -The demo uses the same runner for the command line and the UI. The scripted agents -make detection easy to reproduce; the ReAct scenarios use a real model to choose -tool calls. Controlled scenarios fix the tool responses, not the model's decisions. +Use the UI or Python SDK to compare healthy runs with agents that get stuck. +Scripted agents make detection easy to reproduce; the ReAct scenarios use a real +model to choose tool calls. Controlled scenarios fix the tool responses, not the +model's decisions. The four visual examples below connect what you see to actual Python SDK usage: exact repetition, semantic repetition, a healthy calculation, and an impossible @@ -17,10 +18,10 @@ or run `python main.py SCENARIO` from the repository root. ## Run the visual examples through the SDK -The SDK snippets use `LoopGuard.stream()`, the public wrapper around the runner -used by the demo backend. `get_scenario()` supplies this repository's demo agent, -detectors, and initial input. In your application, pass your own agent and input -or use the [callback integration](/guides/integration). +Use `LoopGuard.stream()` to inspect the same events, signals, and decisions shown +in the UI. `get_scenario()` supplies a demo agent, detectors, and initial input. +In your application, pass your own agent and input or use the +[callback integration](/guides/integration). Run these snippets from the source checkout after [installing the demo dependencies](/demo/running-locally#set-up-the-checkout). @@ -55,6 +56,8 @@ python main.py exact **What happens:** a scripted agent repeatedly searches for the weather in Paris; the tool always returns `no results`. +**Why it matters:** repeated calls become suspicious when the results never change. + **Expected behavior:** exact repetition accumulates, followed by stagnant-result evidence. The policy reaches `STOP`, and the graph turns red in the UI. This scenario uses `LoopDetector` and `StallDetector`, so its stop timing differs @@ -81,7 +84,7 @@ The repeated calls first produce warning evidence. Stagnant output then combines with repetition to reach a `stop` decision; the final `done` message has `interrupted: True`. - + LoopGuard UI showing repeated weather searches, no-results observations, a red graph, and an interrupted verdict @@ -116,7 +119,7 @@ for message in guard.stream(agent, initial=initial): print(message) ``` - + LoopGuard UI displaying a semantic-loop alert, stagnant results, and a stopped paraphrasing search agent @@ -143,17 +146,12 @@ python main.py controlled_503 **What happens:** the tool returns `503 temporarily unavailable` twice and then returns a success message. -**Why it matters:** stopping on the first retry would prevent useful recovery. +**Why it matters:** repetition alone is not a reason to stop. Allowing short +retries lets the agent recover from a temporary failure. **Expected behavior:** retryable-failure evidence can warn, but should allow this short recovery sequence. The model still decides whether to issue the next call. -# [SCREENSHOT NEEDED: CONTROLLED 503 RECOVERY DEMO] - -Capture the completed run with both temporary failures and the later success in -the event log. Include the final verdict and decision panel. Exact capture steps -are in the repository's `docs/README.md` maintainer checklist. - ## Controlled 401 failure ```bash @@ -168,11 +166,6 @@ python main.py controlled_401 the repeated-failure threshold to one, so the first observed auth failure can stop the run. The default live detector uses a threshold of two. -# [SCREENSHOT NEEDED: CONTROLLED 401 FAILURE DEMO] - -Capture the first auth failure, the `STOP` recommendation, the reason, and the red -graph. Do not force additional calls just to make the screenshot look repetitive. - ## Controlled empty results ```bash @@ -181,6 +174,8 @@ python main.py controlled_empty **What happens:** a real model keeps checking a tool that returns `no results`. +**Why it matters:** the agent can change its wording without finding useful information. + **Expected behavior:** repeated or similar actions combined with no progress should reach `STOP`. This scenario includes an embedding-backed detector, even though the tool responses are controlled. @@ -194,6 +189,8 @@ python main.py calc **What happens:** a real agent uses the calculator for `4801 * 379 + 1234` and returns an answer. +**Why it matters:** a normal successful tool call should finish without interruption. + **Expected behavior:** useful output lets the run finish without a fatal loop alert. This is the healthy comparison for the stuck search runs. @@ -215,7 +212,7 @@ for message in guard.stream(agent, initial=initial): print(message) ``` - + LoopGuard UI showing a calculator result, the agent's final answer, and a successful completion verdict @@ -229,6 +226,9 @@ python main.py trap the false claim that the Great Wall is visible from the Moon with the unaided eye. Its goal encourages it to keep refining its search. +**Why it matters:** an impossible goal can keep an agent searching after the +evidence stops helping. + **Expected behavior:** the run can build repeated-intent and no-progress evidence and be interrupted. Real search output and model choices vary; do not treat the recorded demo as a fixed step-count guarantee. @@ -251,11 +251,8 @@ for message in guard.stream(agent, initial=initial): print(message) ``` - + A real search agent caught in a semantic loop while pursuing an impossible goal -The four screenshots above are existing repository captures from an earlier UI -layout. The current UI adds runtime provider information and policy decisions. - [Troubleshoot a demo run](/troubleshooting) · [Understand the policy](/how-it-works) diff --git a/docs/detection.mdx b/docs/detection.mdx index 482bc5c..dc6cf04 100644 --- a/docs/detection.mdx +++ b/docs/detection.mdx @@ -3,9 +3,16 @@ title: Detecting stuck agents description: Recognize repeated actions, stagnant results, failures, cycles, and exceeded budgets. --- -A detector reports evidence. The policy decides what to do with it. The examples -below describe the default behavior; application settings and recent signals can -change the final decision. +Repeating an action can be useful. What matters is whether its results show progress. + +```text +same action + progress -> probably okay +same action + stagnant result -> suspicious +``` + +LoopGuard compares actions with their results. A detector reports evidence; the +policy combines it into a decision. Repetition alone does not force a stop. +Application settings and recent signals can change the final decision. New to the agent execution cycle? [AI agents and loops](/concepts/agents-and-loops) explains why repeated activity can be healthy and when it becomes suspicious. diff --git a/docs/development/contributing.mdx b/docs/development/contributing.mdx index 1cbd72c..db96564 100644 --- a/docs/development/contributing.mdx +++ b/docs/development/contributing.mdx @@ -7,6 +7,10 @@ A useful contribution explains the problem, includes an example, and shows how the change was checked. For detection changes, include both a stuck run and a healthy run that should keep working. +The [published documentation](https://loopguard.mintlify.site/) uses the `docs/` +folder on `master` in `mahimathacker/loopguard` as its Git source. Keep SDK and +documentation changes in the same pull request so they can be reviewed together. + ## Run the local checks From the repository root, with Python 3.11+: @@ -88,8 +92,7 @@ still decides whether the documentation is correct. There is no automated rewrit ## Investigate a weekly external-link failure The schedule is Monday at 08:47 Asia/Kolkata (03:17 UTC), subject to GitHub's -scheduler delays. Scheduled runs use the workflow on the default branch, so the -schedule starts after these changes are merged there. +scheduler delays. Scheduled runs use the workflow on the default branch. Open the failed workflow's job summary and inspect each URL. A 404 may need a new destination. A 403, rate limit, or timeout can mean the remote site blocked or @@ -104,20 +107,18 @@ an exclusion just to get a green check. ## Check publication readiness -Normal PR checks warn about reader-facing hosted-URL and screenshot placeholders -so work in progress remains reviewable. Before publication: +Run the strict documentation check before publishing or tagging a release: ```bash python scripts/check_docs.py --release ``` -The strict check must pass after the hosted URL and required screenshots are real. +The strict check rejects unresolved publication placeholders in reader-facing content. The same check runs for tags beginning with `v` and when **release_ready** is selected in the Documentation workflow's manual inputs. It does not create a release or publish to PyPI or Mintlify. -The maintainer handoff is excluded from publication and can describe the remaining -tasks. Placeholder examples inside code fences do not count as unfinished content. +Placeholder examples inside code fences do not count as unfinished content. ## Add evidence with your contribution diff --git a/docs/docs.json b/docs/docs.json index 32d4380..6f91be3 100644 --- a/docs/docs.json +++ b/docs/docs.json @@ -11,7 +11,7 @@ "instructions": [ "LoopGuard is a Python runtime guardrail SDK. Install loopguard-runtime and import loopguard. Use /getting-started and /guides/integration for application code.", "Use the Concepts section for agent loops, tests versus traces versus evals, and trajectory/convergence. The convergence helper compares counts without checking success; LoopGuard events are not OpenTelemetry spans. Distinguish these general concepts from implemented SDK features.", - "The exact, semantic, calc, and trap UI examples illustrate Python SDK detection behavior. Read /demo/scenarios for runnable SDK equivalents. The demo backend calls stream_run; LoopGuard.stream wraps the same runner.", + "The exact, semantic, calc, and trap UI examples illustrate Python SDK detection behavior. Read /demo/scenarios for runnable SDK examples and /guides/integration to attach callbacks or inspect streamed updates.", "These docs follow the checkout. PyPI 0.2.1 lacks Gemini support, controlled demo scenarios, and additional decision display fields; check /development/status before describing release capabilities.", "Repetition alone does not force STOP. REPLAN and PAUSE are recorded recommendations. Observe-only mode records decisions without raising. Follow /guides/integration for interrupt propagation.", "Use the Python API, CLI, and configuration references for supported names and flags. Screenshots illustrate observed runs, not guaranteed stop counts or benchmark results." @@ -19,7 +19,8 @@ }, "seo": { "metatags": { - "robots": "index, follow" + "robots": "index, follow", + "canonical": "https://loopguard.mintlify.site" } }, "colors": { diff --git a/docs/getting-started.mdx b/docs/getting-started.mdx index 94a0ff9..ece2b4b 100644 --- a/docs/getting-started.mdx +++ b/docs/getting-started.mdx @@ -62,51 +62,29 @@ Decision: stop Tool calls: 3 ``` -LangChain may also log the callback exception. The caught `LoopGuardInterrupt` is -the expected result: three identical calls returned no useful new information. -The exception's wording differs between the published release and the checkout. -The same example is saved in the repository as `examples/first_run.py`. +Three identical calls returned no useful new information, so LoopGuard requests +a stop. The callback raises `LoopGuardInterrupt`, which the example catches. +LangChain may also log the callback exception; that is expected for this run. -`raise_error = True` is a LangChain callback setting. It lets the callback's -exception propagate. If your agent catches tool exceptions and turns them into -tool results, you must also let `LoopGuardInterrupt` escape that error handler. -See [integration details](/guides/integration#let-the-interrupt-reach-your-application). +`handler.raise_error = True` lets LangChain propagate the interrupt. Your agent's +tool-error handlers must also let it escape. Follow the +[integration guide](/guides/integration#let-the-interrupt-reach-your-application) +when attaching LoopGuard to your own agent. -## Observe without interrupting - -Replace the handler construction above with: +To record decisions without stopping execution, replace the handler construction: ```python handler = LoopGuardCallbackHandler(interrupt_on_fatal=False) ``` -All three calls finish. LoopGuard still records the signals and the `stop` -decision. This is useful when checking thresholds against your application's -normal retries before enabling interruption. - - -In the callback report, `interrupted: true` means the policy reached `STOP` at -some point. It does not prove the framework actually stopped. This flag is also -true in observe-only mode when the policy recommends stopping. - - -## Read the report - -| Entry | What it tells you | -| --- | --- | -| `events` | The recorded tool calls, results, and errors. | -| `signals` | Detector evidence, including the reason and score. | -| `decisions` | Policy actions and the evidence used to choose them. | -| `metrics` | Counts such as recorded steps, tool calls, and repeat rate. | - -The report also includes `alerts` for compatibility. For new integrations, start -with `signals` and `decisions`. +All three calls finish, while the report still records a `stop` recommendation. +Its `interrupted` flag records that recommendation, not proof that execution stopped. +See [observe-only mode](/guides/integration#observe-first) and the +[report reference](/reference/python#read-the-result) for details. ## Next steps - [Add LoopGuard to a LangGraph/LangChain run](/guides/integration). -- [Understand detection](/how-it-works). -- [Tune configuration](/guides/configuration). -- [Analyze saved traces](/guides/saved-traces). +- [Understand healthy repetition and stuck runs](/detection). diff --git a/docs/guides/integration.mdx b/docs/guides/integration.mdx index d9eeb8f..30cf1b0 100644 --- a/docs/guides/integration.mdx +++ b/docs/guides/integration.mdx @@ -108,8 +108,8 @@ The helper uses default settings when you omit `config_path`. ## Use the stream wrapper for supported graph updates -The demo backend and CLI call `stream_run()` directly. `LoopGuard.stream()` is -the public SDK wrapper around that same runner: +Use `LoopGuard.stream()` to inspect supported streamed graph updates and the +decisions LoopGuard records: ```python from loopguard import LoopGuard diff --git a/docs/index.mdx b/docs/index.mdx index d04589c..b27134f 100644 --- a/docs/index.mdx +++ b/docs/index.mdx @@ -1,42 +1,38 @@ --- -title: Introduction +title: LoopGuard +sidebarTitle: Introduction description: Catch agents that keep spending steps without getting closer to an answer. --- -LoopGuard watches an AI agent while it runs and looks for signs that it is stuck. -An agent might repeat a tool call, retry a failing request, or keep searching after -every result comes back empty. In a larger system, work can also circle between -agents without anyone finishing it. - - -These docs follow the current repository. The published `0.2.1` package supports -the core callback and trace workflows; Gemini support, controlled demo scenarios, -and additional decision display fields currently require the source checkout. -See [release scope](/development/status). - - -Repetition alone does not tell the whole story. Polling a job until it finishes -can be useful. Asking the same question and getting the same empty answer usually -isn't. LoopGuard looks at both the repeated behavior and the results. +LoopGuard watches an AI agent while it runs and checks whether it is still making +progress. It detects loops, repeated failures, and stalled runs while allowing +useful polling and retries to continue. ```text same tool + useful new result -> continue same tool + same empty result -> stop -temporary error, then success -> continue -same permanent error repeated -> stop +503 -> 503 -> success -> continue +401 -> 401 -> 401 -> stop ``` These are patterns to look for, not guarantees about a fixed number of calls. Decisions depend on the active detectors, thresholds, and recorded evidence. + + + Install the SDK and catch a stuck tool loop without an API key. + + + Explore the Python SDK, examples, and tests. + + + ## See the problem - + LoopGuard demo with a red agent-tool loop, semantic repetition evidence, and an interrupted-run verdict -This screenshot comes from the repository's demo. It shows an earlier UI layout; -the current UI also displays policy decisions and risk scores. The [visual examples and SDK code](/demo/scenarios#run-the-visual-examples-through-the-sdk) show how to inspect the same detection behavior from Python. @@ -51,9 +47,15 @@ The Python package includes LangGraph/LangChain tool callbacks, a live-stream wrapper, and saved-trace analysis. The local demo helps you compare healthy repeats with runs that should stop. + +The published `0.2.1` package supports the core callback and trace workflows. +Gemini support, controlled demo scenarios, and additional decision display fields +currently require the source checkout. See [release scope](/development/status). + + - - Exercise real tool callbacks without a model or API key. + + Attach callbacks and handle runtime stop decisions. Follow an event from the agent to a runtime decision. diff --git a/docs/robots.txt b/docs/robots.txt index 3ce7c2e..9103d92 100644 --- a/docs/robots.txt +++ b/docs/robots.txt @@ -3,4 +3,4 @@ User-agent: * Allow: / # Mintlify generates /llms.txt, /llms-full.txt, and /sitemap.xml. -# Add an absolute Sitemap directive after the documentation domain is confirmed. +Sitemap: https://loopguard.mintlify.site/sitemap.xml