Skip to content

Agent demo: "Ask your data", a brief for whoever asked; hosted serves insurance - #144

Merged
amrit110 merged 2 commits into
mainfrom
demo/ask-your-data
Nov 18, 2026
Merged

amrit110 merged 2 commits into
mainfrom
demo/ask-your-data

Conversation

@amrit110

Copy link
Copy Markdown
Member

Why

The demo page was built for whoever runs the agent: an agent picker, the raw prompt, bash commands and JSON. The people we show it to (analytics teams, business leads, clinician researchers) need to see what it does for them: ask a question about two raw files, and get back an answer they can act on, with the data engineering done for them.

What

A brief for whoever asked. The page opens on the question in plain English with its two files. Ask runs the agent; watch a recorded run replays one. The answer builds as a short brief, one section per step. Each section is a plain question, a one-sentence answer written from the run's own numbers, and one visual:

  • Is the data usable? Five of the messiest raw rows, underlined like misspellings, then shown cleaned, and the spellings merged (33 spellings of province → 9).
  • Can it be predicted? "77 times in 100" against 72 for gradient boosting.
  • Who is most at risk? The expected count and the top ten.
  • What drives it? What the predictions lean on.
  • What could change it? Each what-if as a move from today's risk.
  • The memo, then was it right?: 8 of its top 10 really cancelled; ten at random would hold about 2.

It uses paper and ink, a serif for what is asked and answered, hairlines instead of cards, and color only for data. Show the work opens every command and tabctx call beside the brief (the old timeline, restyled), with the full instructions.

Plumbing:

  • ui/cleaning.py sets the raw export beside the agent's clean table, joined on the id, and is served at /api/runs/{agent}/cleaning.
  • make_showcase.py saves that view as cleaning.json, since showcase runs don't keep the data files.
  • Each scenario's LABELS gain the question, the audience and the column names; insurance gains RAW_COLUMNS and CLEAN_HISTORY.

Hosted serves insurance.

  • The Dockerfile now defaults to DEMO_SCENARIO=insurance.
  • Claude Code's sandbox (bubblewrap) can't run in a container, so the image sets CLAUDE_SANDBOX=off. run.sh then gives Claude Code Bash as the run's own Unix user, which can't read the answers (root-only) or other runs. Codex already runs this way there. Local runs keep the sandbox.

Checks

  • pytest tests/unit/test_agent_demo_*.py: 78 passed. New tests:
    • the cleaning view: rows removed, spellings merged, numbers converted or emptied, the sample, a saved view, and a table that doesn't match;
    • every scenario's question and column names;
    • the hosted Claude shell without the sandbox.
  • ruff check and format: clean.
  • In Chrome:
    • recorded insurance and readmission runs, in light and dark, with Show the work on and off;
    • the flip between the raw and cleaned rows, the memo and the check.
  • Not yet checked: a live insurance run in the hosted container. It will be checked after deploy.

🤖 Generated with Claude Code

… insurance

The demo page was built for whoever runs the agent: an agent picker, the
raw prompt, bash commands and JSON. It is now built for whoever asked
the question (a business lead, a clinician): the question in plain
English with its two files, then the answer as a short brief, one
section per step, each a plain question ("Is the data usable?"), a
one-sentence answer written from the run's numbers, and one visual.
Paper and ink, a serif for what is asked and answered, color only for
data. Every command and tool call is still there behind "Show the work".

- ui/cleaning.py sets the raw export beside the agent's clean table
  (joined on the id): rows removed, spellings merged, the messiest rows.
  The page shows those rows as exported, then cleaned.
  /api/runs/{agent}/cleaning serves it; make_showcase.py saves it as
  cleaning.json, since showcase runs don't keep the data files.
- Each scenario's LABELS gain the question, the audience and the
  column names as people say them; insurance gains RAW_COLUMNS and
  CLEAN_HISTORY.
- The hosted image serves the insurance scenario. Claude Code's sandbox
  (bubblewrap) can't run in a container, so CLAUDE_SANDBOX=off gives it
  Bash as the run's own Unix user, which can't read the answers or other
  runs, as Codex already does there.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@codecov-commenter

codecov-commenter commented Nov 18, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 87.80%. Comparing base (e1490e1) to head (a86f1d4).

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #144      +/-   ##
==========================================
+ Coverage   87.72%   87.80%   +0.07%     
==========================================
  Files          67       67              
  Lines        6723     6723              
==========================================
+ Hits         5898     5903       +5     
+ Misses        825      820       -5     
Flag Coverage Δ
integration 32.20% <ø> (ø)
unit 87.63% <ø> (+0.07%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.
see 2 files with indirect coverage changes

Impacted file tree graph

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Hosting:
- The insurance data's generator is public and its seed was fixed, so an
  agent with a shell could rebuild the withheld outcomes. The hosted image
  now builds its data from a seed drawn at build time (DEMO_SEED) and kept
  nowhere.
- A finished run's folder goes back to root, so the next run given its
  uid (another visitor's) can't read it.
- Codex's label says "shell" where it runs without its sandbox, as
  Claude's now does.

Cleaning view (the agent's clean table is untrusted):
- A saved cleaning.json is read only for showcase runs, and bad JSON
  there gives no view instead of a 500.
- Only plain files inside the run's folder under 50 MB are read, never a
  link, and each version is summarized once.
- True/False and 1/0 columns are categories; blanks are not a merged
  value ("nan"); columns left empty are skipped instead of hiding the
  whole view; duplicate ids in the clean table don't repeat sample rows.

The page:
- One event that fails to draw no longer stops a replay (and the work
  panel still draws it); tool results of unexpected shape are guarded.
- A lost connection says so and offers the way back; "Watch again"
  replays what was shown.
- "Sign in to ask" no longer closes the popover it opens; picking an
  agent keeps it open; it takes focus and Escape returns it.
- Steps the agent never answered say so instead of looking done.
- Regression tasks: the best model for lower-is-better metrics, predicted
  values instead of risk bars.
- A cleaning fetch from an earlier run can't land in a new one.
- Muted text passes WCAG AA in both themes; the switch keeps its name on
  phones; duplicated rules are in one stylesheet.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@amrit110
amrit110 merged commit 468612a into main Nov 18, 2026
5 checks passed
@amrit110
amrit110 deleted the demo/ask-your-data branch November 18, 2026 15:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants