Skip to content

Agent demo: group benefits renewals, the agent's commands as pictures, presenter mode - #145

Open
amrit110 wants to merge 5 commits into
mainfrom
demo-examples-section
Open

amrit110 wants to merge 5 commits into
mainfrom
demo-examples-section

Conversation

@amrit110

@amrit110 amrit110 commented Nov 18, 2026 •

Copy link
Copy Markdown
Member

Four changes to the agent demo, plus fixes from review, both so a room can follow the run on a big screen.

1. Which past rows will it learn from?

The brief now has a section after cleaning, Which past rows will it learn from? A tabular foundation model learns from the past rows it is shown, so choosing that cohort is the agent's main job, not only tidying the data.

It shows three things:

  • Who counts: the rows that count, once each, with the removed copies and non-records.
  • What counts as cancelled: the export's outcome spellings, mapped to the outcome, with the class counts.
  • What each row says: how many columns each row has.

The data comes from ui/cleaning.py: summarize() now returns an examples block (rows, features, and the outcome's classes with their raw spellings). The showcase's cleaning.json is regenerated; its other fields are unchanged.

2. The agent's commands, drawn as what they do to the data

"Show the work" used to list raw commands (Bash python3 - <<'EOF' import pandas…), unreadable on a projector. A new ui/static/actions.js turns each command into a tile: a plain-English title, a small picture read from the command and its output, and the raw command one click away.

Command Tile
ls; head; wc -l Opened the files: each file with its row count
pandas checks on the raw export Checked the raw data for problems: duplicates, a totals row, test policies, and a bar per column written many ways. Consecutive checks fold into one tile (× 4)
Write clean.py Wrote the cleaning script: the steps its code does (remove copies, read every date format, …)
python3 clean.py Ran the cleaning: 6,127 → 6,000 rows, the dropped rows, and each column's spellings collapsing into its values (province 33 → 9)
Edit Adjusted clean.py: lines out and in
md5 re-run Re-ran it to confirm nothing changed
Write report.md Wrote the report: its sections

A picture appears only when the command or its output says it; otherwise the tile shows just the title. Codex's /bin/zsh -lc wrapper is unwrapped.

3. Presenter mode, and a replay you step through

Press P (or open /?present) to give the agent's work most of the screen at 1.3× for a projector; P again returns to the normal page.

In presenter mode a replay stops before each of the agent's commands:

  • →, space, Enter or a clicker's page-down shows the next one;
  • A plays the rest at a pace to talk over (2.4 s a command), A again pauses.

A pill at the bottom shows the step (6 / 27) and the keys. The work panel now follows the newest tile in every mode; a replay used to leave it at the top. A live run is never held back.

4. The insurance scenario becomes group benefits renewals

Each row is now an employer whose group health, dental, life and disability plan renews this year; the question is which employers will leave. Drivers follow what group carriers describe: a renewal increase after a bad claims year (loss ratio), service issues, late payments, small plans and the broker channel push plans out; tenure, broader coverage, ASO funding, online plan admin, a wellness program and member app use keep them. An account review meeting halves the effect of an increase, the what-if lever. About one plan in eight leaves. The export keeps its mess, plus a loss ratio written as a percent in some rows and a fraction in others.

The showcase is a new Claude Code run on this data (2.6 min, $0.92): 6,127 → 6,000 rows; held-out ROC AUC 0.765 vs 0.733 for gradient boosting; against the withheld outcomes 0.776, 9 of the top 10 left, 48% of the top 100 (12% base rate).

Fixes from review

  • Presenter keys: Space and Enter on a focused button keep their meaning; a step key only acts when a step is waiting; non-element event targets are handled.
  • A new run or replay stops the old one's pending timer (it could shift the new queue or start a second drain).
  • The work panel follows the newest tile only while the viewer is at the bottom.
  • actions.js: bounded regexes and output (long agent output could freeze the tab); verify no longer catches pandas .diff(); Gemini's replace/read_file/list_directory and pi's oldText/newText edits; python3 -u clean.py; reads pandas value-count dicts, describe() extremes, "N rows in, M plans out" and per-file lines; a cleaning run that doesn't print its merges gets them from the page's own cleaning view.
  • cleaning.py: rows with no outcome are not examples (counted as unlabeled), and outcome spellings keep their spaces.
  • transcripts.py: a Claude Code system notice whose message is text (permission_denied) no longer crashes the parser.

Tests: tests/unit/test_agent_demo_cleaning.py (two new cases) and the new tests/unit/test_agent_demo_actions.py, which runs actions.js under node on the recorded Claude Code run (skipped without node). All agent demo tests pass (105), including new ones for each agent's tool shapes, escaping of hostile output, a time bound on huge output, unlabeled rows and the group benefits data's levers. Checked in Chrome on the recorded insurance run, in both modes (stepping, auto, and P on and off).

🤖 Generated with Claude Code

A tabular foundation model learns from the past rows it is shown, so
choosing them (the cohort) is the agent's main job, not just tidying.
The brief gets a section after cleaning, "Which past rows will it learn
from?": how many rows count (each once), how the export's outcome
spellings were read (the question's own definition), and how many
columns each row has. The data comes from the cleaning view, which now
also reports these examples; the showcase's saved view is regenerated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@codecov-commenter

codecov-commenter commented Nov 18, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 87.60%. Comparing base (468612a) to head (5f83c5b).

❌ Your changes status has failed because you have indirect coverage changes. Learn more about Unexpected Coverage Changes and reasons for indirect coverage changes.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #145      +/-   ##
==========================================
- Coverage   87.72%   87.60%   -0.12%     
==========================================
  Files          67       67              
  Lines        6723     6723              
==========================================
- Hits         5898     5890       -8     
- Misses        825      833       +8     
Flag Coverage Δ
integration 32.20% <ø> (ø)
unit 87.44% <ø> (-0.12%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.
see 1 file with indirect coverage changes

Impacted file tree graph

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

The work panel listed raw bash and file writes (python3 - <<'EOF' ...),
unreadable on a projector. Each command is now a tile with a plain title
and a picture read from the command and its output: the files and their
rows; what a check found (duplicates, a totals row, columns written many
ways); the cleaning script's steps; the cleaning run as 6,127 -> 6,000
rows with each column's spellings collapsing into its values; edits as
lines in and out. Consecutive checks fold into one tile. The raw command
stays one click away, and a picture appears only when the output says it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@amrit110 amrit110 changed the title Agent demo: show which past rows the model learns from Agent demo: show the examples the model learns from, and draw the agent's commands Nov 19, 2026
P (or ?present) gives the agent's work most of the screen at 1.3x, for
a projector. A replay then stops before each of the agent's commands:
→, space, Enter or a clicker's page-down shows the next one, and A plays
the rest at a pace to talk over (2.4 s a command). A pill at the bottom
shows the step and the keys. The work panel now follows the newest tile
in every mode (a replay used to leave it at the top).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@amrit110 amrit110 changed the title Agent demo: show the examples the model learns from, and draw the agent's commands Agent demo: the examples the model learns from, the agent's commands as pictures, and a presenter mode Nov 19, 2026
amrit110 and others added 2 commits November 18, 2026 20:38
- Presenter keys: Space and Enter on a focused button or link keep their
  meaning, and a step key only acts when a step is waiting.
- A new run or replay stops the old one's pending timer, which could
  shift the new queue or start a second drain.
- The work panel follows the newest tile only while the viewer is at
  the bottom; their own scrolling up is left alone.
- actions.js: bounded regexes and output (an agent's long output could
  freeze the tab); "verify" no longer catches pandas .diff(); Gemini's
  replace/read_file/list_directory and pi's oldText/newText edits;
  python3 -u clean.py; the raw id comes from the scenario (app.py
  sends raw_id), and labels no longer say "policies".
- cleaning.py: rows with no outcome are not examples (counted as
  "unlabeled", no "nan" class), and outcome spellings keep their spaces.
- Tests: each agent's tool shapes, escaping of hostile output, a time
  bound on huge output, the Codex unwrap itself, unlabeled rows.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
For a group benefits audience: each row is an employer whose group
health, dental, life and disability plan renews this year, and the
question is which employers will leave at renewal. The drivers follow
what group carriers describe: a renewal increase after a bad claims
year (loss ratio), service issues, late payments, small plans, broker
channel; tenure, broader coverage, ASO funding, online plan admin, a
wellness program and member app use keep plans. An account review
meeting halves the effect of an increase, the what-if lever. About one
plan in eight leaves. The export keeps its mess, plus a loss ratio
written as a percent in some rows and a fraction in others.

The showcase is a new Claude Code run on this data: 6,127 -> 6,000
rows, held-out ROC AUC 0.765 vs 0.733 for gradient boosting, and
against the withheld outcomes 0.776, 9 of the top 10 left, 48% of the
top 100 (12% base rate).

Also:
- transcripts.py: a Claude Code system notice with a text message
  (permission_denied) no longer crashes the parser.
- actions.js reads more of what agents print: pandas value counts as
  dicts, describe() extremes, "N rows in, M plans out", per-file
  prefixed lines, "(s)" plurals; a cleaning run that doesn't print its
  merges gets them from the page's own cleaning view.
- Presenter keys work when the event target isn't an element.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@amrit110 amrit110 changed the title Agent demo: the examples the model learns from, the agent's commands as pictures, and a presenter mode Agent demo: group benefits renewals, the agent's commands as pictures, presenter mode Nov 19, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants