Repository navigation
FundingCAPTCHA: self-learning depth background, no boot-time standback - #170
Merged
Merged
Conversation
#166) IIVision gains AdaptiveBackground: "the farthest steady depth wins". A person can only make a depth pixel nearer, so each pixel's background is the farthest depth it has held steadily, learned from the first frame with people in view. Rules: seed, slow EMA at background, reveal after reveal_frames farther, absorb after absorb_s steady nearer, and a bump guard that reseeds when most of the scene stays foreground. BlobTracker now reads a small BackgroundModel protocol (background, valid_mask, threshold()), implemented by both Calibration and AdaptiveBackground. Calibrator stays for FlowerBeds. app.py, shader_app.py (and so playlist_app.py) and body_grid_tester.py drop BG_CAL, the standback timer and the cal_progress/cal_done handshake; R now reseeds the background. captcha-settings.json swaps the calibration_* keys for a background block. Per-frame update time is logged every stats_log_s. ADR-0023 supersedes ADR-0013's BG_CAL section. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
On real hardware, launch left small blobs that lasted until absorb_s: - The Gemini 336's first frames read up to ~400 mm too far (1 frame warm, ~5 cold). Seeding bg from them made the true surface "nearer". New background.warmup_s (1.0) ignores the start of a stream. - Seeding bg from one sample is fragile: on a noisy far pixel a single far outlier left every later reading nearer, never at bg, too jumpy to absorb. bg and sigma now come from the mean and spread of the first min_valid_frames samples. Checked against four recorded starts and a live run: no persistent foreground. Live update time is 12-14 ms mean at 640x480. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Contributor
Author
|
Hardware testing found small blobs left on the screen after launch. They lasted until
Checked against four recorded camera starts and a live 30 s run: no persistent foreground. Two numbers in the description above change:
🤖 Generated with Claude Code |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #166.
What
None of the FundingCAPTCHA entry points (
app.py,shader_app.py/playlist_app.py,body_grid_tester.py) has a standback or calibration screen any more. The depth background is learned continuously from the first frame, with people in view: the farthest steady depth wins (ADR-0023).IIVision
AdaptiveBackground/AdaptiveBackgroundConfig(IIVision/adaptive_background.py). All of it is vectorised numpy, with an injectable clock,push(frame)andreseed(). It implements the issue's five rules: seed, EMA at background, reveal afterreveal_frames, absorb afterabsorb_s, and the bump guard.BlobTrackernow reads a smallBackgroundModelprotocol:background,valid_mask,threshold(depth_delta_mm).Calibrationgainsthreshold(), which uses the same 3×IQR formula as before, so FlowerBeds andCalibratorbehave exactly as they did.background.stats_log_s(30 s). It measures about 5 ms at 640×480 on this machine, against a 66 ms budget at 15 fps.FundingCAPTCHA
app.py: the BG_CAL state, the countdown screen, thecollectgate and thecal_progress/cal_donehandshake are gone. It boots straight into SCREENSAVER.Rreseeds the background from the next frame./api/restartstill restarts the camera thread, which now starts a fresh model and returns to SCREENSAVER instead of BG_CAL.shader_app.py: the standback timer and thecollectevent are gone.playlist_app.pyshares this code, so it gets the change too.body_grid_tester.py: the same change. ItsCviews (background, valid mask, noise σ, threshold, delta) now show the live model, refreshed every 0.5 s.Rreseeds.captcha-settings.json: thecalibration_*keys are replaced by abackgroundblock with the issue's defaults.Docs
docs/FundingCAPTCHA.md,docs/agents/architecture.mdand ADR-0007's/captcha/restartline are updated.One deviation from the issue
The issue says σ starts at
depth_delta_mm"sothrstarts atdepth_delta_mm". Withthr = max(depth_delta_mm, noise_k × σ), it actually starts atnoise_k × depth_delta_mm, which is 120 mm with our settings. I kept σ seeded atdepth_delta_mmas the issue's table says. Starting high means early frames read nearly everything as background and the σ estimate settles down from above (time constant about 20 frames). Starting low would bias the estimate low, because it only learns from samples insidethr, and could flash false foreground just after boot. The cost is that for the first few seconds, something within about 120 mm of its background (feet on the floor) isn't foreground. ADR-0023 records the rule as implemented.Tests
IIVision/test_adaptive_background.pycovers every case the issue lists: a person present from frame 0 who then walks off, a person standing still for 120 s, a static object absorbed atabsorb_s, a single-frame far spike, Gaussian noise (σ 20 mm at 3 m, < 0.1 % raw foreground, none after despeckle), a bump nearer that reseeds and a bump farther that heals without one, and the update-time log. Fidget, crowd and reseed-request cases are covered too.ShowControl/FundingCAPTCHA/test_live_background.py:shader_app.start_camerawithMockCamera(bodies present from frame 0) shows a body in the Body texture within 1.5 s. In practice it's frame 10, about 0.63 s. The other tests check that the BG_CAL state and calibration settings are gone.app.py --mock-cameraandbody_grid_tester.py --mock-cameraboth smoke-ran headless without errors. The app reached the Arc 5.6 s after launch, 5 s of which is the attract dwell.Python suite: 498 passed. Two things fail or can't run, and both are the same on
main:test_hunter_bird.py::test_person_has_white_fill_and_ink_outlinefails.FlowerBeds/test_layout_tool.pycan't be collected becausefastapiisn't installed on this machine.Node isn't installed here, so the
node --testhalf ofmake testdidn't run. Nothing JS changed.Needs hardware after merge
Background update … mean/max msline stays within budget at 640×480 @ 15 fps on the show machine.absorb_sagainst real dwell times, and whether a dense crowd sets off the bump guard.🤖 Generated with Claude Code