Skip to content

feat: auto-reconnect the ibidi multiplexer on a dropped USB link - #55

Merged
Heerpa merged 2 commits into
developfrom
feature/ibidi-auto-reconnect
Nov 10, 2026
Merged

Heerpa merged 2 commits into
developfrom
feature/ibidi-auto-reconnect

Conversation

@Heerpa

@Heerpa Heerpa commented Nov 10, 2026

Copy link
Copy Markdown
Collaborator

What

Makes a dropped ibidi USB link a recoverable blip instead of an aborted run. Builds on v0.3.4 (which stopped the multi-day hang); this adds transparent recovery so a brief drop/replug doesn't cost the experiment.

How

When a valve/pump command's normal retries are exhausted (the link genuinely dropped), _command now attempts a bounded auto-reconnect before raising:

  • Same COM port first — covers USB selective-suspend and transient glitches, where the port number is unchanged.
  • Then re-discover the same USB device on a possibly-new COM port, matched by VID/PID/serial (recorded at connect via serial.tools.list_ports). It only ever opens this device's ports, so a blind reconnect can never disturb another instrument on the rig (Hamilton pumps, Arduino, lasers, …).
  • Bounded by reconnect_timeout (default 120 s, configurable via the setup's ibidi: block; 0 disables). If the window expires, it still raises → loud abort — preserving the v0.3.4 "never a forever-hang" guarantee.
  • The keep-alive self-heals an idle drop: on a failed idle probe it runs the same recovery, so the link is already back when the next real command arrives (this is exactly the scenario from the stalled-run log — a drop during a long idle, then an inject much later).

Recovery is single-flight (_reconnect_lock), interruptible by close() and orchestration abort, and never holds the IO lock across the whole window.

Behaviour summary

Situation Before (v0.3.4) Now
Brief USB glitch / suspend Abort run Resume transparently
Replug within the window Abort run Resume transparently
Idle drop, command later Abort on that command Keep-alive relinks first → no stall
Device truly dead Abort after ~20 s Abort after reconnect_timeout (loud, no hang)
Re-enumerates to new COM Abort (couldn't reopen) Finds it by USB id, resumes

Tests

New IbidiReconnectTest: command recovers via reconnect; still raises after the window expires; reconnect_timeout=0 raises immediately (no recovery); relink candidates are same-port-first then the matched device only (never a foreign port); USB identity is recorded for the connected port. Existing 36 ibidi tests + the Legacy integration + emulator/services suites all green; black/flake8 clean.

🤖 Generated with Claude Code

Heerpa and others added 2 commits November 10, 2026 12:38
When a command's retries are exhausted (the link genuinely dropped), the driver
now attempts a bounded recovery before raising: it re-establishes the link for
up to reconnect_timeout seconds (default 120, configurable), trying the same COM
port first (USB selective-suspend / transient glitch) and then re-discovering
the SAME USB device on a possibly-new port (matched by VID/PID/serial via
serial.tools.list_ports, so other instruments are never opened). A brief
drop/replug resumes the run transparently instead of aborting it; the keep-alive
self-heals an idle drop before the next command. A truly dead device still
aborts after the window, preserving the v0.3.4 no-forever-hang guarantee.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Addresses four issues found reviewing the auto-reconnect PR:

1. Keep-alive no longer runs _recover_link itself. On a dead link that
   recovery could run up to reconnect_timeout (120s), outliving
   _stop_keepalive()'s bounded ~3s join and racing close()'s self._serial=None
   teardown (reopen-after-close / AttributeError). The keep-alive now only
   detects+logs; recovery is the next real command's job.
2. Gate recovery on a new self._connected flag (set at end of connect(), cleared
   in close()). A wedged device at the initial identify() handshake no longer
   hangs connect() for the whole window before failing.
3. _write_read raises a clean SerialException when self._serial is None (closed
   concurrently) instead of an AttributeError; _try_relink bails rather than
   re-opening the port after close.
4. Fix the DEFAULT_RECONNECT_TIMEOUT comment: the override lives in the setup's
   hamilton.ibidi: block, not a `multiplexer:` block.

New tests: no recovery before connect; _write_read raises cleanly after close;
_try_relink does not resurrect the handle after close.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@Heerpa
Heerpa merged commit ee9f740 into develop Nov 10, 2026
6 checks passed
@Heerpa
Heerpa deleted the feature/ibidi-auto-reconnect branch November 10, 2026 16:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant