fix: bound plugin shutdown so an orphaned process can't hang forever

[aesslinger]

Summary

Fixes #155 — the plugin process never exits when Tabularis quits while an RPC is in flight, instead reparenting to PID 1 and holding its DB connections open until killed manually.

Root cause

After stdin EOF, main.rs awaited every worker with no upper bound, and none of the awaits a worker can block on has a time limit:

run_reader(req_tx).await;          // returns on stdin EOF
let _ = shutdown_tx.send(true);
for handle in worker_handles {
    let _ = handle.await;          // no upper bound
}

The only timeout in the whole codebase is the startup-script preflight (STARTUP_SCRIPT_TIMEOUT_MS). So a worker blocked on an unbounded await — a pooler that accepted the TCP connection but never answers the startup handshake (the exact repro in #155), a hung query, or a saturated pool wait — keeps the process, and every DB connection it holds, alive forever after stdin closes.

Fix

Drain in-flight RPCs for at most SHUTDOWN_GRACE (5s) after stdin EOF, then force-cancel any worker still blocked:

let deadline = Instant::now() + SHUTDOWN_GRACE;
for mut handle in worker_handles {
    match tokio::time::timeout_at(deadline, &mut handle).await {
        Ok(Ok(())) => {}
        Ok(Err(_join_err)) => {}
        Err(_elapsed) => {
            handle.abort();
            let _ = handle.await;
        }
    }
}

Each JoinHandle is polled by reference (&mut handle) — JoinHandle: Unpin — so timeout_at borrows the handle rather than consuming it. That keeps ownership with us so, on elapsed, we can still call handle.abort(), which cancels the stuck future and drops the held deadpool::Object, closing the DB socket server-side instead of detaching the task and leaking it. The naive timeout(handle).await that drops the handle on elapsed would detach the task (it keeps running) — reproducing #155 rather than fixing it. The abort() is load-bearing.

Why not connect_timeout?

tokio-postgres's connect_timeout bounds only the socket-level connect, not the startup handshake. The #155 repro listener accepts the TCP connection (connect succeeds instantly) then never answers the PG handshake, so the handshake read blocks forever regardless of connect_timeout. Only this shutdown deadline covers every hang mode (connect, handshake, query, pool wait) uniformly. Pool-config timeouts (wait_timeout/connect_timeout) are a complementary fail-fast improvement worth a separate follow-up, but none of them bound the handshake, so they can't be the fix for this issue.

Long-running queries

A query still in flight when the grace elapses is force-cancelled: the abort() drops the deadpool::Object, closing the DB socket, so PostgreSQL cancels the running statement. This is the same outcome the builtin produces when the host process dies and takes its pool with it — not a regression. The result is undeliverable anyway (the host that issued it is gone).

During normal operation (host alive), long queries are bounded by the host's per-call timeout, which sends a cancel notification that triggers server-side pg_cancel_backend (src/cancel.rs, #126) — that is the primary bound on long queries, not this grace. SHUTDOWN_GRACE is only the orphan fallback for when no cancel is coming because the host itself is gone.

Verification

All against the built binary + the live podman pg-tabularis-test DB:

Check	Result
cargo build	clean
cargo clippy --all-targets -- -D warnings	clean
cargo fmt --check	clean
#155 repro (in-flight connect → stdin close)	exit 0 in ~6s (was exit 124 / hung)
Control (plain EOF, no request)	exit 0 instant (preserved)
tests/shutdown.rs (3 new tests)	3/3 pass
tests/live_db.rs (37 tests)	37/0 — no regressions
unit tests (--lib)	410/0

The new tests/shutdown.rs covers: the in-flight hang case (a hanging TCP listener that accepts but never answers), the plain-EOF control, and a live-DB happy path proving a fast in-flight query completes and returns its result before the deadline (guards against the fix aborting legitimate in-flight work).

Checklist

• Fix is scoped to main.rs shutdown path only
• No behavior change during normal operation (grace only fires on stdin EOF / orphan path)
• Regression tests added and passing
• Live DB suite green — no regressions
• Clippy + fmt clean

[github-actions]

Version suggestion

Based on this PR's title (fix) and the prerelease:rc label:

Current	1.0.0-rc.9
Suggested next tag	v1.0.0-rc.10

This is informational only — no tag or release is created automatically yet.