Skip to content

Send Terminate before closing a connection - #42

Merged
quinnj-bot[bot] merged 1 commit into
JuliaDatabases:mainfrom
PingoLee:close-sends-terminate
Nov 5, 2026
Merged

quinnj-bot[bot] merged 1 commit into
JuliaDatabases:mainfrom
PingoLee:close-sends-terminate

Conversation

@PingoLee

@PingoLee PingoLee commented Nov 2, 2026

Copy link
Copy Markdown
Contributor

Closes #41.

DBInterface.close!(conn) closed the socket without first sending the protocol's Terminate message. libpq sends it before dropping the connection (closePGconn → sendTerminateConn in fe-connect.c). Without it the server cannot tell an orderly close from a dropped client, so closing a connection inside a transaction logs unexpected EOF on client connection with an open transaction.

close! now writes Terminate under the connection lock, before closing the socket, as a best effort. It is skipped when the socket is already closed, and a write error is ignored, so close! still never throws. It is also skipped when close! is entered re-entrantly, from a notice or notification callback inside an operation. There the operation still owns the stream, and during executemany the writer task may be blocked on the socket: only closing the socket wakes it. The re-entrancy check reads conn.lock.reentrancy_cnt, a Base-internal field (present on Julia 1.10 through 1.13). I'm happy to switch to another check if you prefer one.

The internal socket closes on error paths are unchanged: those sockets are already broken, and libpq does not send Terminate there either.

No public API or runtime dependency changes.

Validation:

  • New fake-server testsets in test/isvalid_fake_server.jl:
    • an idle close sends exactly 'X' with an empty body, then EOF; this fails on the previous code;
    • a close whose Terminate write fails returns normally;
    • a close from a notice callback (run by isvalid under the lock) sends nothing; this fails with the re-entrancy check removed. The executemany hang itself is not reproduced in a test.
  • Full suite with POSTGRES_REQUIRE_INTEGRATION=true (Linux x64, local Docker): 9304 checks pass on Julia 1.13.2 and 9275 on Julia 1.10.12 against PostgreSQL 16; 9304 on 1.13.2 against PostgreSQL 14, 15, 17 and 18 with SCRAM and PostgreSQL 16 with MD5; and 9304 with julia -t 2. Docs build passes.
  • Fork CI on 24fb06d: tests pass on Linux, Windows and macOS (Julia min, 1 and pre), with Docker integration against PostgreSQL 14, 15, 17 and 18 and SCRAM and MD5 auth. Only the Codecov upload failed, because the fork has no token.
  • The repro from close!(conn) closes the socket without sending the Terminate message #41 against PostgreSQL 16 with log_disconnections = on, over plain TCP and TLS: before the change, closing after BEGIN logs unexpected EOF on client connection with an open transaction; after it, every connection logs only disconnection:.

Drafted with Claude Code (Claude Opus 5.5) and reviewed by me before posting.

🤖 Generated with Claude Code

DBInterface.close!(conn) closed the socket without sending the protocol's
Terminate message, so the server could not tell an orderly close from a
dropped client: closing inside a transaction logged "unexpected EOF on
client connection with an open transaction". libpq sends Terminate first
(closePGconn -> sendTerminateConn).

close! now writes Terminate under the connection lock before closing the
socket, as a best effort: skipped when the socket is already closed, and a
write error is ignored, so close! still never throws. It is also skipped on
a reentrant close from a callback inside an operation, where the operation
still owns the stream and an executemany writer may be blocked on the
socket. Internal closes on error paths are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@quinnj-bot quinnj-bot Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[review generated via AI]

I compared this with libpq's closePGconn in fe-connect.c (REL_16): sendTerminateConn writes Terminate only when conn->sock != PGINVALID_SOCKET && conn->status == CONNECTION_OK, ignores any error, and pqDropConnection follows. close! now does the same: Terminate only on an open socket, write errors swallowed, then the socket closes. A Connection here never exists during startup, so the CONNECTION_OK half has no equivalent.

What I ran on 24fb06d:

  • Full suite with Docker integration (PostgreSQL 16, Julia 1.12.6, macOS arm64): 9304 of 9304 pass.
  • Against a local PostgreSQL 14 with its log open. BEGIN then close(conn): main logs unexpected EOF on client connection with an open transaction, this branch logs nothing. Close on a half-closed socket (another session ran pg_terminate_backend, the FATAL sat unread): no throw and 0.1 ms on both. A double close after an InterruptException closed the socket: a no-op on both.
  • fieldnames(ReentrantLock) includes reentrancy_cnt on 1.10.11 and 1.12.6 here, and the pre CI job passed.

Cases I traced in the code and found fine:

  • close! during a running query or COPY from another task waits in @lock conn.lock for the operation, which ends at ReadyForQuery, so the Terminate follows a drained stream.
  • close! racing another close! or isvalid: the lock serializes them, and the second close sees conn.closed. cancel_query! only takes trylock, so no deadlock.
  • A broken socket: Reseau's _write_ptr! throws SystemError or EOFError on EPIPE or a reset instead of blocking, and the try swallows it. Julia ignores SIGPIPE.
  • Write deadlines: the Terminate write has none of its own, so it could block only on an open, idle, unlocked socket whose send buffer is full. No code path leaves a connection in that state, because every writer holds the lock until ReadyForQuery or closes the socket on failure, and libpq's pqFlush blocks in the same situation.
  • Finalizers: Connection has none, so nothing changes. A user-registered finalizer(close, conn) already takes conn.lock today, and a 5-byte write to an idle socket does not wait.
  • --trim: the field read is concrete, and the trim lane passed.

On conn.lock.reentrancy_cnt: it is Base-internal, so a Julia release could rename it, but it has been stable since 1.0 and the alternatives are no better. conn.lock.locked_by === current_task() before the @lock is the exact test Base.assert_havelock uses, and it is just as internal. Tracking "inside an operation" in the Connection would touch every @lock site. I would keep it as is.

Interaction with #44: #45 removes the socket check in cancel_query!. After an interrupt the socket is already closed, so close! skips Terminate there, as libpq would, and the cancel still goes out with the held pid and key. After an orderly close with Terminate, a later cancel reaches a backend that has already exited and the server ignores it. The two changes touch different functions and git merge-tree reports a clean merge in either order.

Nothing to change. I recommend merging; the merge itself is Jacob's call.

@quinnj-bot
quinnj-bot Bot merged commit 6c12a2a into JuliaDatabases:main Nov 5, 2026
16 checks passed
@quinnj-bot quinnj-bot Bot mentioned this pull request Nov 5, 2026
quinnj referenced this pull request Nov 5, 2026
Co-authored-by: Jacob Quinn <quinn.jacobd@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

close!(conn) closes the socket without sending the Terminate message

1 participant