feat(messaging): a NATS transport, so the bus has a client the fleet shares - #15
Merged
Conversation
…shares NATS JetStream is live on compliance-engine and nothing could talk to it, because no service had a client. This is that client, as transport policy rather than as an abstraction over anything. ADMITTED ON THE kit.clients ARGUMENT, and it arrived on its first copy rather than its third, so it is worth stating what carries it. This is not an abstraction over a domain: it is the ack policy, the redelivery behaviour, the reconnection settings and the header convention, and those are wrong in the same way in every service that decides them alone. They are also wrong invisibly, and are found during an incident, with a consumer that has been quietly reprocessing or quietly skipping. WHAT IT REFUSES TO HOLD is the half that keeps that honest. The payload is opaque bytes: no schema, no event registry, no versioning, because what an event IS belongs to the services that agree on it. It creates no streams and no consumers either, since retention and replica counts are operational decisions and a library that makes them makes them once, wrongly, in production. THE FOUR HEADERS ARE THE JUDGEMENT CALL. They are in because they are how tenancy and idempotency travel: without a fixed convention two services disagree about where the tenant id lives, and a redelivered message cannot be recognised as a duplicate. The event id rides as Nats-Msg-Id rather than a name of ours, which is not cosmetic: that is the header JetStream itself deduplicates on, so publishing is idempotent inside the stream's window for free instead of leaving the whole problem to consumers. AT LEAST ONCE, SAID PLAINLY. fetch returns unsettled messages and settling is an explicit ack, nak or term. Nothing acknowledges on the caller's behalf, because an auto-acking API turns at-least-once into at-most-once at the exact moment a handler raises. A message whose headers cannot be read is terminated rather than naked: it will be malformed next time too, and a nak would spend the whole attempt budget rediscovering that. A failed settle is logged and swallowed. The work happened; only the acknowledgement did not, so the message comes back and the handler's idempotency covers it. Raising there would tell a caller its work failed, which is worse than the truth. nats-py is an OPTIONAL EXTRA, for the reason telemetry is: a dependency here is one all ten repositories inherit and cannot refuse. The import is function local, and CI's minimal-install job now asserts kit.messaging imports without it. Connecting without the extra raises a BrokerUnavailable naming the extra, which is a failure a service that forgot can actually read. Metrics label the STREAM, never the subject. kit's own metrics docstring warns that one label carrying a tenant id turns one series into millions, and a subject carries one in every useful design. The four new instruments are registered in _build, without which record() would have dropped every one of them silently. Version goes to 0.7.0, which also fixes a real bug: pyproject and kit.__version__ both said 0.6.1 at the commit tagged v0.6.2, so three services pinned to v0.6.2 logged the wrong version at boot, defeating the field's whole purpose. test_version compares the source to the installed metadata, and both derive from the same file, so it never caught it. Gate green: ruff, pyright strict, 295 tests, 100% branch coverage, and the package imports with neither extra installed. Closes #14
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #14. NATS JetStream is live on
compliance-engineand nothing could talk to it, because no service had a client.The argument for it being here at all
This arrived on its first copy, not its third, so it is worth stating what carries it rather than waving it through.
It is the
kit.clientscase: not an abstraction over a domain, but the ack policy, the redelivery behaviour, the reconnection settings and the header convention. Those are wrong in the same way in every service that decides them alone, and they are wrong invisibly, found during an incident with a consumer that has been quietly reprocessing or quietly skipping.What it refuses to hold is the half that keeps that honest:
The payload is opaque bytes. Streams are not created here either: retention and replica counts are operational decisions, and a library that makes them makes them once, wrongly, in production, because the call is idempotent and the second caller's arguments are ignored.
One design decision worth reviewing closely
The event id rides as
Nats-Msg-Id, not a name of ours. That is JetStream's own deduplication header: the server refuses a second publish carrying an id it has already seen inside the stream's duplicate window.Using
Scadable-Event-Idwould have looked tidier and left that mechanism switched off, pushing duplicate suppression entirely onto consumers. This way publishing is idempotent inside the window for free, and past it, it is still the id a consumer dedupes on.At-least-once, said plainly
fetchreturns unsettled messages; settling is an explicit call. Nothing acknowledges on the caller's behalf, because an auto-acking API turns at-least-once into at-most-once at the exact moment a handler raises.Two consequences that are deliberate:
The gates this had to pass
nats-pyis an optional extra, for the reason telemetry is: a dependency here is one all ten repos inherit and cannot refuse. The import is function-local, CI's minimal-install job now assertskit.messagingimports without it, and I verified the package imports with neither extra present. Connecting without it raises aBrokerUnavailablenaming the extra.service_client(transport=...)uses.nats-pyships no type information, so the module is narrowed toAnyat the boundary exactly as_metricsnarrows the OTel SDK, keeping strictness on our code rather than turning the setting off._build(), without whichrecord()would have dropped every one silently.A version bug fixed on the way past
pyproject.tomlandkit.__version__both said0.6.1at the commit tagged v0.6.2, soapi,harnessanddispatcherhave been logging the wrong version at boot, which defeats that field's whole purpose.test_version.pynever caught it because it compares the source to the installed metadata and both derive from the same file.Both go to
0.7.0here.After this merges
I cut the v0.7.0 tag.
brainis pinned atv0.5.2, so that is a two-minor jump, andv0.5.2..v0.6.2movedinstall_conventions, so it is not purely additive.