From e1e7e91f51c236272da437ccb5277b0f1573a8ab Mon Sep 17 00:00:00 2001 From: Gabriel Matossian Date: Tue, 20 Oct 2026 20:12:22 -0700 Subject: [PATCH] Lead Messaging with three simple delivery sketches --- content/topics/messaging/delivery-shapes.svg | 57 -------------------- content/topics/messaging/pubsub-delivery.svg | 17 ++++++ content/topics/messaging/queue-delivery.svg | 17 ++++++ content/topics/messaging/stream-delivery.svg | 14 +++++ content/topics/messaging/topic.md | 40 ++++++-------- 5 files changed, 63 insertions(+), 82 deletions(-) delete mode 100644 content/topics/messaging/delivery-shapes.svg create mode 100644 content/topics/messaging/pubsub-delivery.svg create mode 100644 content/topics/messaging/queue-delivery.svg create mode 100644 content/topics/messaging/stream-delivery.svg diff --git a/content/topics/messaging/delivery-shapes.svg b/content/topics/messaging/delivery-shapes.svg deleted file mode 100644 index 78d84af..0000000 --- a/content/topics/messaging/delivery-shapes.svg +++ /dev/null @@ -1,57 +0,0 @@ - - - Messaging delivery shapes - A work queue shares jobs between workers. Pubsub fans an event out to subscriptions. Log consumer groups track separate positions in retained history. - - - - - - - - Work queue: share jobs - - Producer - - Queue: A B C - - - Worker 1: A - - Worker 2: B - - - Different workers receive different jobs. - Retries can repeat a job. - - Pub/sub: fan out an event - - Producer - - Topic: E - - - Sub A: event E - - Sub B: event E - - - Each subscription receives event E. - Its workers can share that backlog. - - Retained log: read at own pace - - Producer - - E1 E2 E3 E4 - - - Group A: pos 2 - - Group B: pos 4 - - - One partition; independent positions. - Reading does not delete the events. - - diff --git a/content/topics/messaging/pubsub-delivery.svg b/content/topics/messaging/pubsub-delivery.svg new file mode 100644 index 0000000..a705fca --- /dev/null +++ b/content/topics/messaging/pubsub-delivery.svg @@ -0,0 +1,17 @@ + + Pub/sub · fan out an event + The same event E goes to independent billing and email subscriptions. + + + + Pub/sub · fan out an event + + Event E + + + + Billing gets E + + Email gets E + + diff --git a/content/topics/messaging/queue-delivery.svg b/content/topics/messaging/queue-delivery.svg new file mode 100644 index 0000000..24a44d0 --- /dev/null +++ b/content/topics/messaging/queue-delivery.svg @@ -0,0 +1,17 @@ + + Queue · share jobs + Jobs A and B are assigned to different workers; retries can repeat a job. + + + + Queue · share jobs + + Jobs A B + + + + Worker 1 gets A + + Worker 2 gets B + + diff --git a/content/topics/messaging/stream-delivery.svg b/content/topics/messaging/stream-delivery.svg new file mode 100644 index 0000000..4bc4642 --- /dev/null +++ b/content/topics/messaging/stream-delivery.svg @@ -0,0 +1,14 @@ + + Stream · retain and replay + Events remain in a retained log and can be read again while retained. Ordering is within a defined boundary. + + + + Stream · retain and replay + + Retained events: 1 → 2 → 3 → 4 + + + Read again + + diff --git a/content/topics/messaging/topic.md b/content/topics/messaging/topic.md index c8aee5f..a3e3aae 100644 --- a/content/topics/messaging/topic.md +++ b/content/topics/messaging/topic.md @@ -16,39 +16,29 @@ childTopicIds: relatedTopicIds: [] --- -## Choose from the requirement +![Queue: jobs A and B go to different workers.](./queue-delivery.svg) -### Work queue +- **Use:** distribute thumbnail jobs among workers. +- **Plan for:** retries can repeat work; make effects duplicate-safe. -Distribute thumbnail jobs among workers. +![Pub/sub: billing and email each receive event E.](./pubsub-delivery.svg) -**Tradeoff:** Retries can repeat work; effects must tolerate duplicates. +- **Use:** let independent applications each react to the same event. +- **Remember:** durable subscriptions or subscriber queues can retain messages while consumers are offline. -### Publish/subscribe +![Stream: retain events and read them again while history remains available.](./stream-delivery.svg) -Let billing, email, and analytics each react to an order event. +- **Use:** retained history, replay, or ordered event processing. +- **Plan for:** retention limits replay; ordering is commonly per partition, not global. Multiple consumers are optional. -**Tradeoff:** Each subscription needs its own backlog and failure policy. - -### Event stream/log - -Retain events for replay or process an ordered sequence. - -**Tradeoff:** Manage retention and consumer positions; ordering has a defined boundary. - -## How delivery differs - -![A work queue distributes different jobs among workers; pub/sub sends an event to each subscription; retained-log consumer groups read at independent positions.](./delivery-shapes.svg) - -## Before choosing - -- **Pub/sub can be durable:** offline consumers do not automatically lose messages. Retention and subscription settings matter. -- **Streams do not require multiple consumers:** retention, replay, and ordered event processing are stronger selection reasons. Ordering is commonly per partition, not global. -- **Products overlap:** queues, pub/sub, and logs are models, not mutually exclusive product categories. +> **Database jobs can also fit.** A durable job table avoids another service, but workers need safe claiming, retries, and recovery; polling adds database load. -A **producer** sends messages; a **consumer** processes them; a **broker** is infrastructure that routes or stores them. A **topic** names a publication destination; its delivery semantics depend on the product. +## Terms and boundaries -> **Database jobs can also fit.** A durable job table avoids another service, but workers need safe claiming, retries, and recovery; polling adds database load. +- **Producer / consumer:** sends / processes messages. +- **Broker:** infrastructure that routes or stores messages. +- **Topic:** named publication destination; delivery semantics depend on the product. +- **Models overlap:** one product can support queues, pub/sub, and logs. ## Sources