Pub/Sub Pattern

Publish/Subscribe (Pub/Sub) decouples message producers (publishers) from consumers (subscribers) through a topic abstraction — publishers emit messages to named topics without knowledge of who will receive them. Unlike point-to-point queues, pub/sub delivers each message to all active subscribers (fanout), enabling a single event to trigger multiple independent downstream reactions. Managed pub/sub services — Google Cloud Pub/Sub, AWS SNS, Azure Service Bus Topics — handle delivery, durability, and backpressure without infrastructure management.

Key Points

• Fanout vs queue: in pub/sub, all subscribers receive every message; in queues, messages are distributed across consumers (competing consumers pattern) — choose based on whether each consumer needs every event.
• Push vs pull delivery: push (Google Cloud Pub/Sub, SNS) delivers to subscriber endpoint via HTTP POST — no polling overhead; pull (Kafka, SQS) requires subscribers to request messages — better backpressure control.
• Subscription filters: SNS message filtering routes only matching messages to each SQS subscription (e.g., filter by eventType attribute) — avoids sending all events to all consumers and processing unwanted messages.
• Ordering guarantees: SNS fanout to SQS FIFO queues with MessageGroupId preserves per-group ordering; Google Cloud Pub/Sub ordering keys ensure ordered delivery per key within a region.
• Durable subscriptions: subscriber must be attached to receive messages; non-durable subscriptions miss messages while disconnected — use durable subscriptions (SQS subscription, Kafka consumer group with retained log) for guaranteed delivery.
• Competing consumers within a subscription group: multiple instances of the same service subscribe to the same subscription — messages are distributed (competing consumers), providing horizontal scaling within the subscriber.
• Dead letter handling in pub/sub: Google Cloud Pub/Sub supports per-subscription dead letter topics after maxDeliveryAttempts (default 5); SNS SQS subscription DLQ captures failed deliveries.
• SNS → SQS fanout pattern: one SNS topic fans out to multiple SQS queues (one per microservice); each service processes messages at its own pace, independently scalable and independently failing.