Yashotantra Academy logo

5 · Real-World Architectures

8. Asynchronous Messaging, Queues & Event Streaming

Point-to-point queues, publish-subscribe topics, Kafka log partitioning, and consumer groups.

10 min read · 3 MCQs

Point-to-Point vs Pub/Sub

Message queues (RabbitMQ, SQS) deliver a message to exactly one consumer worker from a shared pool. Publish-Subscribe topics (SNS, Kafka) broadcast a single event to multiple independent consumer subsystems simultaneously.

Kafka Log Partitioning & Ordering

Kafka organizes topics into append-only partitions. Messages with the same partition key (e.g. `user_id`) are guaranteed to land on the same partition and be consumed in exact strict chronological order. Parallelism is bounded by the number of partitions.

Dead-Letter Queues (DLQ) & Poison Pills

If a consumer crashes or fails processing a message repeatedly, the message must be routed to a Dead-Letter Queue (DLQ) after a retry limit. This prevents 'poison pill' messages from blocking the entire pipeline queue.

Chapter quiz

3 questions · pass mark 75%

  1. 1. How does Apache Kafka guarantee message ordering?

  2. 2. What is a Dead Letter Queue (DLQ) used for?

  3. 3. If a Kafka topic has 8 partitions, what is the maximum number of consumers in a single consumer group that can read concurrently?