Every time a new service needs to talk to another, someone asks: 'should we use Kafka?' The answer depends on what you actually need. These four systems look similar from the outside but have fundamentally different guarantees, operational profiles, and sweet spots.

Redis Pub/Sub and Streams

Redis is not primarily a message broker, but it does pub/sub and streams well. Redis pub/sub is fire-and-forget — if a subscriber isn't connected, the message is gone. Redis Streams (XADD/XREAD) add persistence and consumer groups. Use Redis when you're already using it for caching and need lightweight messaging, or for real-time notifications where occasional loss is acceptable.

Kafka: The Distributed Log

Kafka is a durable, ordered, replayable log. Messages are retained for days or weeks. Multiple consumer groups can read the same topic independently at their own pace. Kafka shines for high-throughput event streaming, audit trails, event sourcing, and feeding data pipelines. The cost: operational complexity. Kafka needs ZooKeeper (or KRaft), careful partition planning, and schema management.

NATS: Simplicity at Speed

NATS is a lightweight, high-performance messaging system written in Go. Core NATS is pure pub/sub with no persistence — extremely fast, sub-millisecond latency, dead-simple ops. JetStream adds persistence, at-least-once delivery, and stream replay. NATS excels in microservice meshes where you want low latency and simple operations. A single NATS binary with no external dependencies.

RabbitMQ: Flexible Routing

RabbitMQ implements AMQP and is built around exchanges and queues with rich routing logic — fanout, direct, topic, headers. It's ideal for task queues where workers pull jobs, acknowledgement matters, and you need dead-letter queues and retry policies. Classic use case: background job processing where you can't afford to lose a task.

The Decision Matrix

• ▸Need replay / audit log / event sourcing? → Kafka
• ▸Need simple task queue with retries and dead-lettering? → RabbitMQ
• ▸Need low-latency microservice communication with minimal ops? → NATS
• ▸Already using Redis and need lightweight pub/sub or stream? → Redis Streams
• ▸High throughput analytics pipeline feeding multiple consumers? → Kafka
• ▸Cloud-native service mesh on Kubernetes? → NATS or Kafka (MSK/Confluent)

The mistake I see most often: using Kafka for a simple task queue because it sounds impressive, then spending a week fighting consumer group offsets. Pick boring tools that match your actual scale.