Why not Kafka, RabbitMQ or Redis/Bull
Parent: events
Status: released in v0.1.76 (2026-09-27) — design and as-built record in docs/design/event-bus-outbox-webhooks.md in the StandMeet repo.
The queue lives in the Postgres we already run, and no new service is added. River is one existing way to build a queue on Postgres; a self-built minimal queue on Postgres keeps the stack just as small. Four reasons rule out a broker.
1. A broker does not remove the outbox
The domain change and "the event happened" must commit together. Kafka, RabbitMQ and Redis cannot join a Postgres transaction. Using them still needs an outbox plus a relay that publishes to the broker, which only adds a hop. A queue in the same database makes enqueueing part of the transaction itself.
2. Wrong semantics
We need a job queue: each delivery retried on its own, on a backoff schedule, with an endpoint disabled after sustained failure.
- Kafka is an ordered log. It has no per-message delayed retry, so it needs hand-built retry and dead-letter topics, and one failing message blocks its partition.
- RabbitMQ can do delayed retry, but only by composing dead-letter exchanges and TTLs, or with a plugin.
3. Our Redis is built to lose data
docker-compose.prod.yml caps Redis at 256 MB with allkeys-lru: when full, it evicts the oldest keys. Sessions and rate-limit buckets survive eviction because they can be rebuilt; queued jobs would be dropped silently. The repo has no Bull or BullMQ, and the backend is Go, so Bull would also add a Node process.
4. Volume and cost
The event volume is hundreds to a few thousand a day: corpus edits, access requests, bookings. Postgres handles thousands of such writes per second. Self-hosting on a 1 GB box is part of the product.
Comparison
| Option | New service | Typical memory | Transactional | Per-message delayed retry | Operations |
|---|---|---|---|---|---|
| Kafka | broker (KRaft still runs a JVM) | 1 GB and up | dual write, needs outbox | no; needs retry topics | partitions, retention, disks, upgrades |
| RabbitMQ | Erlang server | 150 MB and up | dual write, needs outbox | composed from DLX + TTL | queues, exchanges, durability config |
| Redis + Bull | none, but eviction and persistence must change | shared | dual write, needs outbox | yes | an extra Node process |
| River | none (a few tables in the existing Postgres) | near zero | same transaction | yes | backed up and migrated with the database; +1 Go module |
| Self-built on Postgres | none | near zero | same transaction | we implement it | a few hundred lines we own; the builder's SKIP LOCKED lease is a start |
Storage growth does not vanish with another choice either: a Redis queue grows the same way and our Redis evicts silently when full; Kafka has built-in retention by time or size, at the cost of running another service. See storage-bounds.
When Kafka would make sense
A large multi-tenant SaaS, with many independent services consuming and replaying the same stream. Only the job runtime behind Jobs, Inspector and Runtime, and the relay inside internal/infra/events, would change; the domains would not notice (queue-behind-ports).
A further option to shrink the stack
Move sessions and rate limits into Postgres (UNLOGGED tables), and Redis can be removed entirely. This is a separate proposal.