LetItCrash

The BEAM is the broker: pub/sub without a broker

LLM-authored, human-reviewed

Concurrency & the BEAM

Every platform treats publish-subscribe as infrastructure: a broker fleet - Kafka, RabbitMQ, NATS, a managed service - deployed and operated between publishers and subscribers. The BEAM virtual machine, the runtime shared by Erlang and Elixir, takes the opposite position. On the BEAM, message passing is the machine’s fundamental operation. Pub/sub is not a service to install. It is a library-level composition of primitives the VM already provides. That makes the pattern a runtime primitive here, rather than a product, and gives the pattern from the previous article its native habitat. This site is one example: every admin builder LiveView subscribes to a topic like exercises:updated, the context broadcasts on data changes, and the panel updates with no refresh. No framework feature is bolted on. Processes, mailboxes, and message passing are doing the work.

The process model is the messaging foundation

A BEAM process is not an OS process or a thread. It is a user-space unit of concurrency costing roughly 2 KB, with its own heap, its own garbage collector, and preemptive scheduling. A single node can run millions of them without letting one starve the others. Each process owns a mailbox

  • a private, ordered queue - and communicates exclusively by sending immutable messages. There is no shared memory, so there are no data races or locks to guard against them. The runtime guarantees ordering of signals between any two processes. That gives pub/sub per-sender ordering for free: the same guarantee Kafka sells per-partition comes from the primitive. In JavaScript terms, every BEAM process is an isolated worker with its own event queue, except that millions coexist on one VM. The concurrency article explains this model on its own terms. Here, the question is what it means for messaging.

The design comes from 1986 telecom. Ericsson’s switches carried hundreds of thousands of concurrent calls, and three properties were non-negotiable: one call’s failure could not affect others, the system could not stop for upgrades, and the traffic was all message passing. That heritage produced the platform’s two signature moves. Let it crash means a process that hits an unexpected state dies and is restarted into a known-good state, rather than defensively catching every error. In pub/sub, that becomes self-healing subscribers: a crashed channel handler or LiveView is restarted by its supervisor and resubscribes, and no broker ever notices. OTP supervision contains the failure at the level that clears corrupted state. That is what a subscriber needs when its mailbox has been poisoned. The let-it-crash article explains the philosophy; this is the pattern it makes self-healing.

One structural honesty note matters here. The decoupling theory from the previous article is precise: BEAM message passing is space-decoupled (senders address names, not locations, via registries and :pg groups) but not time-decoupled. A message to a dead process vanishes. Nothing stores or replays it. BEAM pub/sub is ephemeral in the temporal dimension, which separates it from broker pub/sub throughout the ecosystem.

A one-process broker

A pedagogical fan-out you can hold in one function: one process owns a subscriber list and copies each publish into every mailbox.

defp loop(subs) do
  receive do
    {:subscribe, pid} ->
      loop([pid | subs])

    {:publish, msg} ->
      Enum.each(subs, &send(&1, msg))
      loop(subs)
  end
end

That sketch is the message-passing idea in isolation - a mailbox, a list, and send. It is not what Phoenix.PubSub is underneath. PubSub on the BEAM is the runtime: subscriptions live in node-local ETS, a :pg process group spans the cluster, and there is no broker process of its own. The next section is that architecture. In the sketch the process is the lock; real PubSub does not serialize every broadcast through one process. Ping-Pong Counter is two processes and no list; Topic Pub/Sub with Wildcards is this loop with topics and a dead-subscriber cleanup.

Phoenix.PubSub: a thin adapter over the runtime

Phoenix.PubSub exposes the conventional API - subscribe, broadcast - but has no broker process of its own. Its default adapter keeps each node’s subscriptions in node-local ETS tables, a :pg group spans the cluster, and a cross-node broadcast forwards exactly one message per remote node. Each node then fans out locally. Distributed Erlang handles all inter-process communication. There is no persistence, replay, or queueing. Delivery is ephemeral fire-and-forget, matching the process model underneath it.

The adapter’s :pool_size option comes from the platform’s most famous benchmark. In 2015 Phoenix reached 2 million WebSocket connections on a single 40-core machine, and at roughly 1.3 million subscribers a single PubSub server and ETS table became the bottleneck. Subscriptions were sharded by subscriber process ID across a pool of servers, each with its own ETS table. That restored one-second broadcasts to all 2 million clients. The important detail is not the number. The fix was a config option on a library that owns no broker. The officially supported Phoenix.PubSub.Redis adapter exists for environments where Erlang clustering is impractical: Kubernetes, Heroku, Fly.io-style platforms where node discovery is awkward, or deployments already running managed Redis. It is a deployment workaround, not the preferred architecture. This site runs the default adapter with zero configuration: single node, ETS tables, no Redis. A broker with no broker.

LiveView is a subscriber

The canonical LiveView pattern is the one this site’s builders use line for line: Phoenix.PubSub.subscribe/2 in mount (guarded by connected?/1 so the initial render is not a pub/sub round-trip), broadcast on data changes, and receive in handle_info/2. Delivery is VM message passing into the view’s mailbox, just like any other process-to-process message. The view’s handle_info clauses are ordinary code, so test them like any message handling. The three higher-level abstractions use the same substrate. Channels multiplex topic-scoped bidirectional messaging over a WebSocket, with a “fastlane” optimization that encodes each outgoing broadcast once and writes it directly to thousands of sockets. LiveView keeps each client’s UI state in a BEAM process. That subscriber is born, crashes, and resubscribes under its supervisor like any other. Presence is the distributed-systems showpiece: it replicates presence metadata via a heartbeat protocol and a CRDT, with no single source of truth and no global process. It explicitly rejects the Redis-backed central store because the runtime does not need one. When the previous article asked what happens to the slowest subscriber, the BEAM’s structural answer for the browser tier was the socket itself: messages to a disconnected client simply do not arrive, and the LiveView that reconnects resubscribes fresh.

Backpressure: the demand inversion

Fire-and-forget pub/sub cannot protect a slow consumer from a fast producer. The BEAM ecosystem’s answer is the fourth philosophy from the previous article: invert control. GenStage is the official Elixir behaviour for exchanging events with backpressure. Producers and consumers subscribe to one another, demand flows upstream and events flow downstream, and the default dispatcher routes events to the consumer with the highest outstanding demand. No consumer receives more than it requested. Events move only as fast as stages pull them. The documented weakness mirrors the strength: demand-lazy pipelines fit hard real-time push streams poorly. In that case, the right move is a process per event. Broadway productizes GenStage for ingestion - producers for Kafka, RabbitMQ, SQS, and Google Pub/Sub, with automatic acknowledgements, batching, rate limiting, and graceful shutdown. It acknowledges only after end-of-pipeline processing succeeds. The architecture to internalize is the ecosystem’s own: Kafka and RabbitMQ remain the durable brokers of record; Broadway is how BEAM applications consume them with backpressure, and Phoenix.PubSub is how they move events inside the cluster without a broker at all.

The two-tier convergence, collapsed

The previous article’s insight about mature real-time systems was that they converge on two tiers: a connection layer holding sockets, and a pub/sub backplane routing between them. Managed services like Ably and Pusher sell that architecture as a product. The BEAM is the notable exception because it collapses both tiers into one runtime: the same VM that holds the sockets also routes the broadcasts. The scale evidence is the strongest argument. WhatsApp pushed a single server past 2 million TCP connections in 2012, sustained more than 70 million Erlang messages per second, and served roughly 450 million users with about 32 engineers on a custom-patched BEAM. Discord runs its chat infrastructure - 20-plus Elixir services, 400 to 500 machines, tens of millions of messages per second - with five engineers. It also maps the model’s limits honestly: default Distributed Erlang is fully meshed, so at that fleet size Discord uses a partial mesh with service discovery, and when pure Elixir hit hot-path limits it dropped to Rust NIFs. The research reaches a precise verdict: BEAM pub/sub trades the broker’s time-decoupling and durability for zero-infrastructure, process-granular delivery inside a trusted, homogeneous cluster. When that trade stops holding - WAN links, untrusted networks, multi-language clients, durability, ephemeral orchestrated platforms - the ecosystem reaches for Redis and Kafka through the Redis adapter and Broadway respectively.

What the platform teaches

Everything this site teaches about concurrency prepares you for this pattern. Processes with private mailboxes are the subscription model. Immutable message passing is the transport. Let-it-crash supervision is the delivery guarantee - at-least-once in the sense that a crashed subscriber restarts and catches up, with idempotency handled the way every system handles it, by the consumer. The concurrency article and the BEAM article give you the machinery; the previous article gives you the pattern; this one joins the two. When you write a broadcast in a context and a handle_info in a LiveView, you are doing publish-subscribe the way the VM intended it: not by installing a broker, but by using the runtime’s fundamental operation. That is why the pattern that takes infrastructure everywhere else takes configuration nowhere on the BEAM.

Where to go next

  • Process Ring - the exercise that makes this article’s mechanism concrete: spawn a ring of processes, wire each to its successor, and pass a token around. The value returned is arithmetic; the ring is the point.
  • Ping-Pong Counter - two processes bouncing a counter. The mailbox is the broker.
  • Topic Pub/Sub with Wildcards - a broker as an exercise: topics, wildcards, and a dead subscriber cleaned up.
  • Publish-subscribe - the pattern itself, and its four embodiments across brokers, browsers, webhooks, and hot reload.
  • What concurrency is
    • processes, mailboxes, and the model this whole article assumes.
  • What the BEAM actually is - the runtime whose fundamental operation is message passing.
  • Let it crash - why the BEAM’s subscribers heal themselves.
  • Orchestration
    • composing processes into services, where the broadcasts originate.
← Back to articles

Exercises