Inside Discourse's DistributedCache: Two Levels and a Deferred Write

The thing nobody tells you about Rails caching at scale: Redis round-trips add up. A Rails.cache.fetch call that takes 1ms looks free until you’re doing a hundred of them per request, in which case it’s 100ms of latency you didn’t have to pay.

Discourse’s answer is DistributedCache — a per-process in-memory Hash plus a MessageBus-driven invalidation channel. The hot path reads a Ruby Hash. The cold path writes through and notifies every other process. This is the pattern I quietly steal whenever I have a small, frequently-read, occasionally-written piece of state.

I’m reading discourse/discourse at SHA 866ae39d4642f667eb75a663e672d0eaf32db595. Discourse is GPL-2.0; excerpts are short and verbatim. Footnote at the bottom.

The deferred get-set

The shortest way into the file is the defer_get_set method:

# discourse/discourse · lib/distributed_cache.rb · @866ae39
def defer_get_set(k, &block)
  raise TypeError if !Rails.env.production? && !k.is_a?(String)

  return self[k] if hash.key? k
  value = block.call
  self.defer_set(k, value)
  value
end

Read it slowly. Three branches:

1. Hot path. hash.key? k checks an in-process Ruby Hash. Microseconds. If the key is present, return it directly — no Redis, no IPC, no nothing.
2. Cold path. Call the block to compute the value.
3. Write path. defer_set schedules the write to happen after the current request, so the client doesn’t pay for the broadcast latency.

The TypeError guard in non-production catches a class of bugs I’ve shipped before — passing a symbol or an integer where a string is expected. Cache key collisions across types are a pain to debug; failing loud in dev is the right call.

What hash is

The hash method (not shown above; it’s inherited or defined nearby) returns a per-process Ruby Hash. Every Puma worker has its own. There is no shared memory, no Redis fetch, no serialization on the read path. That’s the entire point.

The cost: the hash gets out of sync. Worker A writes a new value, Worker B still has the old one. This is where MessageBus comes in.

MessageBus as the invalidation bus

Discourse’s message_bus is a Redis-pub/sub-backed message channel. DistributedCache extends MessageBus::DistributedCache, which subscribes every process to a shared channel. When any process writes a key, every other process receives the notification and updates its local hash.

The flow:

1. Worker A: cache.defer_set("foo", 42)
2. defer_set schedules: after the request, publish {op: "set", key: "foo", value: 42} to the channel
3. All workers (including A’s siblings on the same box, and workers on other boxes) receive the message
4. Each updates its local Ruby Hash

The “deferred” part matters. If the publish were synchronous, every cache write would block on a Redis round-trip plus the time for subscribers to acknowledge. By scheduling the publish for after the current request, Discourse avoids the blocking write on the user’s critical path. Slight staleness in exchange for lower tail latency. That’s a trade I’d make on a forum every time.

When this pattern is right

I’ve used this exact shape (in-process Hash + pub/sub invalidation) for:

• Feature flags. Read on every request, written approximately once a day. The hash is tiny and the staleness window is tolerable.
• Site-wide settings. Same shape. Admin updates a setting, every worker picks it up within a second.
• Compiled regex/parsed config. Anything where the parse step is non-trivial and the parsed value is small.

It’s wrong for:

• Per-user data. The hit rate per worker is low, and you blow up memory holding state for users routed to other workers.
• Large values. Every worker on every box keeps its own copy. A 10MB value times 16 workers times 8 boxes is 1.3GB of duplicated state.
• Anything that needs strict consistency. The window between “Worker A wrote it” and “Worker B sees it” is small but nonzero. If you need read-your-write semantics across requests routed to different workers, this isn’t the tool.

The clear method and transactions

The other line worth flagging from DistributedCache is clear, which optionally wraps the cache invalidation in DB.after_commit:

# illustrative — based on the pattern in lib/distributed_cache.rb @866ae39
def clear
  if DB.transaction_open?
    DB.after_commit { super }
  else
    super
  end
end

(I’m reconstructing from the summary rather than the exact lines, so this is illustrative.) The point: if you’re inside a database transaction, defer the cache invalidation until after the commit. Otherwise you risk publishing “this key is invalid” to every worker, having them re-read from the DB, and getting back the old (uncommitted) value.

This is one of the subtle bugs I’ve shipped multiple times: invalidate the cache, then commit. In the gap, every request misses, reloads the stale data, and re-populates the cache with the pre-commit state. The fix is always “invalidate after commit”. Discourse bakes that into the cache primitive itself.

Stealing the pattern

If you want this shape in your own app without pulling in MessageBus, the rough recipe is:

1. A Concurrent::Map (from concurrent-ruby) per process, holding the hot data
2. A Redis pub/sub subscription on Rails boot, listening for invalidation messages
3. A write method that updates the local map and publishes the invalidation
4. A read method that returns from the local map (no fallback to Redis — the map is the truth, populated by the bus)

That’s maybe 80 lines of Ruby. The harder part is being honest about what belongs in such a cache. Most things don’t. The ones that do feel obvious in retrospect.

License footnote

Discourse is licensed under GPL-2.0. The excerpts above are short illustrative quotations of public source code at SHA 866ae39d4642f667eb75a663e672d0eaf32db595, used here for commentary and education. If you build on Discourse’s source, the GPL applies to your derivative work.