Blog / 2026-05-07

Sizing ring buffers: the retention math

2026-05-077 min read

Retention on Runnev is bounded two ways at once, by age and by bytes, and the defaults are not arbitrary. This is the math we use to size a stream's ring buffer, worked through with real numbers, so you can pick retention_seconds and max_bytes deliberately instead of guessing.

Two bounds, whichever hits first

A stream keeps a batch until it falls outside either its age bound (retention_seconds) or its size bound (max_bytes). Eviction removes the oldest batches first. Two bounds instead of one is not belt-and-braces; each protects against a different failure. The age bound guarantees a slow consumer a minimum time to catch up regardless of volume. The size bound guarantees a runaway producer cannot consume unbounded memory regardless of time.

Start from bytes per event

Everything begins with the size of one event on the wire, including our framing. Measure it; do not guess, because estimates of JSON size are wrong by a factor of two in both directions depending on key names and nesting. For a worked example, take a typical order event at about 240 bytes serialized, batched a hundred to a publish, with framing that rounds a batch to roughly 24.5 KiB.

event_bytes   = 240          # measured, serialized, one event
batch_events  = 100
batch_bytes   = event_bytes * batch_events + framing  # ~= 24_576 bytes
batches_per_s = 8            # your steady publish rate in batches/second

# how many bytes does one hour of retention need?
seconds       = 3600
needed_bytes  = batch_bytes * batches_per_s * seconds
# 24_576 * 8 * 3600  ~= 675 MiB per hour

That single number is the lever. At eight batches a second of 24.5 KiB each, an hour of history is about 675 MiB. If your max_bytes is the 256 MiB default, you do not have an hour; you have about 23 minutes, because the size bound will evict long before the age bound does. The two bounds must be sized together or one of them is a lie.

Making the bounds agree

The rule is simple: set max_bytes to at least the bytes your target retention_seconds will accumulate at your peak publish rate, not your average. Using averages here is the most common sizing mistake we see, because event streams are bursty and the burst is exactly when a consumer is most likely to fall behind and most need the history to be there.

target_seconds = 3600
peak_batches_s = 20          # peak, not average
max_bytes      = batch_bytes * peak_batches_s * target_seconds
# 24_576 * 20 * 3600  ~= 1.65 GiB  -> request this, or lower the age target

If that number is larger than you want to pay for, the honest move is to lower retention_seconds to match the max_bytes you are willing to hold, so the age bound reflects reality. A stream that claims an hour of retention but evicts after twenty minutes under load is worse than one that honestly promises twenty.

Why the defaults are what they are

The default is 24 hours and 256 MiB. That pairing is tuned for the common case we see: modest, structured event streams in the low single digits of batches per second, where 256 MiB comfortably holds well over a day. For that traffic the age bound is the one that does the work and the size bound is a safety net that never triggers, which is exactly the arrangement you want. It is the wrong default for a high-rate firehose, which is why both are configurable at creation, and why this post exists.

Picking your numbers

The procedure, condensed:

1. Measure event_bytes for a real event. Do not estimate.
2. Decide retention_seconds from your worst realistic consumer outage, plus margin. A deploy that takes ten minutes wants far more than ten minutes of retention.
3. Compute the bytes that retention needs at your peak publish rate.
4. Set max_bytes to that, or, if it is too large, lower retention_seconds so the two bounds tell the same story.
5. Alert on sequence gaps in your consumers; a gap means a consumer outran retention and the numbers need revisiting.

Sizing a ring buffer is not glamorous, but it is the difference between a stream that quietly holds your history when you need it and one that evicts it the moment things get busy. Spend the ten minutes with the calculation above.

← All posts