What long-lived HTTP responses taught us about intermediaries
A core promise of Runnev is that you can hold one HTTP response open for as long as you need. In practice that means understanding how the buffers and idle timeouts along the path treat a streaming response. This is a short field guide — nothing exotic, just the streaming-friendly settings any long-lived SSE endpoint benefits from.
The claim we had to earn
"Subscribe with a long-lived GET" is easy to say and easy to demo for thirty seconds. The hard part is the connection that has been open since Tuesday. Every hop in the path, the client's HTTP library, any forward proxy, the CDN edge, our reverse proxy, and the application, has an opinion about how long a response may take and how much of it to hold before passing it on. A long-lived stream just needs each of those hops configured for streaming — which is standard, once you know the settings.
Our own connections routinely stay open for hours. Getting there is not a fight with anything; it is knowing, at each hop, the handful of streaming-friendly settings that most platforms already provide or expose.
Buffering: the silent latency
The first symptom was not a disconnect; it was latency. Batches would arrive in clumps seconds apart instead of as they were published. The cause was response buffering: a proxy accumulating our output until it had a bufferful before forwarding any of it. For a normal response that is a throughput optimization. For a stream it is a bug, because the whole point is to forward each batch the instant it exists.
The fix is one idea, and it is the same one every SSE endpoint needs: turn off response buffering on the streaming path, so each write is forwarded the moment it lands instead of held until a buffer fills. If you run your own reverse proxy that is a one-line change, and most managed platforms and CDNs either do it for streaming responses or expose a switch for it.
We forward each write immediately, and the application sends X-Accel-Buffering: no
on every streaming response — a widely understood hint that asks a reverse proxy in the path
to flush rather than buffer. It is cheap insurance: it costs nothing when nobody is listening
and helps when someone is.
Idle timeouts, stacked
Once batches flowed promptly, the next failure was connections dying at almost exactly 60 seconds when a stream went quiet. That is the classic idle-timeout signature, and the trap is that there is never just one timeout. There is one on the client's HTTP library, one on each proxy, one on the load balancer, and one on our server, and the connection dies at the shortest of them.
Two things fixed it. First, the keepalive comment frame every 15 seconds means the connection is never actually idle from any hop's point of view; there is always traffic within the last 15 seconds, so a 60-second idle timer never fires. Second, we raised the read and idle timeouts on the components we control to comfortably exceed the keepalive interval, with margin, so a single delayed keepalive does not trip anything.
The keepalive interval is a genuine trade-off. Shorter means more resilient to aggressive intermediaries but more wasted bytes on quiet streams; longer means leaner but closer to the edge of a 30- or 60-second timer somewhere. Fifteen seconds sits comfortably under a 60-second timeout even if one frame is delayed, and the byte cost is trivial, a few bytes four times a minute per idle connection.
The client side has timeouts too
A subtlety that cost us a day: many HTTP clients have a single overall request timeout that includes reading the body. For a normal request, capping the whole thing at 30 seconds is prudent. For a subscribe, it guarantees a disconnect at 30 seconds no matter how healthy the stream is.
The rule we now document and implement in every SDK is to separate the two: keep a normal
timeout on establishing the connection and reading response headers, but do not put an
overall deadline on reading the streaming body. Instead, treat the keepalive as the
liveness signal, and reconnect if no bytes, not even a comment, arrive within a window
comfortably larger than the keepalive interval. In Go that means setting
ResponseHeaderTimeout but not Client.Timeout on the subscribe
path. In Python it means a socket read timeout set above the keepalive interval rather
than a wall-clock cap on the whole read.
Why every streaming response is no-store
One more thing worth stating plainly: caching. A cache anywhere
in the path that decides an event stream is a cacheable GET will hold the
response, try to serve it to the next subscriber, and break both. Every
/v1 response, and the subscribe response especially, carries
Cache-Control: no-store. Combined with X-Accel-Buffering: no,
that is the pair of headers that tells the whole path to treat the response as a live
pipe and not an object to store.
The lesson
A long-lived HTTP response is not exotic — it is how SSE works — but it does interact with the buffers and timeouts along the path. Making it reliable is mostly about knowing the handful of streaming-friendly settings above, the same ones any SSE endpoint uses. If you run infrastructure in front of a streaming service, that short list is most of what you need.