Backpressure in Queues

The first law of queues is: Every queue should have a maximum size. Queues must not grow unbounded.

The second law of queues is: Have you considered a maximum size of 0? A queue of size zero in a distributed system is pretty much a load balancer. The “queue’s” job is just directly connecting a producer with a consumer (a client with a server). (Of course, those servers have TCP accept queues… there’s always a queue involved. But those also have bounds!)

It’s surprising how often queues are used when there’s actually little reason to do so. The primary reason queues make sense is switching from synchronous to asynchronous requests. There are other times to reach for one, of course, but they’re often viewed as something helpful. They should be viewed as something potentially dangerous.

There’s nothing better than a queue for smoothing over problems today, only to make them explode bigger in the future.

https://www.tedinski.com/2019/03/05/backpressure.html

strategically place queues in your architecture, artificially limit size of queues in stress tests (like how you can use rlimit for other resources), and track the derivative of queue fill.

Reject new entries when the queue is full — all the way back down the line to whatever wanted to do the work in the first place, such as a user.