How to implement backpressure and load shedding?

Backpressure and load shedding are methods you can use to mitigate queue overload. These are methods that kick in automatically therefore your system needs to have enough instrumentation to know if a queue is overloaded. For example, we can have a queue object which responds to the overloaded? message, (i.e. queue.overloaded?). The definition of queue […]

Why you need backpressure/load shedding for queues?

Backpressure – slowing down producers – and load shedding – dropping messages – are two of the methods you can use to mitigate queue overload. Backpressure and load shedding are reaction mechanisms your producers and/or consumers automatically take during queue overload. These methods are useful because they enforce limits to your queues. A limit is […]

Tracking queue metrics with the ruby-kafka gem

Kafka may be used as a queue to send messages between different systems therefore the relevant metrics should be collected. The ruby-kafka gem has amazing support to track the most important queue metrics. It has out of the box instrumentation for Statsd and Datadog. It also has instrumentation hooks which rely on Active Support Notifications. […]

Tracking queue metrics with Sidekiq

Sidekiq is a critical component of many Rails applications therefore its metrics should be appropriately collected. For example: Imagine you have a Sidekiq job which sends password reset e-mails to customers. If that job’s queue has a latency of 1 hour you want to know about it because it has a significant impact to a […]

Dealing with queue overload

When dealing with queue overload you effectively have two levers: increasing the consumption rate or reducing the production rate. How to increase consumption rate? Add consumers This is about adding boxes/containers. An example is whenever you add Sidekiq workers or Kafka consumers to a consumer group. This is a common resolution because it does not […]

Which metrics to monitor for queues?

Queues are popular building blocks in web applications. They are used to send messages between different systems (e.g. using Kafka or RabbitMQ). They are used to schedule work in background job processing frameworks (e.g. Sidekiq, Resque). They are used by web app servers (e.g. Unicorn) to briefly stage requests during overload. They are the backbone […]

Selling boards

I love board sports. Once I got the hang of the glide I couldn’t let it go. Buying boards is so hard though. Specially when you are a beginner. So many brands, so many shapes, so many camber profiles, so many riding styles. I think the majority of the time when selling/marketing boards should be […]

Some thoughts on learning

I spend a fair amount of my time learning about things. Lately, I’ve been observing myself learning and trying to distill what works, what doesn’t. So, this is a list of some recurrent thoughts, beliefs, approaches on learning I have. Learning new things is hard. I think it is specially hard because as we grow […]

A non exhaustive list of 2017 picks

Naval’s interview on the Farnam Street podcast was the best thing I’ve heard this year. I think about many of the insights daily. One that has particularly stuck with me “all the valuable things in life are long term focused: relationships, family, compounding, careers, health” Release it! and the SRE book were my favorite technical […]