Backpressure – slowing down producers – and load shedding – dropping messages – are two of the methods you can use to mitigate queue overload. Backpressure and load shedding are reaction mechanisms your producers and/or consumers automatically take during queue overload. These methods are useful because they enforce limits to your queues. A limit is […]
Kafka may be used as a queue to send messages between different systems therefore the relevant metrics should be collected. The ruby-kafka gem has amazing support to track the most important queue metrics. It has out of the box instrumentation for Statsd and Datadog. It also has instrumentation hooks which rely on Active Support Notifications. […]
Sidekiq is a critical component of many Rails applications therefore its metrics should be appropriately collected. For example: Imagine you have a Sidekiq job which sends password reset e-mails to customers. If that job’s queue has a latency of 1 hour you want to know about it because it has a significant impact to a […]
When dealing with queue overload you effectively have two levers: increasing the consumption rate or reducing the production rate. How to increase consumption rate? Add consumers This is about adding boxes/containers. An example is whenever you add Sidekiq workers or Kafka consumers to a consumer group. This is a common resolution because it does not […]
A queue is a simple system: producers place messages into the queue; consumers remove messages from the queue; whenever the produce rate is higher than the consume rate for a extended period of time you have queue overload. By overload, I mean the queue either becomes full or is growing without stopping. In order to […]
Queues are popular building blocks in web applications. They are used to send messages between different systems (e.g. using Kafka or RabbitMQ). They are used to schedule work in background job processing frameworks (e.g. Sidekiq, Resque). They are used by web app servers (e.g. Unicorn) to briefly stage requests during overload. They are the backbone […]
I love board sports. Once I got the hang of the glide I couldn’t let it go. Buying boards is so hard though. Specially when you are a beginner. So many brands, so many shapes, so many camber profiles, so many riding styles. I think the majority of the time when selling/marketing boards should be […]
I spend a fair amount of my time learning about things. Lately, I’ve been observing myself learning and trying to distill what works, what doesn’t. So, this is a list of some recurrent thoughts, beliefs, approaches on learning I have. Learning new things is hard. I think it is specially hard because as we grow […]
Naval’s interview on the Farnam Street podcast was the best thing I’ve heard this year. I think about many of the insights daily. One that has particularly stuck with me “all the valuable things in life are long term focused: relationships, family, compounding, careers, health” Release it! and the SRE book were my favorite technical […]
I read this Tweet today before an early morning flight, so I felt a bit pumped to write down some notes on things I’ve learned this year. Disclaimer: This is mostly unedited and it overly focuses on software/programming related things – I could spend days talking about obscure things about house dance but I guess […]