Setting up office hours

I am setting up office hours. My goal is to be available to help folks regardless if they are part of my immediate peer group. I am primarily interested in helping on topics related to career advice or software development. This is still an experiment. Therefore, I am open to see how the format evolves […]

Traveling is a work in progress

Traveling is a system I continuously try to improve. Improving this makes my trips smoother and more enjoyable.  I think improving at traveling requires intention. Traveling takes you out of your natural habitat – your city and your routine. To make traveling more pleasant, you need to actively mold this foreign environment.  Below is a […]

Working in unfamiliar codebases

Working with unfamiliar codebases is uncomfortable. It is easy to get overwhelmed with everything you don’t know. There are deliberate actions you can take to be an “effective beginner” in unfamiliar codebases. Familiar codebases are within your comfort zone. Someone asks you to add a feature and you know exactly where to add it and […]

Always manually send heartbeats when processing message batches in ruby-kafka

I have found it useful to always manually send heartbeats when processing message batches with ruby-kafka. Not doing it often leads to instability in the consumer group at the worst possible time – when you have a performance degradation within the consumer loop. Consumer group instability means: frequent consumer group rebalances. Frequent consumer group rebalances […]

Sidekiq Parameter Object Pattern

In my previous post I explained that the signature of a Sidekiq job should be treated as an interface between the Sidekiq client and the Sidekiq server. Therefore, you should pay attention to backward compatibility whenever changes are made to that interface: adding/removing arguments, changing the class name, etc. The solutions I proposed to the […]

An overview of Redis::Distributed – Redis client side partitioning in Ruby

Redis::Distributed is the Ruby implementation of Redis client side partitioning in Ruby. Partitioning (also known as sharding) is the process of taking the dataset that would originally be held in a single Redis server and splitting it over multiple Redis servers. Partitioning allows you to distribute writes/reads over a set of nodes – horizontal scaling. […]

How to implement backpressure and load shedding?

Backpressure and load shedding are methods you can use to mitigate queue overload. These are methods that kick in automatically therefore your system needs to have enough instrumentation to know if a queue is overloaded. For example, we can have a queue object which responds to the overloaded? message, (i.e. queue.overloaded?). The definition of queue […]