Sidekiq Parameter Object Pattern

In my previous post I explained that the signature of a Sidekiq job should be treated as an interface between the Sidekiq client and the Sidekiq server. Therefore, you should pay attention to backward compatibility whenever changes are made to that interface: adding/removing arguments, changing the class name, etc.

The solutions I proposed to the problem mostly involved creating a new Sidekiq job class with the new signature instead of changing the original one.

But what if you have a Sidekiq job where you expect to see lots of changes to its arguments? What if you don’t want to go through the work of creating a new Sidekiq job?

You can use the Parameter Object pattern to get around this. The parameter object encapsulates all the arguments of the job. When making changes to the job signature the parameter object needs to ensure compatibility between clients.

Let’s use some examples to show how this can be implemented with Sidekiq. When using this pattern instead of the worker below:

class HardWorker
  include Sidekiq::Worker
  def perform(name)
    # do something
  end
end

We would have the following:

class HardWorker
  include Sidekiq::Worker

  def perform(parameter_hash)
    parsed_parameter = Parameter.new(parameter_hash)
    puts parsed_parameter.name
    # do work
  end
end

class Parameter
  def initialize(job_args)
    @job_args_hash = job_args
  end

  def name
    job_args_hash["name"]
  end 

  def to_json
    job_args_hash.to_json
  end
end

# running
parameter = Parameter.new("name" => "john")
HardWorker.perform_async(parameter)

Our Parameter class needs to be serializable to JSON because that is the format in which Sidekiq sends information between clients and servers. We get around that restriction by implementing Parameter#to_json.

Upon execution, Sidekiq will deserialize the JSON representation and transform its argument into a hash. Therefore our Parameter class needs to instantiated from its hash representation whenever the job is executed.

Lastly, the Parameter object needs to provide methods for the job arguments it encapsulates (e.g. Parameter#name).

When we run into a situation in which we want to add an argument like count we add support for it through the Parameter object:

# we only change the parameter object
class Parameter
  def initialize(job_args)
    @job_args_hash = job_args
  end

  def name
    job_args_hash["name"]
  end 

  def count
    # added 1 for compatibility purposes
    job_args_hash["count"] || 1
  end

  def self.from_json(json)
    hash = JSON.parse(json)
    new(hash)
  end

  def to_json
    job_args_hash.to_json
  end
end

class HardWorker
  include Sidekiq::Worker

  def perform(parameter_json)
    parsed_parameter = Parameter.from_json(parameter_json)
    puts parsed_parameter.name
    # job changed to use the new parameter
    puts parsed_parameter.count
    # do work
  end
end

# running
parameter = Parameter.new("name" => "john", "count" => 5)
HardWorker.perform_async(parameter)

The parameter object needs to provide access to the new argument and provide compatibility whenever that argument is not present (e.g. Parameter#count).

This pattern does not cause problems during deploys:

the Sidekiq server from the older version will ignore any Parameter objects with the “count” field in its hash representation. It will be running a version of the code that does not use it even if enqueued.
the Sidekiq server from the newer version is able to work even if the “count” field is not present due to having being enqueued by an older client. This is possible because Parameter#count has a fallback value.

Can’t you use a hash argument instead of using the Parameter Object?

Yes you can. The hash representation (converted to JSON) is eventually what gets transmitted over Sidekiq through Redis so, alternatively you could have used a hash and saved yourself some trouble.

The advantage of the Parameter Object is that you can explicitly encode the expected behavior in a dedicated object. That is particularly useful when defining the behavior for backward compatibility scenarios. The object is also useful for defining other sane defaults within the methods. You can also extend the object to have factory-like methods that conveniently create parameter objects with common attributes.

The Parameter Object does come with additional complexity so you need to consider if it is worth it.

Conclusion

I prefer using the approach I outlined in my previous post for this type of problems: adding a new Sidekiq job with the changes in signature.

Nevertheless, the Parameter Object pattern is an alternative approach you have at your disposable which can be easily implemented. It may be useful if your job arguments have high turnover or if you have a large set of Sidekiq job arguments.

I can send you my posts straight to your e-mail inbox if you sign up for my newsletter.