In my previous post I explained that the signature of a Sidekiq job should be treated as an interface between the Sidekiq client and the Sidekiq server. Therefore, you should pay attention to backward compatibility whenever changes are made to that interface: adding/removing arguments, changing the class name, etc.
The solutions I proposed to the problem mostly involved creating a new Sidekiq job class with the new signature instead of changing the original one.
But what if you have a Sidekiq job where you expect to see lots of changes to its arguments? What if you don’t want to go through the work of creating a new Sidekiq job?
You can use the Parameter Object pattern to get around this. The parameter object encapsulates all the arguments of the job. When making changes to the job signature the parameter object needs to ensure compatibility between clients.
Let’s use some examples to show how this can be implemented with Sidekiq. When using this pattern instead of the worker below:
class HardWorker include Sidekiq::Worker def perform(name) # do something end end
We would have the following:
class HardWorker include Sidekiq::Worker def perform(parameter_hash) parsed_parameter = Parameter.new(parameter_hash) puts parsed_parameter.name # do work end end class Parameter def initialize(job_args) @job_args_hash = job_args end def name job_args_hash["name"] end def to_json job_args_hash.to_json end end # running parameter = Parameter.new("name" => "john") HardWorker.perform_async(parameter)
Parameter class needs to be serializable to JSON because that is the format in which Sidekiq sends information between clients and servers. We get around that restriction by implementing
Upon execution, Sidekiq will deserialize the JSON representation and transform its argument into a hash. Therefore our
Parameter class needs to instantiated from its hash representation whenever the job is executed.
Parameter object needs to provide methods for the job arguments it encapsulates (e.g.
When we run into a situation in which we want to add an argument like
count we add support for it through the Parameter object:
# we only change the parameter object class Parameter def initialize(job_args) @job_args_hash = job_args end def name job_args_hash["name"] end def count # added 1 for compatibility purposes job_args_hash["count"] || 1 end def self.from_json(json) hash = JSON.parse(json) new(hash) end def to_json job_args_hash.to_json end end class HardWorker include Sidekiq::Worker def perform(parameter_json) parsed_parameter = Parameter.from_json(parameter_json) puts parsed_parameter.name # job changed to use the new parameter puts parsed_parameter.count # do work end end # running parameter = Parameter.new("name" => "john", "count" => 5) HardWorker.perform_async(parameter)
The parameter object needs to provide access to the new argument and provide compatibility whenever that argument is not present (e.g.
This pattern does not cause problems during deploys:
- the Sidekiq server from the older version will ignore any
Parameterobjects with the “count” field in its hash representation. It will be running a version of the code that does not use it even if enqueued.
- the Sidekiq server from the newer version is able to work even if the “count” field is not present due to having being enqueued by an older client. This is possible because
Parameter#counthas a fallback value.
Can’t you use a hash argument instead of using the Parameter Object?
Yes you can. The hash representation (converted to JSON) is eventually what gets transmitted over Sidekiq through Redis so, alternatively you could have used a hash and saved yourself some trouble.
The advantage of the Parameter Object is that you can explicitly encode the expected behavior in a dedicated object. That is particularly useful when defining the behavior for backward compatibility scenarios. The object is also useful for defining other sane defaults within the methods. You can also extend the object to have factory-like methods that conveniently create parameter objects with common attributes.
The Parameter Object does come with additional complexity so you need to consider if it is worth it.
I prefer using the approach I outlined in my previous post for this type of problems: adding a new Sidekiq job with the changes in signature.
Nevertheless, the Parameter Object pattern is an alternative approach you have at your disposable which can be easily implemented. It may be useful if your job arguments have high turnover or if you have a large set of Sidekiq job arguments.
I can send you my posts straight to your e-mail inbox if you sign up for my newsletter.