How to autoscale Heroku
Heroku’s awesome, but some stuff is still a pain in the ass.
It’s great I can scale my app with the push of a button. But I don’t want to push a button. I just want it to happen.
We’ve been running Cityposh on Heroku. Some days have big spikes in traffic, and some days have real lows. I don’t want to have to keep looking at our web traffic to see if I need to be asking Heroku to scale for me. And it’s a huge waste of money to just keep a bunch of web workers going if there’s no one using them.
There’s a few solutions for this problem currently. Including a business that will do it for you. And numerous plugins/gems that will try and do it. Someone at Heroku even wrote one:
https://github.com/ddollar/heroku-autoscale
I don’t want to pay more money to make this work since it seems easy enough to fix on my own programatically. But all these gems seem to have some kind of issue. This one from Heroku has this fat warning:
WARNING
This gem is a proof of concept and should not be used in production applications. There is currently no mechanism to prevent multiple web workers on the same app all running this code from fighting each other for control.
The gem is a Rack app that intercepts every single request going to you Rails app. And like it says, multiple dynos are running that same app all concurrently. So you could have multiple dynos all sending commands back to Heroku to scale your app on top of each other. We don’t want that.
But there’s a great way to have things run on Heroku without tying them to every single web worker and do it on a frequent schedule: Clockwork!
https://github.com/tomykaira/clockwork
It’s a replacement for cron.
So my approach was to:
- Have Clockwork + Delayed Job ask our Rails app to auto scale.
- Change ddollar’s gem to scale only when the there’s been an explicit request to do so.
Clockwork is easy to setup. And so is Delayed Job. I’ll assume you have those things figured out.
In my clock.rb file I invoke a Delayed Job I made to ask our app to scale.
every(1.minutes, 'scale_heroku') { Delayed::Job.enqueue ScaleHerokuJob.new }
And all that job does is issue a GET request to Cityposh’s explicit autoscale action (a Rack app).
class ScaleHerokuJob
def perform
HTTParty.get("http://cityposh.com/autoscale/#{ENV['HEROKU_SCALE_KEY']}")
end
end
Next, I just made some quick and dirty changes to ddollar’s gem to make it more explicit.
https://github.com/n8/heroku-autoscale
def call(env)
if env["PATH_INFO"] == "/autoscale/#{options[:autoscale_key]}"
autoscale env
[200, {'Content-Type' => 'text/plain'}, ["Current wait time: #{env["HTTP_X_HEROKU_QUEUE_WAIT_TIME"]}"]]
else
app.call(env)
end
end
Now our Rails app performs all it’s actions normally unless we specifically ask the rails app to autoscale with our special key. The key being in there might be a little unnecessary, but it just protects the url from being called by just anyone reading this blog post.
Make sense?
