DAVID RICE

Software Developer

+44 (0) 7590 538 303

46 Hill Street
Belfast, Northern Ireland
BT1 2LB

Faster Rails 3 Deployments on Heroku

14 May 2012 {View Comments}

Deployments on Heroku have gotten slower since the cedar stack, Rails 3.2 and the asset pipeline. Don’t get me wrong, I love all of these things like I love Street Fighter II, we’ve got over the novelty of new, just give me the turbo edition.

Benchmark

For the test, some things were kept controlled each time

  • Emptied the S3 bucket (for asset_sync to fully run)
  • No new bundled gems
  • No migrations

So, using a stupidly basic Rails application (we use for testing releases of asset_sync) https://github.com/rumblelabs/asset_sync_test_3_2

git push heroku  0.02s user 0.01s system 0% cpu 1:27.61 total
git push heroku  0.02s user 0.01s system 0% cpu 1:28.91 total
git push heroku  0.02s user 0.02s system 0% cpu 1:34.35 total
git push heroku  0.02s user 0.02s system 0% cpu 1:28.45 total
git push heroku  0.02s user 0.01s system 0% cpu 1:26.65 total

=> [87.61, 88.91, 94.35, 88.45, 86.45]

AVERAGE: 89.154

Note: yes this app I am testing with is only “just” more complicated than rails new, so doing some timing on a completely fresh Heroku app would give a more accurate estimate.

Custom asset precompile task

My first idea was to only run rake assets:precompile:primary not for the faint hearted, but if you have built your application correctly to use digested assets throughout, it’s a no-brainer.

This ruby buildpack allows configuration of the task to execute for precompiling assets ENV['RAILS_ASSETS_PRECOMPILE_TASK'].

heroku config:add BUILDPACK_URL='https://github.com/rumblelabs/heroku-buildpack-ruby.git#custom_asset_precompile'
heroku config:add RAILS_ASSETS_PRECOMPILE_TASK=assets:precompile:primary

git push heroku  0.02s user 0.01s system 0% cpu 1:13.54 total
git push heroku  0.02s user 0.01s system 0% cpu 1:18.94 total
git push heroku  0.02s user 0.01s system 0% cpu 1:13.07 total
git push heroku  0.02s user 0.01s system 0% cpu 1:21.01 total
git push heroku  0.02s user 0.01s system 0% cpu 1:13.80 total

=> 73.54, 78.94, 73.07, 81.01, 73.80

AVGERAGE: 76.072

Woot! That’s an average saving of 13.082 seconds per deploy. Nice.

Note: I had to add an extra hack to get asset_sync working on deploy as we changed the precompile task.

Debug Buildpack

I was wondering if I could save even more time… so here’s a debug buildpack with a quick hack to output timings with every logged message.

heroku config:add BUILDPACK_URL='https://github.com/rumblelabs/heroku-buildpack-ruby.git#buildpack_debug'

The output looks like the following:

➜  myapp git:(master) git push heroku
Counting objects: 5, done.
Delta compression using up to 2 threads.
Compressing objects: 100% (3/3), done.
Writing objects: 100% (3/3), 280 bytes, done.
Total 3 (delta 2), reused 0 (delta 0)

-----> Heroku receiving push
-----> Fetching custom buildpack... done
-----> Ruby/Rails app detected
======> 2012-05-14 06:17:13 +0000
-----> 0.000107585 : Installing dependencies using Bundler version 1.2.0.pre
       0.073559081 : Running: bundle install --without development:test --path vendor/bundle --binstubs bin/ --deployment
       0.734531922 : Using rake (0.9.2.2)
       0.735997602 : Using i18n (0.6.0)
       0.736449923 : Using multi_json (1.3.4)
       0.737163615 : Using activesupport (3.2.0)
       0.737576858 : Using builder (3.0.0)
       0.738049721 : Using activemodel (3.2.0)
       0.738518515 : Using erubis (2.7.0)
       0.739293731 : Using journey (1.0.3)
       0.739688269 : Using rack (1.4.1)
       0.740488658 : Using rack-cache (1.1)
       0.740923127 : Using rack-test (0.6.1)
       0.741390205 : Using hike (1.2.1)
       0.742001929 : Using tilt (1.3.3)
       0.742994436 : Using sprockets (2.1.2)
       0.743355851 : Using actionpack (3.2.0)
       0.743878451 : Using mime-types (1.18)
       0.744305491 : Using polyglot (0.3.3)
       0.744781819 : Using treetop (1.4.10)
       0.74568785 : Using mail (2.4.1)
       0.747632709 : Using actionmailer (3.2.0)
       0.747760727 : Using arel (3.0.2)
       0.748128902 : Using tzinfo (0.3.32)
       0.748552448 : Using activerecord (3.2.0)
       0.74907975 : Using activeresource (3.2.0)
       0.749463748 : Using excon (0.13.4)
       0.749979571 : Using formatador (0.2.1)
       0.750377764 : Using net-ssh (2.3.0)
       0.750844938 : Using net-scp (1.0.4)
       0.751279192 : Using nokogiri (1.5.2)
       0.760296186 : Using ruby-hmac (0.4.0)
       0.76162912 : Using fog (1.3.1)
       0.778984194 : Using asset_sync (0.4.0) from git://github.com/rumblelabs/asset_sync.git (at master)
       0.779744754 : Using coffee-script-source (1.2.0)
       0.780276683 : Using execjs (1.3.0)
       0.78084057 : Using coffee-script (2.2.0)
       0.781371467 : Using rack-ssl (1.3.2)
       0.781935092 : Using json (1.6.6)
       0.782476196 : Using rdoc (3.12)
       0.783709004 : Using thor (0.14.6)
       0.784896558 : Using railties (3.2.0)
       0.785776908 : Using coffee-rails (3.2.2)
       0.786299563 : Using jquery-rails (2.0.1)
       0.787018031 : Using pg (0.13.2)
       0.787546051 : Using bundler (1.2.0.pre)
       0.788116001 : Using rails (3.2.0)
       0.788627757 : Using sass (3.1.15)
       0.790148688 : Using sass-rails (3.2.4)
       0.790633587 : Using uglifier (1.2.3)
       0.793843146 : Your bundle is complete! It was installed into ./vendor/bundle
       0.818492178 : Cleaning up the bundler cache.
-----> 2.714365124 : Writing config/database.yml to read from DATABASE_URL
-----> 4.776435204 : Preparing app for Rails asset pipeline
       4.77724766 : Running: rake assets:precompile
       22.609565246 : AssetSync: using default configuration from built-in initializer
       22.609565246 : AssetSync: using default configuration from built-in initializer
       22.609565246 : AssetSync: Syncing.
       22.609565246 : Using: Directory Search of /tmp/build_1w8jlj9e4aoaz/public/assets
       22.609565246 : AssetSync: Done.
-----> 22.63974359 : Rails plugin injection
       22.639908428 : Injecting rails_log_stdout
       22.875385302 : Injecting rails3_serve_static_assets
-----> Discovering process types
       Procfile declares types      -> (none)
       Default types for Ruby/Rails -> console, rake, web, worker
-----> Compiled slug size is 18.8MB
-----> Launching... done, v48
       http://myapp.herokuapp.com deployed to Heroku

To git@heroku.com:myapp.git
   d2b28d4..d0fc25e  master -> master

Looking at the output we can see that my custom buildpack timings only kick in within =====> CUSTOM BUILDPACK

➜  myapp git:(master) git push heroku
Counting objects: 5, done.
Delta compression using up to 2 threads.
Compressing objects: 100% (3/3), done.
Writing objects: 100% (3/3), 280 bytes, done.
Total 3 (delta 2), reused 0 (delta 0)

-----> Heroku receiving push
-----> Fetching custom buildpack... done
-----> Ruby/Rails app detected

=====> CUSTOM BUILDPACK

-----> Discovering process types
       Procfile declares types      -> (none)
       Default types for Ruby/Rails -> console, rake, web, worker
-----> Compiled slug size is 18.8MB
-----> Launching... done, v48
       http://myapp.herokuapp.com deployed to Heroku

To git@heroku.com:myapp.git
   d2b28d4..d0fc25e  master -> master

Conclusion

Taking my baseline measurements from earlier, we have “control” of about 23 Seconds of heroku deploy time out of the average of 89 Seconds measured. Therefore a saving of 13 Seconds is then pretty good.

That’s then putting an estimate that Heroku’s baseline deployment time is around 66 seconds.

So we may be able to cut our asset compilation time in half with that custom asset task buildpack. However with a baseline of 66 seconds spent “in Heroku” we can’t really get much faster than that without some help from Heroku, c’mon lads, lets get back to sub-minute deploys! As this is just the basics of an app, anything more complicated and you could easily be talking several minutes for a deploy.

Am going to implement some better statistics gathering of all our continuous deployments at Rumble. Would be very interested in hearing what other people’s average deploy times are.

Rails 4

As an aside. I recently started some work on a pull request to make rake assets:precompile faster by default in Rails 4 by making digested assets (rake assets:precompile:primary) the default and killing off the public directory entirely, moving all of those static files to app/assets.

«