This is the first post of my pre-RailsConf series of blog posts on Rails application performance optimization. I'll be presenting at RailsConf about this and many other aspects of performance optimization on May 5th, so you're welcome to join me at RailsConf and please watch for more performance-related articles in this blog.
Imagine a situation when your Ruby code is slow. Not so hard to imagine, isn't it? Now imagine that your ran a profiler (like we did) and figured out that you can do nothing to optimize.
Hard to imagine, you say? Not when you heavily use Ruby Date class. So what to do when there's nothing to optimize in Ruby? Simple answer - use C! Here is the story how we optimized our application by using Date class partially rewritten in C.
Why Date Class Is Slow in Ruby
Ruby Date implementation is an amazing piece of code handling probably any possible date usecase. But that flexibility comes not without a cost - the performance of Date class is very poor. It takes a long time to create Date objects, print them and compare them.
This simple benchmark should give you an idea of how slow Date is:
> puts Benchmark.realtime { 1000.times { Date.civil(2008, 7, 15) } }
0.080
> puts Benchmark.realtime { 1000.times { Time.mktime(2008, 7, 15, 0, 0, 0) } }
0.005
Wtf you say? Contructing Date's is 16x slower than Time's? Yes it is. To understand the reason we need to dig into Date class internals.
When you create a Date from "civil" Gregorian year, month and day, the constructor converts those into the Astronomical Julian Day number which is a number of days since midday January 1st, 4713BC UTC:
> d = Date.today
> d.to_s
"2008-07-15"
> d.ajd
Rational(4909325, 2) #this is a fractional astronomical Julian day number
> d.ajd.to_f
2454662.5 #2454662 days and a half passed since the initial epoch
As you can see, any civil date is assumed to be the "start" of a day hence it always have half of the day subtracted from it. Note also that Ruby implementation uses Rational class to store the fraction without losing precision.
It's extremely easy and flexible to work on dates as on numbers but to obtain those numbers you have to do conversion which is roughly 20 arithmetic operations and 3 round-downs. This conversion takes place every time you construct or print the date. In addition to that, the constructor has to create an instance of Rational class. Just because Ruby interpreter is genuinely slow, all those operations take time... too much time and you can do absolutely nothing to optimize them!
The Impact
Now we know that Date is slow, but will it hurt the real world application? The answer really depends on how much you use dates. If you store large amounts of dates in the database, be prepared for the worst.
As usual my example will be Acunote, our online project management application written in Rails. One of the things Acunote does is it keeps track of people's progress. To do that it records all changes in estimated and remaining times for each task in the system thus storing a lot of <date, number> tuples.
Using the historical data, Acunote draws burndown graphs where horizontal axis represents dates and vertical - total work remainings for each date. Obviously, to draw such graphs we need to fetch data from database, perform simple additions and output it. Sounds simple but works slow.
Here's what profiler says for the burndown drawing request:
Total: 2.490000
%self total self wait child calls name
7.23 0.66 0.18 0.00 0.48 18601 <Class::Rational>#reduce
6.83 0.27 0.17 0.00 0.10 5782 <Class::Date>#jd_to_civil
6.43 0.21 0.16 0.00 0.05 31528 Rational#initialize
5.62 0.23 0.14 0.00 0.09 18601 Integer#gcd
4.82 0.12 0.12 0.00 0.00 1 Magick::Image#to_blob
...
The top 4 method calls have nothing to do with burndown drawing. As it turns out they are called from Date#civil and Date#<=> methods:
Total: 2.490000
total
1.56 <Class::Date>#civil
0.11 Date#<=>
...
Great! We spend 65% of the request time just constructing and comparing dates. How to optimize? We had no idea because it's quite hard to optimize 20 additions and multiplications that take place inside Date class without rewriting the whole thing. The only workable solution was to rewrite slow parts in Date in C and we had been dreaming about that until we found that Ryan Tomayko already did exactly that.
Date::Performance To The Rescue
Date::Performance library implements several core methods of Date class in C replacing the slow Ruby ones. What's important, Date constructor and conversion methods were rewritten and are now 20 times faster!
> puts Benchmark.realtime { 1000.times { Date.civil(2008, 7, 15) } }
0.004
> puts Benchmark.realtime { 1000.times { Time.mktime(2008, 7, 15, 0, 0, 0) } }
0.005
In Acunote the burndown operation immediately became 2.5x faster after we used Date::Performance and all Date/Rational related records were absolutely absent from the profiler output!
Total: 0.950000
%self total self wait child calls name
12.63 0.12 0.12 0.00 0.00 1 Magick::Image#to_blob
8.42 0.17 0.08 0.00 0.09 602 Array#each_index
5.26 0.07 0.05 0.00 0.02 6992 Class#new!
5.26 0.09 0.05 0.00 0.04 42683 Kernel#===
5.26 0.05 0.05 0.00 0.00 2 Magick::GradientFill#fill
Date::Performance is designed as an addition to the original Date class. You just install date-performance from source from the GitHub project page and require 'date/performance' in your code.
Date::Performance versions 0.4.6 and earlier didn't overwrite Date::<=> method (which was also reported by the profiler as a slowdown) so I went on and implemented it in C as well. The implementation is in Date::Performance 0.4.7 or on our company GitHub date-performance fork. This gives additional 10% improvement.
There's one important thing about Date::Performance - make sure to run tests for your code and check that it works. Date::Performance is not a drop-in replacement. Known regressions are:
- month argument to the constructor can not be negative;
- day argument to the constructor can not be negative in date-performance versions < 0.4.6 (0.4.6 has my patch applied already);
- Date cannot be compared with Float in 0.4.7 and later
There might be other regressions I don't know about yet.
Conclusions
By using Date class partially reimplemented in C you can speedup your application 2.5x! To do that:
- install Date::Performance library from GitHub;
- require 'date/performance' in your code;
- run the tests and if everything works - enjoy!
PS: many thanks to Ryan Tomayko for his rocket-fast Date implementation.
I tried getting this to work with config.gem but with no luck. Any advice?
Posted by: someone | April 23, 2009 at 03:47 PM
There's also a patch for 1.8.x to make gcd much faster:
http://rubyforge.org/tracker/index.php?func=detail&aid=19034&group_id=426&atid=1700
I think that particular patch isn't necessary for 1.9, though.
==r
Posted by: Roger Pack | May 06, 2009 at 10:43 AM