News for hackers. Stuff that matters.
Re:Adding Concurrency to Rails Apps with Unicorn by btilly on Wednesday February 27, 2013 @12:46AM (#5290915)
And, since Unicorn forks as worker processes, so, 2 unicorn workers will use up double the memory of a typical Rails app.

Thanks to copy on write, the amount of memory needed for 2 processes is strictly less than 2. Just make sure to preload as much data as you can before you fork. And in many use cases memory slowly comes unstuck (change anything, and a whole page of data comes unshared) so it can make sense to kill workers every so often and let them refork with everything properly shared.

Re:Adding Concurrency to Rails Apps with Unicorn by markov_twain on Wednesday February 27, 2013 @01:29AM (#5291001)
Unfortunately, Ruby 1.9.x is not copy-on-write friendly, mainly due to the garbage collection strategy. I would venture to guess that most Rails apps these days are running on some version of Ruby in the 1.9 series. So siong1987 is generally correct in the assumption that forking a Rails process n times will consume about n times the amount of memory as a single process. However, Ruby 2.0 has a new garbage collector (called bitmap-marking) that promises to be copy-on-write friendly, so as adoption of that increases your suggestions will become more important.
Re:Adding Concurrency to Rails Apps with Unicorn by zwily on Wednesday February 27, 2013 @02:02AM (#5291075)
Brightbox has Ruby 1.9 packages with the GC copy-on-write-friendly patches included. We run it in production with good results.
Re:Adding Concurrency to Rails Apps with Unicorn by markov_twain on Wednesday February 27, 2013 @02:23AM (#5291125)
AFAIK, to use the brightbox packages on heroku you'd have to write a buildpack that either downloads all the dependencies and runs through the same build process that brightbox uses, or use something like heroku-buildpack-fakesu to create a fakeroot type of environment where you can install debs.

Another issue with using the brightbox packages is that if you happen to run into a bug, you'll have to figure out if the bug was caused by something non-standard in your ruby installation or if it's actually a bug in ruby.

One last thing to note is that it looks like the latest brightbox release targets patchlevel 327, while ruby core is at 392 (not counting 2.0.0-p0), so you're missing a lot bug fixes until the brightbox team gets around to building against the latest release.

Re:Adding Concurrency to Rails Apps with Unicorn by zwily on Wednesday February 27, 2013 @09:42AM (#5292551)
We don't deploy on Heroku, so I didn't know about that process.

Your other two points are true though. We consider the trade-off worth it for the memory gains.

Re:Adding Concurrency to Rails Apps with Unicorn by pixelcort on Wednesday February 27, 2013 @01:28AM (#5290999)
Will Unicorn automatically kill older forked processes periodically and fork off new ones, or is that something that would need to be done separately?
Re:Adding Concurrency to Rails Apps with Unicorn by swampthing on Wednesday February 27, 2013 @02:23AM (#5291120)
One of the comments on the Heroku article mentions this gem, which does just that:

https://github.com/kzk/unicorn-worker-killer

Re:Adding Concurrency to Rails Apps with Unicorn by kawsper on Wednesday February 27, 2013 @04:31AM (#5291472)
Regarding memory, I think you can do something like this in your Unicorn config:

after_fork do |server, worker| size_in_bytes = 60010241024 # megabytes Process.setrlimit(Process::RLIMIT_AS, size_in_bytes)

Re:Adding Concurrency to Rails Apps with Unicorn by lucaspiller on Wednesday February 27, 2013 @04:57AM (#5291539)
We haven't had any issues with memory leaks since we've been using Unicorn and Rails 3. Back in the old days of Mongrel and Rails 2 we had to restart the processes periodically because of this, but we haven't seen any issues with this setup. Here are the stats:

    www-data 21714  0.0  5.6 102928 87164 ?        Sl   Feb26   0:30 unicorn_rails master -c production.conf.rb -E production -D                                                        
    www-data 21762  0.2  8.5 150672 131996 ?       Sl   Feb26   1:58 unicorn_rails worker[0] -c production.conf.rb -E production -D                                                     
    www-data 21768  0.2  8.5 152036 132352 ?       Sl   Feb26   1:58 unicorn_rails worker[1] -c production.conf.rb -E production -D                                                     
    www-data 21774  0.2  8.1 145248 125668 ?       Sl   Feb26   1:56 unicorn_rails worker[2] -c production.conf.rb -E production -D                                                     
    www-data 21780  0.2  8.3 148792 129736 ?       Sl   Feb26   2:01 unicorn_rails worker[3] -c production.conf.rb -E production -D                                                     
    www-data 21786  0.1  8.3 148384 129260 ?       Sl   Feb26   1:48 unicorn_rails worker[4] -c production.conf.rb -E production -D                                                     
    www-data 21792  0.1  8.6 152156 133880 ?       Sl   Feb26   1:52 unicorn_rails worker[5] -c production.conf.rb -E production -D
Re:Adding Concurrency to Rails Apps with Unicorn by lucian1900 on Wednesday February 27, 2013 @06:22AM (#5291772)
Why would that be desirable? (I'm not a Ruby developer, so I have no idea)
Re:Adding Concurrency to Rails Apps with Unicorn by kawsper on Wednesday February 27, 2013 @04:26AM (#5291461)
I think Unicorn kills off old workers and spawns new regularly.