News for hackers. Stuff that matters.
Re:Ruby Garbage Collection: Still Not Ready for Production by otterley on Friday March 28, 2014 @03:52PM (#7489569)
I think a little extra memory in exchange for a whole lot of safety and code simplicity is a fair tradeoff. If you don't have enough memory to spawn a few hundred subprocesses I'd suggest you tackle that first.
Re:Ruby Garbage Collection: Still Not Ready for Production by ikawe on Saturday March 29, 2014 @12:56AM (#7491477)
The first step of scaling an app is not to make sure you have a server with at least 43GB of RAM.

If we say a typical Rails process is 150MB, then consider several hundred of them...

150MB * 300 = 43GB

So, while your point is valid, in that there is value in safety and code simplicity, we're not talking just "a little" memory. Consider your use case, that's all.

Re:Ruby Garbage Collection: Still Not Ready for Production by otterley on Saturday March 29, 2014 @03:21PM (#7493555)
Your point is about economics, not scaling per se. A program that requires lots of memory may be wasteful, but it could still very well scale (in terms of its ability to service requests under increasing load).

Nevertheless, the memory requirements of a multiprocess server are not a linear function of the footprint of the master process. fork(2) on modern Unices (including Linux, since, like forever) has copy-on-write semantics for a child's memory usage. So when you fork a new process, the memory overhead is relatively small (only the page tables are copied, not the heap, and never the program text or shared libraries).

Only when a child process modifies or allocates a new heap page or exec()s another program will it incur additional memory overhead. So the additional memory required by a child Unicorn process (assuming the parent preloads the application server code) won't usually be anywhere near 150MB (and if there's a memory leak it's trivial to free it up by killing the child after it services a request).

Re:Ruby Garbage Collection: Still Not Ready for Production by steveklabnik on Friday March 28, 2014 @06:09PM (#7490263)
Thank you. "Threads are always better than processes, lololol" really gets under my skin. There are pros and cons to each approach.
Re:Ruby Garbage Collection: Still Not Ready for Production by yummyfajitas on Saturday March 29, 2014 @02:27AM (#7491607)
What simplicity/safety does using processes buy you relative to a good threaded runtime (read: JVM/Haskell/Go)?
Re:Ruby Garbage Collection: Still Not Ready for Production by otterley on Saturday March 29, 2014 @03:27PM (#7493574)
Building a good threaded runtime is difficult in the first instance, and even then, shared memory requires special handling (mutexes).

There has been plenty of ink spilled on the subject, but here's a good read: http://msdn.microsoft.com/en-us/magazine/cc163744.aspx

Re:Ruby Garbage Collection: Still Not Ready for Production by yummyfajitas on Saturday March 29, 2014 @10:21PM (#7494876)
Building a JVM is harder than building an RVM. I was asking from the perspective of user code. Shared memory requires mutexes in both threaded and forked code - this is not an advantage of multiple processes. In fact, multiprocess shared memory is generally harder to work with than threaded shared memory.
Re:Ruby Garbage Collection: Still Not Ready for Production by otterley on Monday March 31, 2014 @02:49PM (#7503364)
We're talking about Ruby here, though. MRI threading has concurrency limitations, so processes are better here IMO.

I tend to think out of the box for multiprocess shared memory - Kyoto Cabinet and Redis can be helpful here, to name a few. Even SysV shared memory is pretty easy to use.