I think you have missed where he said it had 20 people using it.

Not 200,000. A single process wasn't able to handle 20.

You'll be hard pressed intentionally writing code that performs that slowly in a framework in another language.

Ruby by default has an incredibly long list of "you can but you shouldn't" and things you need to be aware of, especially around concurrency.

Other languages are mostly fast by default. When you follow some nodejs/express/whatever introduction tutorial and implement your first application in the most naive way possible, you'll end up with something that will easily scale to hundreds if not thousands of users out of the box.

Do the same with ruby (or even with some python frameworks, to be fair), and stuff can break at less than a hundred users.

And that's why yes, you can blame that on ruby. Or at least ruby how it is commonly taught.

There's nothing about the code being slow in the example. The example said the processes were io bound. The speed of ruby code is irrelevant. If it was io bound demo system had 2 processes sitting 99% idle.

The example said there was 0.5 seconds of IO per request so each process could serve 2 requests per second. 2 processes would serve 4 requests per second. The mistake was running only one process per core. 10 processes would server 40 requests per second, and the cores would still be mostly idle. With rails you always run as many processes as memory allows, not one per core.

Now that rails is thread safe you can have each process run a few threads. If you ran 10 processes and 4 threads per process you are up to 120 requests per second.

If you really need massive concurrency that isn't anything special. But that is more than enough for 90% of the web apps out there.

This is nonsense, there is boatloads of frameworks and CMSes which can easily run into the 10s or 100s of milliseconds for the main request ( WP for example ).

Serving only 20 users only has to do with how many workers were assigned.

I actually implemented a high traffic RoR payment provider on minimal infra ( 1 small VPS ), and benchmarked a couple of Python CMSes and compared them to WP.

> I actually implemented a high traffic RoR payment provider on minimal infra ( 1 small VPS ), and benchmarked a couple of Python CMSes and compared them to WP.

I'd like to see your results if you still have them.

There is a lot we don't know with the example given. It's unfair to blame Ruby.

But it's not unique to this example. It's common; if you don't start out concerned about concurrency in Ruby, and just do the naive thing, you are going to fall over at -extremely- low numbers compared with most other modern languages.

If it is so common as you claim, could you point us to some public examples of this?

I have never ever been concerned about concurrency in Ruby, or Python or PHP. Just deployed the number of workers which would be able to serve our peaks.

I do write high concurrency application though, but those are not web apps and are not written in these languages.

I've used Ruby and rails for close to a decade and have never seen/experienced something like what they're describing, so I doubt this is really that common.

The "naive" thing in Ruby, if you're serving HTTP requests, is to use some kind of app gateway that allows concurrency, such as puma or unicorn. It's just not all that viable to do something more basic than that unless you're willing to write all the request handling stuff by hand.

That aside, a ruby script that can't handle 20 req/s on 2 vcpus seems rather suspect. I'd be really curious what that code looked like. I'm doubtful that it has much to do with the language and not the code design.

You're absolutely right! The biggest things I learnt from that were:

- overprovision resources for big demos by a significant amount

- test your demo before you give it

- deploying pets (servers you ssh into and lovingly configure) instead of cattle (instance abc123 that's created automatically) means you can't scale them as easily

- if you have concerns about a big demo, particularly ones that could be fixed by spending $10 on EC2 instances, you should fix them yourself instead of emailing the tech lead and assuming he'll handle it

- spinning up processes with phusion passenger is awful[0] and we should have been using something threaded like puma

That being said, the same footguns apply at various points as you scale, because Ruby's GVL still exists, and because even though threads are lighter weight than processes, they're still relatively expensive to switch between and to maintain in memory compared to e.g. Go/Elixir lightweight threads or evented IO.

Unfortunately (or perhaps fortunately) the problems with threaded IO start applying well above 20 concurrent users, so I don't have any fun stories about failed demos involving Puma instead of Passenger.

[0] Specifically, Rails processes take up a lot of memory, none of it is shared, context switching between processes is pretty expensive, and so you're limited to a smaller number of processes that can become IO-bound very easily

> Threads are lighter weight than processes they're still relatively expensive to switch between and to maintain in memory ... context switching between processes is pretty expensive

This is a really common misconception. Both switching between threads and processes is _extremely_ fast compared to what web applications do.

An exceptionally fast web response is generally <1ms. Linux thread context switching takes ~2 microseconds. Forking a new processes takes ~20 microseconds.

Relatively expensive was probably not phrased well, but a goroutine context switch is ~200 nanoseconds and instantiating a new one takes ~1 microsecond, from memory and a quick google.

Doing nothing 10x as fast is not all that useful. In a real app it's unlikely the difference in context switching being 0.2% of your CPU time or 0.02% of your CPU time matters.

It's also not a fundamental restriction on threads vs Goroutines either. switchto is very nearly as fast a Goroutine context switching https://www.phoronix.com/scan.php?page=news_item&px=Google-U...

> threads are lighter weight than processes, they're still relatively expensive to switch between and to maintain in memory compared to e.g. Go/Elixir lightweight threads or evented IO

Not true. Context switching pthreads is orders of magnitude faster than any Go/Elixir concurrency mechanism.

So-called "lightweight" threads exist to fix the concurrency problem with garbage collection in scripting languages. Without garbage collection pthreads are fast enough for any problem.

Lightweight threads in Erlang/Elixir were created as a unit of fault tolerance. A lightweight thread can suddenly exit when it hits an error, and the VM mechanisms give you guarantees that any resource it owned will be freed (memory, sockets, registered name, ETS tables, etc.) Using signals and monitors you can extend this to any other type of resources, for instance DB connections in an SQL adapter library.

That's the original advantage over OS threads.

No, you don't need cooperative threads for this. Proper exception handling fixes this problem with OS threads for you.

Passenger can be configured to restart processes after they reach a memory threshold.

Another optimization is using jemalloc. Going to try this soon with Sidekiq workers.

This is the right answer. The demo failed because the dev team failed to understand the performance and scaling characteristics of the code they wrote. This was a failure of people, not technology.

Eh - it was a failure of the (people, technology) tuple. Different choices by the engineers on the team could have fixed that problem. And different technology choices by the engineers behind ruby (/rails) could also have fixed the problem. (Eg async io, much larger thread pools, etc).

Everyone dropped the concurrency ball hoping that was someone else's problem. Its the dev team's fault because everything is ultimately the dev team's fault. But it would also be great if it was fixed in ruby + ruby's ecosystem.

If something is a known-known, you have to take personal responsibility when you don't take that thing into account and get bit. It's not appropriate to even mention some person you have not met as part of the reason a demo failed. Personal accountability it key in this industry, do your due diligence and events like were described will not happen.

Easier to blame the seemingly nameless technology and completely change the conversation by using some other stack entirely that accounted for your lack of due diligence.

Yes but the point is that lots of people became disenchanted with ruby because it was notably harder to get this sort of thing right than it was for the alternatives with more built in support for in-process concurrency. Maybe it's not the case anymore, but back around 2010, the only people who knew what they needed to do to configure a rails app to scale decently were those who had battle scars of the kind in the parent comment. And nobody came out of those battles feeling all sunshine-and-roses about the platform.

Ruby had support for in-process concurrency via threads for the last 14 years.

And yet, see the comment that started this thread. It is simply true that concurrency was a pain point for rails for a lot of people for a long time. (I don't know if it still is, I've been out of that game for five years.)

By resources do you mean hardware resources or expertise or what?

Cause running more than 2 processes on a 2-core server just sounds weird.

I thought Node's doctrine was "One process per core", and (async) Rust and Go seem to do fine with managing their own thread pools. I haven't _tried_ to do a realistic benchmark of Hyper, but I've never seen it struggle with concurrency either, even on long-polling applications where concurrency was the norm for all clients.

More processes would work, but it doesn't seem like the right layer of abstraction for 2015, now that Nginx and Node have made the event model so popular.

More hardware resources.

The example given was a Rails app. I don't think one process per core is common for Rails deployments but I'm not sure.

Process per core with threading for parallel IO has been the default for almost a decade now.

Default in what app servers?

Puma is part of the default Rails Gems and was first released all the way back in 2011.