I do not have experience with Ruby, but I think it's similar to Python (in performance) and even if it's let's say 5 times slower (which I don't think it is), a webservice on a simple machine (workstation) in Ruby can easily serve north to 1k (or more) requests per seconds (behind nginx or apache or whatever), and in no way would 20 users be a problem.

It doesn't matter if that's async or not (note, not being async does not rule out concurrency or parallelism), but there was something seriously wrong with your application (if you have to serve long-running tasks, you have to make a msg queue anyway and shouldn't do that in your HTTP response directly). You may gain from async if you are IO bound, but with 20 users you are not IO bound, believe me.

You're absolutely right when you say there was something seriously wrong with the setup! The problem is that it was more or less what you'd get if you set up a default Rails app behind Passenger, and Passenger did this because some of the major Ruby gems weren't threadsafe (e.g. Rails, until 2008), which comes back to concurrency problems with the ecosystem.

The 20/20 hindsight is not meant to retroactively solve their problem, it's mean as a counter-argument, that the problem was their design, not Ruby.

It sounds like the concurrency was set much too low. That's not limited to the configured max workers but there will be similar settings for the web server, database connection layer and the database itself.

A similar thing would have happened with C++ or Java if there was a connection pool of two and database queries that took 500 ms to complete. I've seen worse mistakes in producion code.

The lesson to learn from this is probably not that goroutines makes everything faster, but that you have to understand the tools you use. Also, test demos. And always keep a backup plan. That's what Steve Jobs did, relentlessly, for all his demos.

You can get around it by using async instead of blocking, using fibers which don't block, or using a thread pool so other threads can keep serving.

I'm not experienced with Ruby but it sounds like Ruby doesn't do any of those

Modern ruby can do any of those, though in 1.8 or earlier it could do fewer, and less well.

The typical approach for Ruby, Python, and similar languages is just to run more processes.

Yes, but this article is about speed, not concurrency, because speed is a thing 3.0 didn't do a major improvement to from the last 2.x release.

There's no need for a “Why didn't Ruby 3 improve concurrency/parallelism” article, because it did.

Fiber#scheduler addresses your IO issue, and Ractors (while still experimental) are aimed at the parallelism issue.

JS has an equally bad concurrency story as Ruby. Maybe even worse. There are no threads, you can only use a single core per instance. In practice the problem is ram. JS already uses a lot of ram compared to Java and running an instance for each core makes it way worse.IMO only crazy people run JS backends when there's so many other similar options

This is absolutely false. Yes, JS is single threaded, but all IO is async and happens via a worker pool. On top of that it is 3-10x faster than Ruby for most tasks due to the insane amount of development effort that went into the browser runtimes, it’s often within 50% of C speeds.

And then the entire ecosystem is async. This is why you’ll see JS frameworks near the top of TechEmpower benchmarks achieving massive concurrency numbers, and also why nobody recommends any kind of CPU-heavy workload in node.

Imagine Ruby came with Puma built-in, and all libraries only exported async interfaces.

Node has both blocking and async versions of a lot of IO operations. But async IO is more commonly used certainly. The real question might be whether a thread pool backed async IO implementation makes your program do concurrent IO better than a GIL + blocking IO calls that release the GIL, it's not obvious to me.

(Doesn't Node have threads too nowadays?)

Node’s threads are for cases you really want to do heavy computation without resorting to another language. All the I/O is still backed by LibUV and an event loop, threads don’t really offer any improvement there.

JS is strange because of all the optimization. It's technically multi threaded with IO but in a way that you only control the "main thread". Reminds me of old UI work

Well, it also has Workers, which is.

Basically almost every high performing server atm is running on some form of event loop / cooperative multi-tasking. Ruby Puma, Gunicorn, rust’s tokio and async/await, goroutines... Java is probably the big exception here (if you’re not using VertX et al) and we all know the pain of thread safety.

As far as threading, Java is adding userspace M:N threads which will be far superior in most use cases to C#'s async. And unlike async it gives an automatic performance boost to most existing code

C# has a long history of breaking changes. One of the reasons Java is more popular is 90% of decades old code runs fine on the newest version. Java's backwards compatibility is absolutely unmatched.

I'm fairly sure GIL is released when waiting for external I/O, at least all the decent C database libraries do so. So in your case threads would have help quite a bit.

In general yes, the GIL is imho a problem and I do not like it, but at the same time there are ways to configure applications so that they do not mind that much.

For non-thread safe things (are there still any?) there is always the goold old way of just running multiple processes (with preloading the memory usage of even 10, 15 puma workers is often acceptable).

I really, really dislike Rails for any number of reasons, but not having spent the time to understand how to set it up is not Rails fault.

Processors have only been going sideways for a while and python makes this a pain.

Elixir and Rust are a little less painful in that regard.

The team could have booted up more processes. Isn't that the answer?

Or have I misinterpreted your comment?

Not 200,000. A single process wasn't able to handle 20.

You'll be hard pressed intentionally writing code that performs that slowly in a framework in another language.

Ruby by default has an incredibly long list of "you can but you shouldn't" and things you need to be aware of, especially around concurrency.

Other languages are mostly fast by default. When you follow some nodejs/express/whatever introduction tutorial and implement your first application in the most naive way possible, you'll end up with something that will easily scale to hundreds if not thousands of users out of the box.

Do the same with ruby (or even with some python frameworks, to be fair), and stuff can break at less than a hundred users.

And that's why yes, you can blame that on ruby. Or at least ruby how it is commonly taught.

The example said there was 0.5 seconds of IO per request so each process could serve 2 requests per second. 2 processes would serve 4 requests per second. The mistake was running only one process per core. 10 processes would server 40 requests per second, and the cores would still be mostly idle. With rails you always run as many processes as memory allows, not one per core.

Now that rails is thread safe you can have each process run a few threads. If you ran 10 processes and 4 threads per process you are up to 120 requests per second.

If you really need massive concurrency that isn't anything special. But that is more than enough for 90% of the web apps out there.

Serving only 20 users only has to do with how many workers were assigned.

I actually implemented a high traffic RoR payment provider on minimal infra ( 1 small VPS ), and benchmarked a couple of Python CMSes and compared them to WP.

I'd like to see your results if you still have them.

I have never ever been concerned about concurrency in Ruby, or Python or PHP. Just deployed the number of workers which would be able to serve our peaks.

I do write high concurrency application though, but those are not web apps and are not written in these languages.

That aside, a ruby script that can't handle 20 req/s on 2 vcpus seems rather suspect. I'd be really curious what that code looked like. I'm doubtful that it has much to do with the language and not the code design.

- overprovision resources for big demos by a significant amount

- test your demo before you give it

- deploying pets (servers you ssh into and lovingly configure) instead of cattle (instance abc123 that's created automatically) means you can't scale them as easily

- if you have concerns about a big demo, particularly ones that could be fixed by spending $10 on EC2 instances, you should fix them yourself instead of emailing the tech lead and assuming he'll handle it

- spinning up processes with phusion passenger is awful[0] and we should have been using something threaded like puma

That being said, the same footguns apply at various points as you scale, because Ruby's GVL still exists, and because even though threads are lighter weight than processes, they're still relatively expensive to switch between and to maintain in memory compared to e.g. Go/Elixir lightweight threads or evented IO.

Unfortunately (or perhaps fortunately) the problems with threaded IO start applying well above 20 concurrent users, so I don't have any fun stories about failed demos involving Puma instead of Passenger.

[0] Specifically, Rails processes take up a lot of memory, none of it is shared, context switching between processes is pretty expensive, and so you're limited to a smaller number of processes that can become IO-bound very easily

This is a really common misconception. Both switching between threads and processes is _extremely_ fast compared to what web applications do.

An exceptionally fast web response is generally <1ms. Linux thread context switching takes ~2 microseconds. Forking a new processes takes ~20 microseconds.

It's also not a fundamental restriction on threads vs Goroutines either. switchto is very nearly as fast a Goroutine context switching https://www.phoronix.com/scan.php?page=news_item&px=Google-U...

Not true. Context switching pthreads is orders of magnitude faster than any Go/Elixir concurrency mechanism.

So-called "lightweight" threads exist to fix the concurrency problem with garbage collection in scripting languages. Without garbage collection pthreads are fast enough for any problem.

That's the original advantage over OS threads.

Another optimization is using jemalloc. Going to try this soon with Sidekiq workers.

Everyone dropped the concurrency ball hoping that was someone else's problem. Its the dev team's fault because everything is ultimately the dev team's fault. But it would also be great if it was fixed in ruby + ruby's ecosystem.

Cause running more than 2 processes on a 2-core server just sounds weird.

I thought Node's doctrine was "One process per core", and (async) Rust and Go seem to do fine with managing their own thread pools. I haven't _tried_ to do a realistic benchmark of Hyper, but I've never seen it struggle with concurrency either, even on long-polling applications where concurrency was the norm for all clients.

More processes would work, but it doesn't seem like the right layer of abstraction for 2015, now that Nginx and Node have made the event model so popular.

The example given was a Rails app. I don't think one process per core is common for Rails deployments but I'm not sure.

It's hard to remember what we had to do back then, but Passenger was probably there. My first though: did you use a connection pool and a C driver to make parallel requests to the database?

It's almost always worth doing some concurrency/scaling sanity checks on anything you're preparing to demo IME. There's plenty of ways any particular app, no matter what language or framework, could unexpectedly fall down when facing 20x traffic. If not from the language itself, perhaps from various external dependencies. It may be true, particularly at some previous times, that doing the most naive thing in Rails hosting was significantly worse than doing the most naive thing in Java hosting, but still, always worth checking.

The 50ms DB round-trip 10x per request smells funny. Most queries to a closely located DB with demo app data should be more like 5ms or faster. Maybe there's good reason for it to take so long, I'm not sure. Perhaps the DB also had some slowdown from so many of those queries being executed concurrently, or maybe it would have if you had had a faster Rails setup or used Java instead. All the more reason to always test it and not just assume it'll be fine because Java should be fast.

Each individual web request is still gonna be rather slow if it's gotta do multiple 50ms DB round trips. I don't know Java that well, but C# and Ruby both have pretty decent tools for making that IO concurrent in the scope of an individual request. You have to deliberately use it though, since only you know which queries actually depend on data from other queries.

I've been working professionally with Rails for several years. If I was deploying such an app now, it'd be simple and fast to say, I suppose I'll serve it with Puma. 2 cores, so 2 processes. 5-10 threads per process is pretty normal, maybe make it 20 here since I expect a lot of time waiting on DB IO. Now we've got 40 workers and it ought to serve traffic from 20 users fine. Assuming some other part doesn't fall down first.

Ruby traded safety (GIL) for absolutely speed, although it should be noted that Ruby's C interfaces do allow you to op out of the GIL, which means if speed is absolutely paramount you can write a Ruby C extension to opt out of it.

I've done it in the past, but OTOH if you're to that point maybe Ruby no longer makes sense.

“Perl 6” hasn’t been a thing for more than a year.

As for Ruby 3, the concurrency improvements are here today. The Actor-model parallelism of Ractors is here, if experimental, today. JIT improvements are here today.

Yes, there's also more stuff on the horizon building on that, but it is simply not the case that efficiencies from Ruby 3 are so e kind of continuously receding mirage.

Anyway, I like Ruby more than any of those languages, so hopefully speed in 3.0 is at least on par with Python and Perl now.