Don’t get me wrong, I'm not suggesting that everyone should be building their site like it’s Facebook or Google, but when every public website is already sat on a proverbial motorway, it can be dangerously shortsighted to build your site to only meet demand with no room to scale. CGI falls under that category. There are plenty of platforms that are still very easy to build and deploy, developer friendly, and still runs circles around the performance of CGI.
(Please note that I am talking about public sites specifically and not intranets or other IP white listed resources)
If you have an app with an admin interface/CMS/monitoring interface and you know there won't be 50 administrators for your app before the end of the year this works great (assuming you can reverse proxy this to add auth or something).
Hence why I specifically said "public website". Intranets and other IP whitelisted resources are clearly a different topic entirely. Sites only intended for 50 administrators but are not hidden behind a firewall make me nervous for a whole other set of reasons but I accept that's got to happen sometimes (but even there, most of the best / developer friendly CRUD platforms these days aren't CGI so there is little reason to use CGI even for that specific use case).
Performance arguments aside and given the type of site you describe, wouldn’t it be more convenient for the developer to use Wordpress or one of those website builder as a service things instead of inventing something from scratch in CGI?
I get the argument that some personal sites wouldn’t get much traffic but the argument that CGI is easier to build than any non-CGI alternative simply isn’t true any more and hasn’t been the case for more than a decade.
I know I’m coming across as passionately against CGI and I assure you that isn’t the case (I recently chose to use CGI as a private API endpoint for some Alexia skills I’d written for myself). But for a public site there isn’t really a strong argument in favour of CGI anymore given the wealth of options we have available.
The irony of decrying poor performance yet suggesting WordPress is pretty priceless :)
I agree that most people would be better served with Squarespace/Wix but I run into the situation or static site + 1-2 bits of dynamic form processing frequently enough to warrant having a simple solution for it. When the site doesn’t warrant spending money on, sometimes a shared host + CGI is just right. But I think this is for some very rare cases. Most people should outsource these types of headaches.
Event the non-static stuff will be hit by bad bots (and by "bad bots" I mean any crawler - malicious or otherwise - that doesn't obey robots).
> The irony of decrying poor performance yet suggesting WordPress is pretty priceless :)
It's not ironic at all. I've done extensive benchmarking on this in a previous job and discovered that even the out-of-the-box Wordpress experience would easily outperform the same equivalent in CGI - and that's without taking into account all the caching plugins available for Wordpress (sure you could optimize your CGI as well, but you'd have to write all that yourself where as with Wordpress it's a 2 minute install).
Even DB and page caching aside, you still have a problem with CGI forking for each web request where as with PHP you have mod_php (Apache) or php-fpm which do opcode caching and such like. This alone can make a dramatic difference once you start piling on the requests.
> When the site doesn’t warrant spending money on, sometimes a shared host + CGI is just right. But I think this is for some very rare cases. Most people should outsource these types of headaches.
Do shared hosts even allow CGI? I'd have thought that was a bit of a security faux pas for a shared host. In any case, almost all of them support PHP (and those that don't are special purpose ones for node or other frameworks) so there's no forced requirement to use CGI even on shared hosting. In fact I'd go further and argue that PHP is even easier to write than CGI so you're better off using that regardless of whether CGI is available.
Disclaimer: I pretty much hate PHP as a language and not too fond of Wordpress either. But I'm being pragmatic in this discussion and leaving out my own personal biases. Personally I wrote my blog in Go (which, as it happens, was originally written in CGI/Perl, then ported to Apache+mod_perl before being ported to Go about 9 years ago and where it's been running ever since)
If this isn't all stuff you already have set up on your dev environment then you might find hardening CGI becomes as much work as re-writing those tools in a more secure framework.
Also your coats are hugely optimistic. I’ve done benchmarks with CGI and non-CGI code in previous companies and found it wasn’t just a couple more servers, it was often 10x more. Even at $0.26 an hour, that quickly adds up over the course of a month and year. Plus the slower throughout also has a knock on affect in your database as well. You’ll find as you’re running fewer connections on each web server you’d end up with smaller connection pools per node but more overall DB connections across the farm with those connections held open longer per web request. That means you then need to beef up your RDBMS instance and that gets very costly very quickly (even on the cloud)! And we’ve not even touched on the options of serverless et al that aren’t even available for CGI which would further bring down the cost of hosting a non-CGI site.
Let’s also not forget one of the key metrics when building a commercial web platform: performance form a UX perspective. Amazon, Google, etc have all done studies on page load times and user patterns. Their somewhat predictable result was that sites with slower response times will see more users leave that site in favour of a competitors one than sites with faster response times. So if your website is your business, running CGI could cost you in lost revenue as well as running costs.
None of what I say above is theoretical costs - this is actual data from my experiences migrating CGI platforms to non-CGI alternatives. I’ve done the benchmarking, cost analysis and so on and so forth. The arguments in favour of CGI are simply untrue for the modern era of web development.
fork() only takes around 8ms on my Linux machine and I can get 100,000 posix_spawn() per second there with 100MB RSS.
That's "fast enough" for a large number of applications.
†: fork() is a lot slower (over 20x) on Windows
On the other hand a Node.js or Go program can have a preestablished pool of keep alive connections to the backends already ready to go and reuse that connection pool for many hundreds or even thousands of concurrent websocket connections. You can approximate something like this with the fork() model by having a local daemon process that manages the connection pool and have your forked process talk to that local helper daemon when it needs a connection, but you are still going to pay a penalty for that compared with having a fully preestablished connection ready to use right there in the process already
fasthttp(go) can do around 80k/sec on my laptop; dash(C) can do almost 88k/sec on my laptop†. As soon as dash does IPC, it drops to around 51k/sec, and as soon as it needs a reply, we're down to 25k/sec††. I see no reason to believe fasthttp would do any faster.
That means I'm spending around 70% of my time in IPC -- something posix_spawn() would let me avoid (if my application were designed to do so). My same laptop will do 100k/sec posix_spawn() so I'd find this difficult to believe (1-4 msec per call) fork() or exec() is the bottleneck for any application with this architecture. Do you think posix_spawn() represents 70% of your costs? If our goal is to beat 51k/sec requests, sure, but NodeJS on my laptop (btw) gets 10k/sec, so if it's a contender, I'd say posix_spawn() is as well.
† https://github.com/geocar/dash/blob/master/README.techempowe...
What is just another way to make a NxM server, so maybe forget about it...
That means it's only viable for connections where the connection is very long lived and messages are very sparse (because context switches).
--
Meta observation:
Maybe this is the bane of smartphone era and small screens, but the context of the discussion seems to disappear instantly.
Subject: websockets > forking processes > .. aaand the websocket context is lost and we are talking generally about forks in web applications with growing thread.
Pretty-much true, but I remember a funny story from Dropbox where their websocket service couldn't come back up after a crash because their normal users trying to re-open super long lived connections all at once was well-beyond the capacity of the system
It also means you should use posix_spawn instead of fork+exec since you can control when the page faults occur better.
In an ideal scenario, you shouldn't have more active processes than you have CPU cores. As soon as that happens, context switching kicks in and performance degrades sharply.
If you can run multiple containerized apps side-by-side on the same machine at the same time on a single CPU core, then you can be sure that there is some kind of context switching happening.
Modern Operating Systems are good at minimizing the amount of context switching. If you run 4 CPU-intensive processes at the same time on a machine which has 4 CPU cores, then the OS will typically assign each process to a different CPU core (with minimal context switching). Then if you launch a 5th CPU-intensive process, then the OS will have no choice but to start doing context switching since it doesn't have any idle cores left.
On Linux, based on tests I did a couple of years ago with a multi-process WebSocket server, I can confirm that the penalty of context switching is proportional to the CPU usage of each process. So for example, if you have a very CPU-intensive process sharing a CPU core with a non-intensive process, then the penalty will be small, but if you have two intensive processes sharing the same core, the penalty will be high.
You were right about performance being relative. And of course the trade off between ease of development, use and performance. At the end of the day, practical considerations are going to determine what is "expensive".
Also depends what you're pinging. I'm in the UK, so everything in America is 30-80msec away anyway.
It certainly would hold up inordinately poor to a DDoS attack.
But CGI doesn't fork per TCP connection; it forks per HTTP request/response.
2) that's a default - most systems allow tuning
You can have pretty decent performance with forking models if you 1) have an upper bound for # of concurrent processes 2) have an input queue 3) cache results and serve from cache even for very small time windows. Not execve'ing is also a major benefit, if your system can do that (e.g. no mixing of threads with forks). In forking models, execve+runtime init is the largest overhead.
It will not beat other models, but forking processes offer other benefits such as memory protection, rlimits, namespace separation, capsicum/seccomp-bpf based sandboxing, ...
YMMV
IRC shows us that maintaining a reliable socket for most folks is next to impossible.
That they're visible to clients is an issue with how channels spans servers and how operator status and channel membership is tied to who happens to be on a channel on a specific partition at a certain time, and how messages are propagated when splits resolve, and how inter-server communications happens.
So it has plenty of lessons if you want to build a chat network, but nothing with it suggests maintaining a connection is otherwise a big problem.
In general what it boils down to is the word "reliable": You would want to write your app so that a client that disconnects and reconnects gets a sensible behavior on reconnecting, e.g. by queuing messages when it makes sense, and discard them if it does not, to paper over temporary connection failures.
But you would need to do that if you were to use stateless request/response pairs anyway.
Sockets need keep alives and resources allocated et al.
Anyhoo - same arguments about forking, process & memory limits arose ; so things like FastCGI and ISAPI were the solution to reduce/avoid some of those overheads.
I suspect Rust based CGI endpoints would have similar <quote>issues<unquote>.
But I've been thinking about this CGI comeback from time to time - the left side of backend infrastructure (load balancerss, reverse proxies etc) and middle tier machine resources are much better now. Docker containers and lambda spin up is similar to some degree in terms of topology but maybe with even more overhead?
..so does does make one wonder if cgi like simplicity make a comeback with today's order of magnitude better infrastructure? I keep waiting for someone to come up with "lambdaGI" ..with will lead to "fastLambdaGI" and so on (and that's not a snarky comment).
Here's my point of view:
1. Forking is first and foremost a system call. (To be fair, I realize even memory allocation is much of the time, but still.) The kernel is going to do a bunch of work (as fast as it can, of course) and you're going to end up with two different OS level tasks by the end.
2. Those two tasks are now scheduled in tandem by the OS scheduler. For two tasks, this is fine. For hundreds of tasks, it becomes less effective. Threads will rapidly go in and out of I/O wait and the scheduler has to balance all of this.
3. CGI dies right here, though: CGI is not just a fork. It is a fork and exec! The exec, of course, also run in the kernel. It's going to effectively load the binary from scratch. Then you probably hit the linker in usermode, which has to go through the shared library resolution, resolving and mapping shared objects into memory and filling out import tables. This stuff has some caching as far as I know, but still... it's not free.
4. Now you are in the entrypoint of your program... or are you? If you are using CGI with Perl or Python or even Bash, we're not done yet because the script interpreter has to load all of its state from scratch, parse your script, bla bla, and THEN finally we can run your program.
5. Your application now has to do all of its common setup. Every. Connection. If you need to connect to a database, you can't connection pool: you have to open a new socket every time. Redis? same thing. Need to read a config file? Yep, every dang time. You can hack around some of this but in general it's probably going to be like this for most CGI applications. There's a reason why FastCGI exists after all.
The forking model of servers is elegant... but it doesn't work so well in my opinion. The cost of OS-level context switching is non-trivial, forking is relatively expensive, and things get worse when you are talking about something like CGI where your app effectively gets loaded from scratch each time.
My favorite model is definitely the Go model, where the language schedules lightweight threads or fibers across n OS threads (where n = number of logical threads.) It is cheap for the OS scheduler, and the language scheduler can very efficiently deal with things like I/O blocking and GC without huge latency hits.
But you can go about it many ways. Node.JS's event loop model proves pretty effective. Node is far from perfect but I think I would bet on a Node.JS server with a proper event loop over a CGI server forking a C program, myself.
Of course I'm really no expert. But, I think the jump from CGI to FastCGI told me all I need to know: CGI just didn't scale well at all. FastCGI was a much better experience for me, though I no longer use it.
Surprisingly, even thousands of threads can often have better throughput than using event loops and non-blocking IO: https://www.slideshare.net/e456/tyma-paulmultithreaded1
> The cost of OS-level context switching is non-trivial
On modern hardware it's non-zero but trivial: https://eli.thegreenplace.net/2018/measuring-context-switchi...
Hundreds may be a bad example, it's going to vary based on how big your server is but there is a tipping point where threads become very infeasible. I've yet to hit that limit with Goroutines and have literally hit millions of them on relatively small allocations without much of a hitch. Event loops and non blocking IO may perform worse than threads, but I have extreme doubts that threads beat goroutines (and equivalent models like erlang processes.)
Why? Goroutines use the same syscall heavy non-blocking IO as event loops, just with coroutines for programmer convenience.
The biggest difference between Goroutines and pthreads is that Goroutines only have a 2kb default stack size and pthreads have a 2MB stack.
In high thread concurrency situations it's common to turn the pthread stack size down to 48 or 64kb to allow running tens or hundreds of thousands of OS threads. There's even a JVM config flag for it.
2, because the Go scheduler has awareness about the state of the goroutines that an OS scheduler would not, so it can make more intelligent decisions about what goroutines to wake up and when.
You can really have pretty much as many goroutines as you want. Hundreds of thousands of OS threads lands you firmly into tweaking kernel settings land. My local thread max on my desktop is just 127009 - that wouldn't fly for a huge machine running many Go apps, which is exactly the kind of circumstance I was in (using Kubernetes, to be exact.)
This is why thread per request servers like jlhttp are right up there with fasthttp etc in terms of total throughput.
https://www.techempower.com/benchmarks/#section=data-r17&hw=...
Thread pooling is an effective solution to improve webserver performance, and it generally works well. In these synthetic benchmarks, you can't even really see much of a downside. In reality a lot of these benchmarks are only so good because they have requests that complete very quickly, and I think if you add a random sleep() into them many of them will just die outright because they can't handle that much concurrency and block waiting for free workers. You might think that is unrealistic, but consider that many people have Go servers that are making RPCs all over the place, and an actual large amount of time can just be spent waiting on other RPCs. It's a real thing!
And if the world were just responding to HTTP requests, obviously something like Goroutines would be overkill. But, one of my favorite uses of Goroutines was implementing a messaging server where each consumer, queue, exchange were all their own Goroutines. I was inspired by RabbitMQ for the design but unfortunately could not use it in this use case. Luckily Goroutines worked really great here and I was able to scale this thing up hugely. To me this is where they're really great: they're super flexible. They work pretty well for short-lived HTTP requests, but also great for entirely different and more complicated use cases.
Looking back at the benchmark, one of the more interesting approaches here is go-prefork[1], which works by spawning n/2 executables with 2 threads each. I can only imagine the optimal amount of threads was complicated to determine and maybe even has something to do with hyperthreading. Of course the advantage here is the reduced amount of shared state that leads to less contention, and it does indeed show up on the benchmark. In this setup, it looks weird because there's no load balancer (could be something as simple as some iptables rules) or anything in front. In practice, this would be much akin to running separate instances on the same box, which is also a reasonable approach, and I used this approach myself when scheduling servers. Oddly, I don't think they tried the same approach for fasthttp.
I think what else the benchmark shows is how clever you don't have to be in Go to get good performance. go-postgres is routinely in the middle of the pack and it is literally just using the standard library and goroutines in the most basic fashion. It's effectively not optimized. And in reality, in many cases with more complex servers, the overhead is low enough that it isn't worth your time to optimize it much more.
[1]: https://github.com/TechEmpower/FrameworkBenchmarks/blob/mast...
Quite frankly, because Go developers don't learn from other's mistakes and reinvent square wheels.
Kernel threads will always be better and faster than usermode threads.
This is because any inefficiency in threading comes from the scheduler. Your usermode scheduler will always necessarily be slower and worse; it just makes so much sense to put your scheduling code where your context switching and process isolation code already is.
Sure, you can get a short-term gain by iterating fast and testing code if kernel developers are too slow and/or unwilling; but then eventually your code will get merged into the kernel anyways once you're done. So really usermode threads are only good as a rapid prototyping tool, not something for serious use.
Your response doesn't even start on the right foot, since Java never used green threads for performance and therefore is not a good place to hedge this argument at. Citation needed if you're going to start with that.
You can also go ahead and insult the Erlang developers for Processes since they did it first:
http://erlang.org/doc/reference_manual/processes.html
>This is because any inefficiency in threading comes from the scheduler. Your usermode scheduler will always necessarily be slower and worse; it just makes so much sense to put your scheduling code where your context switching and process isolation code already is.
You know, a Go context switch doesn't hit the kernel. It's not more expensive than a kernel context switch. I don't know why you'd think it is. Why not inline scheduling to the process where it knows what's blocked on what?
>Sure, you can get a short-term gain by iterating fast and testing code if kernel developers are too slow and/or unwilling; but then eventually your code will get merged into the kernel anyways once you're done. So really usermode threads are only good as a rapid prototyping tool, not something for serious use.
You can hold your breath for threads to become cheaper than goroutines, but careful not to suffocate. A goroutine pretty much just needs a stack, 4kb. The kernel thread has structures, well, in the kernel, for paging, for the task itself, and the stack structure is bigger (think you can at least adjust that though.)
And as for integrating the Go GC into the scheduler... I'd love to see what Linus's thoughts on merging that to kernel are!
Doesn't matter if you just have a few thousand threads but that's a limiting way to look at something. RabbitMQ can have processes all over the place for everything and have more processes than you could ever have threads and it remains among best in class for performance.
Go on the other hand is based on CSP principles and a threading model that looks like actors. The threading model is deeply ingrained in the language, and it is designed around it. Goroutines are very cheap in Go, and the scheduler is fairly effective because it knows what threads to wake when - it's not blind. Goroutines are scheduled across n threads, not just a single OS thread, so they can take good advantage of multicore or multi CPU systems. This design does hurt C interoperability a bit, but imo it's greatly worth it.
Go is not the only language that works this way. I believe its concurrency model was inspired a lot by Erlang with its 'processes' model.
Edit: If you think I am wrong, could you please explain what is wrong?
Erlang's don't, thus processes. Go's do, thus threads.
There is a terminology issue though, you're right, and that comes with using OS terms for userspace scheduling. Go coming up with their own term - goroutines - significantly simplifies the conversations. Gorountines, Erlang processes, green threads and fibers are all fundmentally the same thing - M userspace 'tasks', scheduled on to N operating system threads of execution, likely in one OS process. There are some language details as to how they are presented to the user.
At the kernel level, both "processes" and "threads" have PIDs since they're the same thing. In fact, before NTPL was implemented Linux broke posix because it returned the kernel PID directly when queried[0]. That's no more than an interface choice of POSIX though. And POSIX threads still have a thread id, a (pid, tid) is a unique visible identifier. So the distinction seems completely arbitrary.
And Erlang processes do have (erlang-level) PIDs. In fact, they get (erlang-level) PIDs across multiple machines when in a cluster.
And again I think ancillary properties are not what matter, a PID is a consequence of being a process, not a cause. The cause of being a process is not sharing internal state, that's the useful bit. Erlang's tasks are not os processes (that would rather defeat the points), but it doesn't seem useful to call them "not processes".
[0] it now returns the tgid, a "thread group" at the kernel level is what you see as a process from userland: the tgid is the pid of the original task, creating a "process" will create a new task with a new tgid while creating a "thread" will create a new task with the existing tgid https://stackoverflow.com/a/9306150/8182118 provides an excellent primer on how this works
Erlang processes are threads as seen by the OS, but the Erlang runtime implements process-y restrictions that enforce isolation and forbid shared memory between Erlang processes that do infact exist as threads inside te Erlang VM.
The programming model, as seen by the Erlang/Elixir prgrammer, is thus anologous to Unix processes, just with lower overheads.
Erlang is a dynamic language without any such type system checks. The process separation is all just based on the fact that it's impossible to get or make a shared value, it's just not a concept in the language. You send and receive messages, which implies a copy, and you faff around with local values inside your process.
Re shared-memory support in Unix: Yeah, you have escape hatches from the memory models in Unix processes, and Rust, and probably Erlang and Haskell. But they're exceptions and safe to ignore when discussing terminology to describe the platform's native model. Also the Unix shared memory APIs were a late addition to the OS and everyone agrees they're ugly :)
Not true, Erlang processes are userspace threads, not kernel threads. It's an M:N model -- you can run thousands of Erlang processes on a single OS thread. See:
http://erlang.org/euc/08/euc_smp.pdf
and note that each "scheduler" runs on a single OS thread, whereas many processes can run on each scheduler.
This and "async is always faster" are two things which are no longer true on modern hardware.
Forking a process on Linux uses the same clone() syscall as creating a thread so forking a small binary takes only tens of microseconds, leaving plenty of time for <1ms total response times.
https://eli.thegreenplace.net/2018/launching-linux-threads-a...
And even forgetting the costs of forking, threads are not necessarily much better anyways. I can easily have a single Go program scheduling literally hundreds of thousands of Goroutines across just a few OS level threads and have no problems whatsoever, and in fact I've done exactly that in production, whereas I would never even dream of doing that with threads.
Node.JS's event loop model also doesn't need to spawn threads per connections.
When I say processes and forking are relatively expensive, I don't mean compared to doing the exact same thing with threads; I mean compared to more modern alternatives, like using an event loop or Goroutines.
Event loops and userland cooperative multitasking like Goroutines predate Linux 2.6 kernel threads. They're not "more modern".
The conclusion to be drawn is "Linux threads are costly", not "Linux processes are cheap".
https://github.com/blitmap/lua-snippets/blob/master/scgi-ser...
*On the Windows subsystem, of course. There's the lower level ZwCreateProcess function which can/could be used to fork, but it's undocumented and I believe it only existed for the old SfU. Now that that's gone and Linux Subsystem uses something called Pico Processes, I'm guessing this old fork flag is pretty much unsafe to use for anything at all.
These days, folks often write the app to contain a web server. Write your app in JavaScript, pull in a dependency that serves HTTP, write enough code to route requests and start the server listening- welcome to Node. Python and Ruby offer the same. Swift gets this functionality from Kitura and Vapor. I’ve experienced C and C++ varieties.
Back in the day when Apache was seriously hot, mimicking today’s architecture would have meant writing an Apache module - the app would then live within the server and no communication channel/pipe/socket/fork/spawn would be required.
But Nginx does have a Lua module, and, I think, Perl. Afaik both are in the free version. Mostly used to augment configuration, or for lightweight interfacing directly with databases (ahem Redis and Memcached cough).
Looks like one can also write nginx modules, but I’m not familiar with nginx licensing to know if a purchase is required to use that in a commercial deployment.
Both Apache and Nginx can proxy HTTP, out of the box.
(Only, CGI managed to get headers sorta right by embedding them in protocol vars and not the other way around.)
After doing a little research, it appears this can be done by implementing or writing modules that will talk directly to server internals if I'm understanding this correctly.
CGI and FastCGI actually have the exact same flow of information as HTTP (to my knowledge). Anything else is either implemented on top of that, or you'll need a module reaching into internal functions.