News for hackers. Stuff that matters.
Re:Managing two million web servers by signa11 on Monday March 14, 2016 @06:25AM (#11281971)
> Also if an apache process dies it will not crash other requests

can you have 2m Apache processes on your machine ?

Re:Managing two million web servers by janvidar on Monday March 14, 2016 @06:40AM (#11282034)
I suppose this depends on your machine.
Re:Managing two million web servers by 59nadir on Monday March 14, 2016 @07:39AM (#11282238)
What if there was a model where it depended less heavily on your machine because of over 20 years of development with the goal of providing stable, robust systems aimed specifically at allowing you to run many things in parallel? What if the framework/API for it was developed for production systems from the beginning, meaning its API was made singularly for being able to easily spawn, manage and control these processes?

The whole point of this article is that there is an architecture behind this and that this architecture is believed to be generally useful.

Re:Managing two million web servers by jontro on Monday March 14, 2016 @06:32AM (#11282001)
Well that would depend how long you run it for. They don't have to be simultaneous. It wouldnt make much sense to recycle the process for each request however.

MaxConnectionsPerChild defines how often the process is recycled

Re:Managing two million web servers by klibertp on Monday March 14, 2016 @06:58AM (#11282094)
> They don't have to be simultaneous.

Now you're just moving a goalpost. We're talking about Erlang processes - you can have millions of them simultaneously.

> It wouldnt make much sense to recycle the process for each request however.

It does make a great deal of sense from the fault-tolerance perspective. It also helps security.

I have an unpleasant feeling that you don't know much about Erlang and its philosophy. The "one process per connection" architecture has many benefits; it's a canonical way of getting concurrency and parallelism on Unix systems, for example. The problem is the way OSes handle and schedule processes, it simply doesn't work with a very large number of them. Erlang implements its own kind of processes, which enforce the same constraints os OS-level processes without taking megs of RAM each and without being pressed after spawning a few thousand processes.

Re:Managing two million web servers by jontro on Monday March 14, 2016 @08:02AM (#11282298)
What I'm saying is that it's misleading to call one process a web server when they really are saying "How we handle 2 million simultaneous web requests". Also I would be very interested in reading such a writeup. There are many different approaches to dealing with it (i.e. on thread per connection / one process / poll based approch). I certainly can see the benefits of having it in lightweight threads called processes in this context.
Re:Managing two million web servers by klibertp on Monday March 14, 2016 @09:02AM (#11282585)
> What I'm saying is that it's misleading to call one process a web server when they really are saying [...]

No. It's simply a terminology used in Erlang, there's nothing misleading in it, once you know the definition. I mean, there is very specific thing called "server" in Erlang: http://erlang.org/doc/man/gen_server.html and also "processes" are described here: http://erlang.org/doc/reference_manual/processes.html

What's highly misleading is using more general definitions of such words instead of Erlang-specific meanings. This is a conversation about Erlang and it makes sense to use the terms accepted in Erlang; you can't blame anyone for the fact that some of these terms have meanings different than you'd expect.

Re:Managing two million web servers by Vendan on Monday March 14, 2016 @10:54AM (#11283177)
It's misleading because there are now 2 different definitions of the exact same word at work. It's a computer running a process running an Erlang VM that is then running multiple Erlang processes. It is completely reasonable to be annoyed at that. Many languages have a concept of threading in userspace rather then kernel space, but they don't just call it threading, they give it another name so that it can be distinguished from the existing concept. For example, stackless python's tasklets, Go's goroutines, green-threads.
Re:Managing two million web servers by klibertp on Monday March 14, 2016 @11:59AM (#11283581)
Well, you can't really expect to hold a meaningful conversation about a language without knowing, at least, its basic concepts. With Erlang "processes" are front and center; you read about them in every tutorial and reference, in most blogposts, basically everywhere Erlang related, ad nauseam. So you're guaranteed to understand what kind of processes we're talking about if you have any familiarity with it at all.

And there is a good argument in favor of calling them "processes". Conceptually, both are providing a very strong separation/isolation and cleanup guarantees, which is not true for any of other constructs you mentioned. The range of operation you can perform on a process: spawning, monitoring, sending signals - is the same for both OS and Erlang processes. You can very much think and use both kinds in exactly the same way, other than rather excessive memory usage by the OS-level kind.

I actually have a somewhat related talk tomorrow, about - in essence - replacing Celery in a Python project with Elixir. Integration via ErlPort makes it natural to think of Python worker processes as simply (Erlang) processes - you spawn and link and monitor them just like you would normal (Erlang) processes. It even works with Poolboy out of the box.

Re:Managing two million web servers by Vendan on Monday March 14, 2016 @02:04PM (#11284371)
On the other had, the exact same comparisons hold true for threads vs. green threads. They are extremely similar, they perform almost identically to threads in many languages, and yet, green threads have their own name, as they are a different thing. I can understand that Erlang is built around calling them just "processes", but that's, in my opinion, a bad idea. To me, the fact that Erlang developers insist on co-opting the name "process" seems arrogant.
Re:Managing two million web servers by phamilton on Thursday March 17, 2016 @01:45PM (#11306120)
Time out. Let's go back to the late 80s, when Erlang was written. It ran on Ericsson's switches. The platform needed to describe a unit of concurrency that provided execution and memory isolation. Process fits that definition perfectly.

Fast forward to today. Look at projects like LING, where the Erlang VM runs directly on Xen, without a traditional operating system. Processes in Erlang are 100% analogous to OS processes.

Also look at something like Virtualbox or VMware. Is your process running in a VM less of a process than the OS process powering the VM itself?

Re:Managing two million web servers by klibertp on Tuesday March 15, 2016 @06:06PM (#11293385)
The slides from the mentioned talk, in case someone is interested: https://klibert.pl/statics/python-and-elixir/
Re:Managing two million web servers by jontro on Monday March 14, 2016 @12:33PM (#11283777)
Jerfs comment here pretty much sums up my thoughts and what I was trying to argue (definitely in a much better way): https://news.ycombinator.com/item?id=11282501
Re:Managing two million web servers by toolz on Monday March 14, 2016 @08:47AM (#11282514)
>it's misleading to call one process a web server

How is that even slightly misleading? That's exactly what many web servers on erlang do.

Re:Managing two million web servers by karmajunkie on Monday March 14, 2016 @10:20AM (#11283017)
Erlang processes aren't threads and its misleading (or at least misguided) to use that terminology for them. [1]

[1] http://stackoverflow.com/a/32296577

Re:Managing two million web servers by Vendan on Monday March 14, 2016 @10:49AM (#11283152)
Calling them processes is even more misleading. A process has a specific meaning on most operating systems. Erlang "processes" should, if anything, be referred to as "green-processes" or "lightweight processes", as to do otherwise leads to dangerous comparisons, such as implying that a server running an Erlang server has 2 million processes running, when in reality it has 1 (or several) Erlang processes, running a VM that is in turn running 2 million Erlang "green-processes". And while "threads" is incorrect, Erlang processes are closer to green threads then to any other concept.

    The Erlang virtual machine has what might be called "green processes" – they are like operating system processes (they do not share state like threads do) but are implemented within the Erlang Run Time System (erts). These are sometimes referred to as "green threads", but have significant differences from standard green threads.
Re:Managing two million web servers by 59nadir on Monday March 14, 2016 @07:37AM (#11282224)
> Well that would depend how long you run it for. They don't have to be simultaneous.

The long way of saying "No, I can't, but I really don't want to accept that there might be a better model for this than my out of the box Apache setup."

Re:Managing two million web servers by laumars on Monday March 14, 2016 @09:00AM (#11282578)
Theoretically yes. Pragmatically no. There comes a point when it makes more sense to scale sideways (more OS instances) instead of up (more processes on beefier hardware).

It's no secret that Apache isn't the leanest of HTTP daemons, but that's really a separate point to the argument of whether you can reasonably argue a process as a web server.