News for hackers. Stuff that matters.
Re:The C10K problem by pyalot2 on Monday February 17, 2014 @02:03AM (#7250958)
The 10k problem is still not solved. A run of the mill VPS typically manages around 1k simultaneous connections and 10k requests/s. That is, over plain http. Over TLS this goes down to about 100 connections and 600 requests/s. A cheap amazon instance will usually be around 1/5th of all that.

With a bit of tweaking you can get the plain http case a bit higher, but the TLS route will not get much better, because TLS isn't written with speed in mind (the handshake is slow and expensive) and the prevalent implementation (OpenSSL) is not written for high performance servers (it basically dictates that you run blocking sockets in a thread per connection).

Unfortunately spdy and various http2 proposals rely on TLS (in order to punch trough proxies), which means going back in server performance about 10 years.

So it is of little surprise that companies have started offering "cloud" solutions, because the typical VPS can't handle todays high traffic internet over TLS (worse than plain http by a factor of 10) and the typical cloud server is worse than a VPS by a factor 5, creating a 50x performance degradation, artificially. Obviously when faced with the question of running 10 servers, or 100, most small companies turn to the even worse "cloud" solution, requiring even more servers (500).

The whole affair is a sodding mess, and we're wasting massive amounts of energy and capital on insisting on doing things inefficiently. This is because by rights our VPS servers should easily be able to break trough the 10k limit in every way, but it can't because the OS wastes a lot of time running an inefficient network stack as well as that TLS and OpenSSL can't be bothered to get their act together.

And that is how, in the year of our lord 2014, more than 15 years after somebody writing the 10k problem article, and after webservers becoming at least 128x faster then back in the 90ties, most websites out there can still not stand up to serious traffic, take ages to load, and are hosted by an infrastructure (routers and whatnot) that easily buckle under even light DDOSing.

Re:The C10K problem by Nursie on Monday February 17, 2014 @02:56AM (#7251049)
There are many commercial alternatives to OpenSSL. I've worked with one or two. There's bound to be one that suits your needs. Of course you may also need to go to a full commercial, non-FOSS stack, but I'm sure plenty of companies will have those to sell too.
Re:The C10K problem by wbsun on Monday February 17, 2014 @02:33AM (#7251007)
When it goes to transactions and stateful sessions, C10K, or even C1K is far from solved yet.
Re:The C10K problem by pyalot2 on Monday February 17, 2014 @02:44AM (#7251023)
But that's a bit absurd isn't it? A recent core i7 reaches 124850 MIPS. That means that if it takes 1/1000th of a second to handle a connection, the CPU could execute 124 million instructions during that time. At 10/10000th it could still execute 12 million instructions. A minimal asynchronous connection handling certainly doesn't require more than say 10'000 instructions, so our servers are under-delivering on their performance at least by a factor of 1200x, perhaps even by a factor of 12000x.

Our servers should be able to surpass C1k easily, even C10k shouldn't tax them. They should, by rights, only be taxed by the C10m problem.

Re:The C10K problem by sharpneli on Monday February 17, 2014 @04:48AM (#7251293)
There is one number that has not really changed. It's memory latency, and another is the processor clock speed.

The latency of main memory read is still around 100ns. It has been around that for over 10 years now. It means your CPU will have to wait for hundreds of clock cycles to get a read from RAM if it's not in the cache, and in huge datasets it probably is not in cache.

Another issue is the processor clock speed. Yes it is true that modern i7 can reach 124850 MIPS. However that number comes from having 4 cores with each of them being able to reach up to 8 instructions per clock. You are still limited in executing dependent instructions.

That sounds a lot. But one must remember that it reaches 8 instructions per clock only when the instructions are a good mix of float/int instructions, no branches and the instructions are not dependent on eachother. In practice you reach maybe 1-2 instructions per clock. In some code it can go even to 0.5 IPC (bunch of unpredictable branches and whatnot).

Writing a code that takes advantage of large memory bandwidth and poor latency combined with massive CPU performance if the instructions are not too dependent on eachother is almost like writing modern GPU programs.

It would be interesting to see what kind of an web server perf one could get by carefully writing it in OpenCL (using CPU target, not GPU).

Re:The C10K problem by pyalot2 on Monday February 17, 2014 @04:54AM (#7251311)
Yeah I'm not disputing that there aren't bottlenecks in the system (it's not only the memory, the bus between NIC and CPU is also to blame).

But, blaming bad I/O performance on large datasets misses the point slightly. You can perfectly well write a program that doesn't use heap, and has less stack use than those processors have L2 cache... (of course that's a test program).

But the network performance is probably not bound by system latency as much as by an abysmally bad software stack, started with the kernel to the networking stack to the sheer idea of TLS and to the implementation of TLS (OpenSSL).

Indeed, there's been calls to get rid of it all, all the layers and whatnot, and bann the OS from all but one or two cores and get rid of the whole network stack and layers and implement the networking directly in the application that needs to do it.

Re:The C10K problem by rbanffy on Monday February 17, 2014 @05:27AM (#7251390)
> there's been calls to get rid of it all, all the layers and whatnot, and bann the OS from all but one or two cores and get rid of the whole network stack and layers and implement the networking directly in the application that needs to do it

You certainly know building and supporting that would cost more or less the same as building and operating a sizeable datacenter. If it succeeds.

Using all processing power a modern CPU offers on real code with real data is almost impossible. And it's not only memory latency and instruction interdependence - there are latencies all over a PC even before you leave the rackmount chassis. The supporting network is another source of uncontrollable latencies. Most apps I manage spend 99.99% of their time waiting for something to happen, be it the next packet, be the results from another server, which is actually a cluster behind one or more load balancers.

You may get some better cache hit ratios by tweaking thread/core affinity, but it won't take you to where you want to be.

If you really need that much performance, I'd suggest building your own VLIW architecture and generate the instruction mix on auxiliary CPUs as a single continuous thread on the fly based on all incoming requests for the VLIW core to devour. That would be a huge undertaking, but it would also be pretty cool CompSci.

Re:The C10K problem by vidarh on Monday February 17, 2014 @07:17AM (#7251695)
> You certainly know building and supporting that would cost more or less the same as building and operating a sizeable datacenter. If it succeeds.

There are plenty of solutions for that already. For the simplest case of the user-space networking, you can pick a number of "off the shelf" solutions for it:

http://lukego.github.io/blog/2013/01/04/kernel-bypass-networ... http://www.openonload.org/

Re:The C10K problem by theknown99 on Monday February 17, 2014 @04:16AM (#7251230)
Not really true unless you use crappy software.

I run a few thousand simultaneous connections on AWS small instances without any issues. (HTTP + HTTPS).

The key is writing good software. Which a lot of programmers are still really bad at.

Re:The C10K problem by pyalot2 on Monday February 17, 2014 @04:31AM (#7251260)
You know, it's cute when somebody who wasn't even born when I started writing software for servers tries to discredit me.
Re:The C10K problem by theknown99 on Monday February 17, 2014 @05:38AM (#7251418)
It's cute when you make ridiculous assumptions about when people were born.

Lets stick to the facts...

You stated that Amazon instances couldn't cope. They can. They can cope just fine with thousands of simultaneous HTTP and HTTPS connections. If you can't get your amazon instance to do this, you're using crappy software.

> "The 10k problem is still not solved. A run of the mill VPS typically manages around 1k simultaneous connections and 10k requests/s. That is, over plain http. Over TLS this goes down to about 100 connections and 600 requests/s. A cheap amazon instance will usually be around 1/5th of all that."

Where the hell are you getting those poor figures from? Are you using Lisp or something?

> "and the prevalent implementation (OpenSSL) is not written for high performance servers (it basically dictates that you run blocking sockets in a thread per connection)."

Again - you're using crappy software. Don't use it.

The 10k problem is historically interesting, but for anyone who knows how to program, it's not really an issue.

Re:The C10K problem by pyalot2 on Monday February 17, 2014 @05:47AM (#7251446)
If you use anything like Apache, lighttpd, Cherokee, Nginx, Tux, etc. they all use OpenSSL, and nothing else.
Re:The C10K problem by theknown99 on Monday February 17, 2014 @08:51AM (#7252045)
Well don't use them then!

Seriously. Writing a webserver isn't a big job. It's not very hard to program something that runs better than the ones you list, for specific use cases.

Re:The C10K problem by pyalot2 on Monday February 17, 2014 @11:09AM (#7252687)
Been there done that, but the fact is, there is no alternative to OpenSSL I could see if you want to have SSL/TLS 1.1, 1.2 and 1.3.
Re:The C10K problem by vidarh on Tuesday February 18, 2014 @04:44AM (#7257237)
Any reason why none of the open implementations listed here are suitable:

http://en.wikipedia.org/wiki/Comparison_of_TLS_implementatio...

?

Re:The C10K problem by theknown99 on Monday February 17, 2014 @12:04PM (#7252973)
If only there was a way to replace the functionality provided by OpenSSL, by writing some code :(
Re:The C10K problem by pyalot2 on Monday February 17, 2014 @05:10PM (#7254859)
To rewrite OpenSSL you'll need to be a security and cryptography programmer. And even if you are, the task is daunting to say the least.
Re:The C10K problem by rbanffy on Monday February 17, 2014 @05:14AM (#7251353)
It would be better if you both gave more details on what and how you do so the possible mistakes from both sides could be pointed out and we could learn from them.

From a quick glance on your post, I too suspect there is something wrong - your performance shouldn't be that bad, but there is not enough information to point where.

Re:The C10K problem by pyalot2 on Monday February 17, 2014 @05:27AM (#7251387)
That's really too long to list, I could conceivably write a book about it.

But, fortunately it's relatively easy to test. You get whatever server you prefer, and install nginx and install a bunch of performance testing tools (like siege, ab etc.) and then you test different concurrency load scenarios.

No custom software, and widely regarded the fastest webserver out there.

Re:The C10K problem by rbanffy on Monday February 17, 2014 @05:30AM (#7251396)
As I pointed elsewhere, most recent servers are just overgrown IBM 5150 PCs, with much faster CPUs, immense amounts of memory and storage and somewhat faster buses. They are desktop PCs misused as servers.
Re:The C10K problem by theknown99 on Monday February 17, 2014 @05:40AM (#7251423)
> and install nginx

Yeah there's your problem...

Re:The C10K problem by pyalot2 on Monday February 17, 2014 @05:51AM (#7251454)
You can replicate the exact same experience (only worse) with any other webserver like apache, lighttpd, cherokee, etc.

Tux may be a bit faster, but it can't serve dynamic content nearly as fast as it serves files, so that's pretty much useless.

Re:The C10K problem by rbanffy on Monday February 17, 2014 @06:43AM (#7251596)
Have you tried other operating systems?
Re:The C10K problem by theknown99 on Monday February 17, 2014 @08:49AM (#7252038)
Have you tried writing your own webserver? Then tuning it to your particular use case?
Re:The C10K problem by pyalot2 on Monday February 17, 2014 @11:08AM (#7252679)
Yes
Re:The C10K problem by theknown99 on Monday February 17, 2014 @11:59AM (#7252953)
hmm so I've written a webserver and am able to get thousands of concurrent requests on a small amazon instance without issues.

And you've written a webserver and you can't.

Interesting...

Re:The C10K problem by rbsn on Monday February 17, 2014 @06:30PM (#7255230)
Can we see your results?
Re:The C10K problem by seunosewa on Monday February 17, 2014 @02:00PM (#7253725)
If you care about efficiency you should not use VPSs. You should dedicated servers.