News for hackers. Stuff that matters.
Re:Nxweb – Fast and Lightweight Web Server by ktRolster on Saturday February 27, 2016 @02:52AM (#11186451)
A lot of ad-tech companies build ad-servers in C, because the latency is so crucial in that context.
Re:Nxweb – Fast and Lightweight Web Server by geocar on Saturday February 27, 2016 @03:38AM (#11186545)
Specifically it's garbage collection.

e.g. node.js has no problem getting 20k/sec per core, but a stall at the wrong time kills every pipelined HTTP request that follows (until you tear down the connection and restart it).

Re:Nxweb – Fast and Lightweight Web Server by ktRolster on Saturday February 27, 2016 @01:33PM (#11187928)
Worth also mentioning that ad servers tend to have massive RAM requirements (again, for speed). GC in a 30GB JVM can take 10 minutes. To handle it, companies mark the boxes as 'inoperable' when they are in GC mode and remove them from the cluster until thy are ready to return.

All of this is motivation to rewrite everything in C.

Re:Nxweb – Fast and Lightweight Web Server by paulddraper on Saturday February 27, 2016 @10:03PM (#11189623)
10 minutes is nowhere even close to my experience, especially because Oracle/OpenJDK has incremental GC.
Re:Nxweb – Fast and Lightweight Web Server by ex3ndr on Saturday February 27, 2016 @05:09AM (#11186677)
20k/sec is a joke performance. Java/Scala with Akka handles millions of packages per sec and this is performance.
Re:Nxweb – Fast and Lightweight Web Server by geocar on Saturday February 27, 2016 @06:17AM (#11186805)
40k/sec http requests (no pipelining) per core is about as fast as it gets unless you move TCP into user space.

Reports of "millions per sec" are usually talking about messages on established channels across all CPUs.

If you're actually aware of a java based web server that can beat even 150k http requests measured by wrk or similar on local host I'd like to see it.

Re:Nxweb – Fast and Lightweight Web Server by zzzcpan on Saturday February 27, 2016 @10:56AM (#11187412)
> 40k/sec http requests (no pipelining) per core is about as fast as it gets unless you move TCP into user space.

I think it can be micro-optimized beyond that. With predictions to avoid unnecessary syscalls, with syscalls grouped together to make cpu more efficient for the rest of the time it spends in event loop, and if it's possible to modify kernel a bit - with batching syscalls together to make them very cheap.

Re:Nxweb – Fast and Lightweight Web Server by geocar on Saturday February 27, 2016 @11:37AM (#11187548)
At some point, your complexity gets bigger than simply coding a state machine that operates directly on the network buffers themselves:

That is to say, I suspect that if micro-optimisations can double our performance, they will be more complicated than just writing a customised ring0 that implements HTTP directly inside the network driver.

Here is how I'm looking at it:

• 10Gb/sec network port

• 4k max requests and responses

• == 1.3 million HTTP requests per second.

Now the problem is that main memory is not much faster than our fastest network: About 15Gb/sec, so what we're talking about here is code and state staying entirely in L1, and streaming the network buffers across the CPU, and responding in one pass, to get that 1.3 million optimal performance.

My dash server gets ~135k HTTP requests per second on localhost (I should be able to approach 300k/sec over a network if I ever get around to it). That's 22% of our optimal performance, and a lot better than any other HTTP server I'm aware of.

At this speed, one of those micro-optimisations `writev()` is actually slower than `write()` -- likely because the code path is shorter in the simpler codebase -- but it illustrates my concern nicely: That we are close to that break-even point with the optimisations we can make. If we make our server bigger and more complicated, it might not make our programs any faster.

That suggests to me that the solution is actually fewer, simpler syscalls, not more, bigger ones.

Re:Nxweb – Fast and Lightweight Web Server by vegardx on Saturday February 27, 2016 @02:09PM (#11188030)
What kind of "main memory" are you talking about? Regular, consumer grade memory, will have a bandwidth at least ten times faster than your 10Gb/s network interface. Change the bit to a byte and you're a little closer.
Re:Nxweb – Fast and Lightweight Web Server by pjmlp on Saturday February 27, 2016 @03:39AM (#11186546)
Or they don't know any better.

I took part in a few projects that replaced high throughput servers handling mobile network traffic from C++ to Java.

Re:Nxweb – Fast and Lightweight Web Server by eva1984 on Saturday February 27, 2016 @05:08AM (#11186675)
Can confirm.

I work at two ad-tech related companies, one large one medium, both use Java from the beginning. And I know other company using Python/Golang as well.

Didn't know C(not C++) is particular popular until today. Note that ad company are pretty business focused, add or remove features for big clients are pretty common, so development efficiency matters a lot.

Re:Nxweb – Fast and Lightweight Web Server by brobinson on Saturday February 27, 2016 @02:49PM (#11188173)
We went from C++ to Go, and started to auto-lose any auctions whose bid requests were in progress during a GC cycle. Go 1.1/1.2 had atrociously long GC pauses (sometimes up to 500ms). You are correct that development efficiency was very important, so losing the auctions was justifiable.
Re:Nxweb – Fast and Lightweight Web Server by iampims on Saturday February 27, 2016 @04:26PM (#11188594)
Are things different with Go 1.5/1.6 now that GC has start to be optimized?
Re:Nxweb – Fast and Lightweight Web Server by elithrar on Sunday February 28, 2016 @10:32AM (#11191109)
As of Go 1.5 (1.6, released last week, makes further improvements):

> The "stop the world" phase of the collector will almost always be under 10 milliseconds and usually much less.

https://golang.org/doc/go1.5

Re:Nxweb – Fast and Lightweight Web Server by brobinson on Saturday February 27, 2016 @04:44PM (#11188663)
Not sure, sorry. I haven't written any high-performance stuff in it since 2014.
Re:Nxweb – Fast and Lightweight Web Server by cmrdporcupine on Saturday February 27, 2016 @07:48AM (#11186951)
Did you have tight and consistent latency requirements?

GC pauses are a killer. They can be worked around, but it takes intensive tuning.

The RTB bidder I wrote many moons ago at a startup was fast as hell, but had problems in the 95th percentile of requests meeting the latency targets, due to GC.

The ad servers at the big ad tech heavy weights are in C++.

Re:Nxweb – Fast and Lightweight Web Server by pjmlp on Saturday February 27, 2016 @08:49AM (#11187067)
Of course, people aren't going to be happy when they packets get dropped or the network monitoring software wasn't able to provide a (soft) real time view of what was happening.

Also note I wasn't doing this alone, it was a very big project a mobile operator.

Re:Nxweb – Fast and Lightweight Web Server by ktRolster on Saturday February 27, 2016 @04:18PM (#11188560)
Your experience isn't representative, so there were probably other confounding factors, like the C++ wasn't very well written, or you weren't dealing with petabytes of data or something like that.
Re:Nxweb – Fast and Lightweight Web Server by faint_coder on Saturday February 27, 2016 @03:58AM (#11186578)
Or maybe THEY don't know any better way to optimize/use C++ instead of switching to Java
Re:Nxweb – Fast and Lightweight Web Server by pjmlp on Saturday February 27, 2016 @09:14AM (#11187122)
Given that I remember the days when C and C++ compilers generated code worse than a junior Assembly programmer, I always find such comparisons interesting.

Not that they aren't true, rather their validaty depends a lot of programmer skillset and compilers being used.

Re:Nxweb – Fast and Lightweight Web Server by goldenkey on Saturday February 27, 2016 @04:01AM (#11186588)
Java is garbage unless you specific 20 -D options to bend it to your will.
Re:Nxweb – Fast and Lightweight Web Server by pjmlp on Saturday February 27, 2016 @09:12AM (#11187113)
I could say the same thing of compiler optimisation flags.
Re:Nxweb – Fast and Lightweight Web Server by goldenkey on Saturday March 05, 2016 @10:10AM (#11229673)
Pretty sure -O3 and we are set. False dichotomy. Java has a lot of users but sheep still get slaughtered.