News for hackers. Stuff that matters.
Re:Making 1M requests with Python-aiohttp by imaginenore on Saturday April 23, 2016 @12:49PM (#11556279)
1,000,000 requests in 52 minutes is just 320 req/sec.

Am I missing something? What's so amazing about this?

I just deployed some production feed that serves at 1955 requests/second on a cheap VPS in freaking PHP, one of the slowest languages out there.

Re:Making 1M requests with Python-aiohttp by kh_hk on Saturday April 23, 2016 @01:38PM (#11556498)
> Am I missing something? What's so amazing about this?

The article is not about testing performance of a web server, but showcasing performance differences between synchronous and asynchronous code using asyncio. So, not about serving requests, but consuming.

Re:Making 1M requests with Python-aiohttp by lossolo on Saturday April 23, 2016 @02:40PM (#11556783)
Then he should change the title.
Re:Making 1M requests with Python-aiohttp by kh_hk on Saturday April 23, 2016 @07:00PM (#11557785)
Making, not serving. I think the title is pretty accurate.
Re:Making 1M requests with Python-aiohttp by nathancahill on Saturday April 23, 2016 @01:19PM (#11556413)
I don't care for PHP as much as the next guy, but it's usually in the top 25 of the web framework benchmark (most of the other top langs are Java, Go and C++): https://www.techempower.com/benchmarks/
Re:Making 1M requests with Python-aiohttp by 50CNT on Saturday April 23, 2016 @01:58PM (#11556588)
Just curious, what's up with this Ur language at both position 1 and 4? Never heard of it, and probably not experienced enough to make sense of it, but how is it that a language that doesn't even have a full official tutorial to its name beat out java, C++ and Go in those rankings by a factor of >2?

I'm genuinely curious.

Re:Making 1M requests with Python-aiohttp by nathancahill on Saturday April 23, 2016 @03:10PM (#11556973)
Ur (Ur/Web?) seems to be built very specifically for the exact things that this benchmark checks (dynamic web pages with SQL queries). So it's not surprising that those code paths are highly optimized in the language.
Re:Making 1M requests with Python-aiohttp by aaossa on Saturday April 23, 2016 @12:52PM (#11556293)
Why you say is not amazing? Honestly curious here :)
Re:Making 1M requests with Python-aiohttp by imaginenore on Saturday April 23, 2016 @01:02PM (#11556336)
Because it's trivial.

I would be interested in anything doing 10,000+ req/sec on a cheap VPS. 320 is nothing.

People achieve 2 million requests/second with C++ on EC2:

https://medium.com/swlh/starting-a-tech-startup-with-c-6b5d5...

Re:Making 1M requests with Python-aiohttp by aaossa on Saturday April 23, 2016 @01:13PM (#11556384)
Oh I see now... This speaks by it self:

C++/Proxygen =1,990,130 requests per second

Python/Tornado = 41,329 requests per second

Thanks for sharing btw

Re:Making 1M requests with Python-aiohttp by e12e on Sunday April 24, 2016 @12:02AM (#11558493)
Interesting article. Not sure that the conclusion is all that solid though: "I quantified that 1 C++ server is roughly equivalent to 40 load-balanced python servers for raw computational power based on our HTTP benchmarking. Thus using C++ can really squeeze all the computational juice out of the underlying hardware to save 1/40 off server costs."

Well, for this particular start-up, I think c++ was an excellent choice (especially as they already knew c++!) -- but what if you could still run the service on a single service with python? You would still need 2-3 servers with the c++ version (failover, test etc).

Or it might turn out that for production load, you'd need 10 python server-instances. Sure, one server could handle this with c++ -- but you're not actually saving 39/40, you're only saving 9/10, because you didn't need "the full 40".

Not to mention, that it seems unlikely that the http request stuff is the limiting factor in a distributed OLAP system. So you might do 100 "OLAPS/s", and have them easily served by a 40Kreq/s python service.

"I guess we could have written it in Python to start off with but, economically, it would be a wastage of labor cost and time because, at some stage, we would have to scrap it for a C++ version to get the performance we need. The Python code will have no economic value once scrapped."

As mentioned, I think c++ was an excellent choice, but the above assumes they'd have to rewrite the entire system in c++, and scrap all the python. Granted, if you don't know python, but do know c++ very well -- it's doubtful that it would be faster to prototype in python than to just use c++.

But if (wild guesstimate) you could get 90% of the features, at 10% the loc/dependencies in python -- who's to say that wouldn't make sense? Maybe parts could be done in c/c++, or the project could move to pypy for enough of a speedup that no complete rewrite would be needed... etc.

Overall though, I really think this is a great illustration that high-performance compiled languages offer great performance, and can be rather pleasant to work with.

Re:Making 1M requests with Python-aiohttp by jorge_leria on Saturday April 23, 2016 @05:17PM (#11557457)
The article is not about serving, but about consuming. Not the same beast.
Re:Making 1M requests with Python-aiohttp by Cyph0n on Saturday April 23, 2016 @01:35PM (#11556486)
Absolutely excellent article. I always keep C++ at the back of my mind in case I need it some day, so I think the list of libraries they used will be useful for me in the future. Thanks.
Re:Making 1M requests with Python-aiohttp by gst on Saturday April 23, 2016 @02:04PM (#11556606)
If you want to write fast C++ Web services I recommend a look at Seastar: http://www.seastar-project.org/
Re:Making 1M requests with Python-aiohttp by coldtea on Sunday April 24, 2016 @01:37AM (#11558683)
Because it's like Dr Evil asking the UN leaders for "ONE MILLION DOLLARS" to not destroy the world...

https://www.youtube.com/watch?v=cKKHSAE1gIs

Re:Making 1M requests with Python-aiohttp by merdreubu on Saturday April 23, 2016 @03:51PM (#11557130)
Because you can get 540 req/s on Raspberry Pi 2 with Elixir/Phoenix.

http://blog.onfido.com/using-cpus-elixir-on-raspberry-pi2/