News for hackers. Stuff that matters.
Re:Show HN: Python 3.5 Async Web Crawler Example by dham on Monday March 21, 2016 @01:44PM (#11330007)
What is the advantage of this over say using threads? Web scraping is pretty much all IO so you get big wins using threads in Python and Ruby.
Re:Show HN: Python 3.5 Async Web Crawler Example by takeda on Monday March 21, 2016 @02:28PM (#11330462)
Actually because it is all I/O is what makes it more suitable than threads. You don't have all the overhead that comes with threads, but you still get all the concurrency.

That's why for example Nginx when it was released was so much faster compared to Apache.

Re:Show HN: Python 3.5 Async Web Crawler Example by poooogles on Monday March 21, 2016 @01:54PM (#11330112)
Threads in Python creates a full posix thread which is very heavyweight compared to using AsyncIO. You should get the same throughput for a far lower resource usage.
Re:Show HN: Python 3.5 Async Web Crawler Example by platz on Monday March 21, 2016 @03:16PM (#11330953)
does asyncio in python = green threads in python, therefore defeats the GIL?
Re:Show HN: Python 3.5 Async Web Crawler Example by takeda on Monday March 21, 2016 @04:38PM (#11331756)
It does not defeat GIL, but the way asyncio is working makes GIL irrelevant.

In simplest use case you have a single thread (an actual OS thread) and as soon as certain operation blocks (e.g. waiting a response from server) another coroutine is scheduled.

The reason why GIL is irrelevant is because only a single coroutine executes at given time[1]. The way it works is similar to cooperative multitasking. The new task that was scheduled will execute until it encounters an operation that will make it block (typically an I/O operation).

[1] You actually can spin multiple threads and each could have its own event loop and then schedule coroutines on different threads.