News for hackers. Stuff that matters.

A comment on Tiny, dirty, iffy, good enough, basic multi-threaded web crawler in Python. Parent comment

Re:Tiny, dirty, iffy, good enough, basic multi-threaded web ... by sebcat on Wednesday August 12, 2015 @12:22PM (#10048674)
You can have several outgoing connections in one single thread, at the same time.

Threading is for CPU bound problems that requires a shared memory space.

I/O multiplexing and/or cooperative multitasking means you can have thousands concurrent connections per thread.

[ Parent | on HN ]
Re:Tiny, dirty, iffy, good enough, basic multi-threaded web ... by tokenizerrr on Wednesday August 12, 2015 @03:06PM (#10049917)
Sure. The parent comment referred to one download per thread so I did as well.
[ Parent | on HN ]