Re:Shoestring Budget? Starting to feel growth issues on you...
by silentbicycle on Tuesday September 09, 2008 @01:37PM
(#299628)
Or, briefly: Certain classes of Python programs can actually be faster than average C, because they consist of small amounts of Python code connecting libraries that are already written in very thoroughly tuned C.
Re:Shoestring Budget? Starting to feel growth issues on you...
by jrockway on Tuesday September 09, 2008 @01:48PM
(#299640)
Why use threads when you can use an event-based IO system? (I think Twisted is the Python way of doing this.)
Re:Shoestring Budget? Starting to feel growth issues on you...
by jmtulloss on Tuesday September 09, 2008 @01:57PM
(#299649)
I was just going off of what he had already done. I agree that async is a better way to do this.
Twisted is one way, but Python ships with asynchronous libraries http://docs.python.org/lib/module-asyncore.html.
Re:Shoestring Budget? Starting to feel growth issues on you...
by llimllib on Tuesday September 09, 2008 @01:57PM
(#299648)
> The problem would be the GIL (Global Interpreter Lock). You couldn't actually have more than 1 thread running at a time.
Not actually true; socket calls release the GIL when they're not active, as do many of python's C libraries. Especially the better-written standard ones.
Re:Shoestring Budget? Starting to feel growth issues on you...
by jmtulloss on Tuesday September 09, 2008 @01:59PM
(#299654)
True, although I would argue that my statement is still correct. You cannot have more than 1 thread running simultaneously.
The GIL is not as big of a problem as most people see it, it only interferes with CPU bound, highly parallelizable tasks.
1. Spawn some threads (Python uses native threading) 2. Connect to and load some URLs (This isn't going to be slow anywhere.) 3. Run some regular expressions (Python's regexp engine is all C) 4. Write to the disk. (Python uses the standard system libraries for this)
It wouldn't be as fast, but it might be surprisingly close. There would be fewer lines of code to boot.
The problem would be the GIL (Global Interpreter Lock). You couldn't actually have more than 1 thread running at a time. It sounds like the man only has 1 processor, so that wouldn't be the end of the world, but if he has more, you can just swap in the process module for the threading module. Of course, then you might have some RAM issues.