Shoestring Budget? Starting to feel growth issues on your back-end? Embrace unix and C

Re:Shoestring Budget? Starting to feel growth issues on you... by zenspider on Tuesday September 09, 2008 @02:03PM (#299658)

perl threading was (and I'm sure still is) absolutely horrid.

I'd be interested in seeing the results of a rewrite not to C, but to python or ruby where the threading support is much much better. Then you could rewrite functions at a time in C as needed, but not have the extra burden of rewriting the whole thing.

I totally agree with the rest of the approach. Going low tech and using unix tools is a very good way to reduce overhead, increase parallelism, and delay calculations. One of the nice things about this approach is you can cobble up another $50 unix box to do some of the bulk processing via nfs or other means.

Congrats... It sounds like a very interesting project.

[ Parent | on HN ]

Re:Shoestring Budget? Starting to feel growth issues on you... by kaens on Tuesday September 09, 2008 @03:14PM (#299733)

Threading support is better in python than in perl? I'm not familiar with perls threading model (or libraries), but this seems wrong - considering things like the GIL in python.

Could you elaborate a bit on this?

[ Parent | on HN ]

Re:Shoestring Budget? Starting to feel growth issues on you... by zenspider on Tuesday September 09, 2008 @03:48PM (#299782)

I'm much more familiar with threads in ruby. It would actually be a real boon to this project imo.

I don't have as much experience with python threads as I do ruby. I've toyed with the internals of python and perl as well and know perl to be a wasteland and python to be rather elegantly clean. So I would expect python's threads to be better implemented. Poking at it, it does appear to be so.

[ Parent | on HN ]

Re:Shoestring Budget? Starting to feel growth issues on you... by jmtulloss on Tuesday September 09, 2008 @06:24PM (#299885)

Doesn't ruby only have green threads?

[ Parent | on HN ]

Re:Shoestring Budget? Starting to feel growth issues on you... by michaelneale on Tuesday September 09, 2008 @07:56PM (#299942)

Yes - unless its jruby then they are non green (forget the term for it at this time of the morning).

[ Parent | on HN ]

Re:Shoestring Budget? Starting to feel growth issues on you... by zenspider on Wednesday September 10, 2008 @02:59PM (#300645)

yes... and they're great. Esp for situations like the OP was describing. Fire off a ton of incredibly lightweight threads and they'll quietly sit waiting for IO... perfect.

[ Parent | on HN ]

Re:Shoestring Budget? Starting to feel growth issues on you... by kingkongrevenge on Tuesday September 09, 2008 @06:33PM (#299888)

The thing with perl threads is that pretty much nobody uses them. They work fine, as far as I know, but with a default 16MB stack size and "share nothing" semantics don't start more than a few. Perl is more unix centric and people just prefer to fork and use the well baked IPC mechanisms. Various libraries such as POE make fork+IPC easy enough that it's hard to see the need for threads in the kinds of domains where a language like perl is applicable.

[ Parent | on HN ]

Re:Shoestring Budget? Starting to feel growth issues on you... by kaens on Tuesday September 09, 2008 @07:09PM (#299915)

people just prefer to fork and use the well baked IPC mechanisms

Ahh, that makes sense. I'm willing to bet that the vast majority of perl code runs on *nix systems (I know that with the exception of one app, all the perl I've ever written was only run on linux servers).

[ Parent | on HN ]

Re:Shoestring Budget? Starting to feel growth issues on you... by zenspider on Wednesday September 10, 2008 @02:57PM (#300642)

they certainly didn't work fine when I tried to use them... after they'd gone into production releases of perl, the canonical demo/example scripts that were out there all crashed the perl runtime in a fiery horrible death.

I can only hope that it has improved since then, but I've moved on to greener pastures.

[ Parent | on HN ]

Re:Shoestring Budget? Starting to feel growth issues on you... by kingkongrevenge on Wednesday September 10, 2008 @03:35PM (#300678)

They were clearly marked as experimental in the documentation all the way up to recent 5.8 releases. From what I hear they work fine now.

[ Parent | on HN ]