Re:Ask HN: Do you have any code or projects of your own you ...
by wehriam on Thursday April 30, 2009 @05:44AM
(#586607)
I'm working on a AWS stack based web crawler. It uses S3 and SimpleDB to coordinate instances. It's implemented in Python/Twisted, and extensible via plugins.
It has a low cost of entry in terms of code and costs and scales horizontally, so it's useful for all sorts of aggregation.
It has a low cost of entry in terms of code and costs and scales horizontally, so it's useful for all sorts of aggregation.