News for hackers. Stuff that matters.
Re:IndexTank / 80legs Crawlathon (developer contest) by btucker on Wednesday June 15, 2011 @03:05PM (#2658561)
Is there any documentation on how to get started using these two products together?
Re:IndexTank / 80legs Crawlathon (developer contest) by diego on Wednesday June 15, 2011 @03:20PM (#2658619)
It should be pretty straightforward:

- run a crawl at 80legs - download the results as a csv or xml - feed them to IndexTank using your client of choice

Over the weekend I'll put on my (rusty) hacker hat, do an example and blog about it.

Re:IndexTank / 80legs Crawlathon (developer contest) by jdrock on Wednesday June 15, 2011 @03:24PM (#2658641)
Specific documentation for 80legs is available at http://wiki.80legs.com. To get fancy, you may want to check out http://wiki.80legs.com/80apps to make your own custom extractors. Note that any custom extractors you write will output files in binary in 80legs, so you'll need to convert the byte array to .txt, .csv or .xml or whatever format you want.

(We post custom extractor results as binary because you can also return stuff like images!)