Re:IndexTank / 80legs Crawlathon (developer contest)
by va_coder on Wednesday June 15, 2011 @03:23PM
(#2658632)
IndexTank as opposed to WebSolr has a proprietary api. I don't think you can easily move the data to your own servers like you can with WebSolr
[ Parent
| on HN ]
Re:IndexTank / 80legs Crawlathon (developer contest)
by diego on Wednesday June 15, 2011 @05:29PM
(#2659102)
Yes you can! Our indexes are persisted in Lucene format internally. Nobody has asked to download an index but if they asked we'd make it available.
Re:IndexTank / 80legs Crawlathon (developer contest)
by personalcompute on Wednesday June 15, 2011 @03:35PM
(#2658685)
Another problem is 80legs will only crawl pages <100kb with the free/contest plan. This means it is useless for all but the smallest pages.
Re:IndexTank / 80legs Crawlathon (developer contest)
by jdrock on Wednesday June 15, 2011 @03:42PM
(#2658712)
Er.. not true, the contest plan (which is different from the free plan) allows up to 10 MB download. Registered contestants should have access to their plans within an hour of registering.
And if you don't have it, just contact us: http://www.80legs.com/contact.html.
Re:IndexTank / 80legs Crawlathon (developer contest)
by personalcompute on Wednesday June 15, 2011 @08:54PM
(#2659699)
Ah, awesome, thank you for the correction, this allows me to continue with my plan. I said that within an hour of registering, but had asked on their support page as well.