News for hackers. Stuff that matters.

A comment on Ask HN: Do you know a good resource for large data scraping job?.

Re:Ask HN: Do you know a good resource for large data scrapi... by devs1010 on Thursday February 09, 2012 @06:13PM (#3573629)
I'm not sure exactly what sort of answer you are expecting. Unless the data you want is in a standardized format (such as a standardized XML schema), any effort to extract data would require writing custom parsers for each set of data that has a different structure. I'm not sure if you are asking for advice on which technology stack to use for writing this or are looking for a pre-made tool that can extract this for you? There may be some tools that can "attempt" to do this without requiring you to write custom code but I am not sure how effective they would be.
[ Parent | on HN ]
Re:Ask HN: Do you know a good resource for large data scrapi... by hugo31370 on Thursday February 09, 2012 @07:55PM (#3573986)
I believe it has to be a person. I've used Mechanical Turk in the past and it's great for easy, simple tasks. This one requires a little learning, which means sticking to one person/team would be best because they can quickly get faster and more efficient.

I'm looking for advice on companies or people you've used in the past that you liked. Thanks!

[ Parent | on HN ]