News for hackers. Stuff that matters.
Re:Announcing Google Refine 2.0, a power tool for data wrang... by zachster on Wednesday November 10, 2010 @06:28PM (#1892193)
This is pretty neat, but it seems like an advanced version of Google Docs Spreadsheets. I wonder if they'll roll these features into that. The OSS project is nice for confidential data, but I think a lot of people would use a hosted version. Anyone going to set one up?

I literally just did the same exercise as demonstrated in the second video, parsing a Wiki document (the list of world religions) from Wikipedia. But it took about 30 lines of PHP.

Maybe they'll add the ability to import a web page as a data source, and export the script that does the transformations as a python script?

Okay. I'm rambling...

Re:Announcing Google Refine 2.0, a power tool for data wrang... by nswanberg on Wednesday November 10, 2010 @07:06PM (#1892282)
I haven't used some of the cooler import features of Google Docs Spreadsheets, but my guess is that Refine will be better if

a) you are not skilled in a language that handles text easily (I have friends who might have a need to aggregate that data but would have no clue how to begin writing a program to parse it).

b) don't have a Peter Norving-like facility with algorithms: http://norvig.com/ngrams/ i.e. you could code the transformation given sufficient time, but Refine would be faster, both in development and, for large data sets, run time.

Re:Announcing Google Refine 2.0, a power tool for data wrang... by thejefflarson on Wednesday November 10, 2010 @09:40PM (#1892653)
You can grab data from a web api, for example, geocoding:

http://code.google.com/p/google-refine/wiki/Geocoding