It shouldn't take 64 lines of code to do something really simple

Re:It shouldn't take 64 lines of code to do something really... by georgemcbay on Sunday October 02, 2011 @06:29PM (#3064248)

I don't doubt this guy's assertion that the data could be formatted better, but if I were working on something like this I'd just be glad that I could access the data at all.

And 64 lines of code... bfd? That's much better than the bad old days of custom binary data formats where you have to write a thousand line data loader in C, worrying about issues like lsb vs msb, whether the number storage is properly following IEEE 754, etc.

I guess I'm just...old, because my first response to 'I had to write 64 lines of Python code to get this data into shape' is to shake my head and think "Kids these days!".

[ Parent | on HN ]

Re:It shouldn't take 64 lines of code to do something really... by repiret on Sunday October 02, 2011 @11:34PM (#3064931)

I wish I could parse everything I've been asked to parse with 64 lines of Python.

FWIW, I find it ironic that the author is complaining about whitespace-implied structure in his source data, yet uses Python.

[ Parent | on HN ]

Re:It shouldn't take 64 lines of code to do something really... by john_horton on Monday October 03, 2011 @12:13AM (#3065017)

My point was not that this was absolutely hard, but rather that's way too hard for the totally mundane thing I was trying to accomplish---namely getting public data from a US-funded statistical agency into a form suitable for further statistical analysis (which is the whole reason they make this data public).

Also, I don't see how it's ironic that I used Python---it seems irrelevant. Whitespace in Python has a well-defined, commonly understood purpose; using tabs or spaces to indicate hierarchal data relationships is not at all standard, presumably because it creates messy dependencies across rows of data.

[ Parent | on HN ]

Re:It shouldn't take 64 lines of code to do something really... by lurker19 on Monday October 03, 2011 @07:31AM (#3065879)

"Messy dependencies across rows" is exactly how Python whitespace works. That data file exactly Pythonic indentation in the main content section.

[ Parent | on HN ]

Re:It shouldn't take 64 lines of code to do something really... by john_horton on Monday October 03, 2011 @10:31AM (#3066408)

The reason indentation to imply structure doesn't make sense for datasets is that in data analysis you are constantly subsetting and sorting data at the row level---something you generally don't do to the lines of a compter program.

[ Parent | on HN ]