News for hackers. Stuff that matters.

A comment on Get your act together, Data.gov. Parent comment

Re:Get your act together, Data.gov by llimllib on Friday November 13, 2009 @06:24PM (#941074)
I almost always write my own stream parser with regular expressions to deal with large XML files (especially very regular ones), though it should be noted that there are stream XML parsers.
[ Parent | on HN ]
Re:Get your act together, Data.gov by physcab on Friday November 13, 2009 @08:58PM (#941259)
Knowing regular expressions is an all around good idea when doing data processing. Steep learning curve but pays itself off in increased productivity.

What stream XML parsers do you use? I just get my data ready for Hadoop and let it go.

[ Parent | on HN ]
Re:Get your act together, Data.gov by llimllib on Friday November 13, 2009 @10:58PM (#941385)
To be honest, I just kind of think I know that there are stream XML parsers? I've used cElementTree when I have small XML documents and written my own regex for larger ones. (cElementTree is definitely not a stream parser)
[ Parent | on HN ]