Re:Don’t Slurp: How to Read Files in Python
by paulgb on Friday September 27, 2013 @07:47PM
(#6460251)
I agree that it's good to have an easy way to read a whole file in, but when I think about it I can't think of any case where I've had to write code to read a file that I didn't want to process line-by-line, in which case the non-slurping method is actually less code.
Re:Don’t Slurp: How to Read Files in Python
by j_baker on Saturday September 28, 2013 @12:53PM
(#6462319)
It sounds like you've only read in line-by-line files then. I mean, it doesn't make sense to read in a JSON or XML document line-by-line. Nor does it make sense to read in most binary files line-by-line.
Some formats, such as csv do make sense to read in line-by-line though.
Re:Don’t Slurp: How to Read Files in Python
by paulgb on Sunday September 29, 2013 @09:14PM
(#6467277)
I've read in lots of formats, including many XML and binary files. 99% of the time there is a library that already handles the low-level interaction with those files. The only times I've had to write original Python code that reads files directly have typically been things that could be processed line by line (eg various record-based data formats). I'm trying to think of a counter example but I'm coming up blank.
depends on how much you care about performance/memory usage.
for one, variance in the line lengths (by byte count) will probably force the usage of inefficient buffer sizes
also, the parent comment gave one universal example: binary files
Re:Don’t Slurp: How to Read Files in Python
by repsilat on Friday September 27, 2013 @08:55PM
(#6460434)
Even with binary files, the only reasonable ways I can think to process them are stream-wise and with mmap. (In fact, why would you ever slurp a file when you can mmap it instead?)
Re:Don’t Slurp: How to Read Files in Python
by vacri on Friday September 27, 2013 @08:14PM
(#6460333)
And for someone just learning Python
The context of the question and answers was 'a LinkedIn group for professional Python programmers'.
Re:Don’t Slurp: How to Read Files in Python
by j_baker on Friday September 27, 2013 @09:42PM
(#6460538)
If you don't know how to slurp a file, you're not a professional Python programmer.
Re:Don’t Slurp: How to Read Files in Python
by vacri on Saturday September 28, 2013 @09:59PM
(#6463801)
The context of the answer was an expert talking to the experts who answered the newbie.
Re:Don’t Slurp: How to Read Files in Python
by rlwolfcastle on Friday September 27, 2013 @11:15PM
(#6460745)
1. Get it to work
2. Test that it works
3. Optimize (if needed)
If your file is small enough and is a trivial percentage of the overall run time, who cares how it is read?
And besides that, the author's solution is just good for one situation: when you want to read something line-by-line, which isn't always the case. For binary files, you may want to do something like this (untested code):
Also, it seems like the author hasn't heard of enumerate: http://docs.python.org/2/library/functions.html#enumerate