Re:Don’t Slurp: How to Read Files in Python
by icebraining on Friday September 27, 2013 @06:23PM
(#6459956)
Nitpick: it's better to do "with open('file') as file: for
line in file: ..." instead, but otherwise yes, iterating over file objects is great.
Another option is using mmap[1], particularly when the file is already in memory or you need more random access to it. It worked well when I was trying to parse some lines from the end of an open log file.
Re:Don’t Slurp: How to Read Files in Python
by richardjs on Friday September 27, 2013 @07:14PM
(#6460132)
Is there a difference between the two? I assumed that without the "with", the file would still be closed once the loop was exited (and thus the reference to the file is dropped), but I'm open to the possibility that I'm mistaken.
Re:Don’t Slurp: How to Read Files in Python
by tjgq on Friday September 27, 2013 @07:34PM
(#6460216)
Without the context manager (with statement), the underlying file is closed when the file object is garbage-collected.
In CPython, since reference counting is used for GC, this occurs when the loop exits. However, other implementations (e.g. PyPy) may use different schemes that do not guarantee collection as soon as objects go out of scope. As an extreme, a valid and occasionally useful GC strategy is to never collect anything at all [0]!
Hence, if you want to portably ensure the file is closed, you should either use the context manager or call close() explicitly.
[0] http://blogs.msdn.com/b/oldnewthing/archive/2010/08/09/10047...
Relying on GC to close the file isn't safe. See http://stackoverflow.com/questions/1834556/does-a-file-objec...
Edit: The idiom's mentioned in the Python docs on IO [1] as "memory efficient, fast, and leads to simple code"
[1] http://docs.python.org/2/tutorial/inputoutput.html#methods-o...