Advanced Python Mastery – A Course by David Beazley

Re:Advanced Python Mastery – A Course by David Beazley by PennRobotics on Wednesday July 19, 2023 @12:10PM (#36788726)

It's not better than a generator, but I'm surprised nobody has mentioned the very terse and still mostly readable

    header, *records = [row.strip().split(',') for row in open(filename).readlines()]

but then you need a way to parse the records, which could be Template() from the string library or something like...

    type_record = lambda r : (r[0], int(r[1]), float(r[2]))

At this point, the two no longer mesh well, unless you would be able to unpack into a function/generator/lambda rather than into a variable. (I don't know but my naive attempts and quick SO search were unfruitful.) Also, you're potentially giving up benefits of the CSV reader. Plus, as others have clarified, brevity does not equal readability or relative lack of bugs:

In the course example, it's reasonably easy to add some try blocks/error handling/default values while assigning records, giving you the chance to salvage valid rows without affecting speed or readability. In fact, error handling would be a necessity if that CSV file is externally accessible. Contrast that with my two lines, where there's not an elegant way to handle a bad row or escaped comma or missing file or virtually any other surprise.

Anything else I can think of off-hand (defaultdict, UserList, "if not portfolio:") has the same initialization step, endures some performance degradation, is more fragile, and/or is needlessly unreadable, like this lump of coal:

    portfolio = [record] if 'portfolio' not in globals() else portfolio + [record]

So... your technique and generators. Those are safe-ish, readable, relatively concise, etc.

[ Parent | on HN ]

Re:Advanced Python Mastery – A Course by David Beazley by CogitoCogito on Wednesday July 19, 2023 @01:14PM (#36789857)

> It's not better than a generator, but I'm surprised nobody has mentioned the very terse and still mostly readable

> header, *records = [row.strip().split(',') for row in open(filename).readlines()]

Better would be:

    header, *records = [row.strip().split(',') for row in open(filename)]

No need to read the lines all into memory first.

Edit: Also if you want to be explicit with the file closing, you could do something like:

    with open(filename) as infile:
        header, *records = [row.strip().split(',') for row in infile]

That is if we wanted to protect against future changes to semantics for garbage collection/reference counting. I always do this, but I kind of doubt it will ever really matter in any code I write.

[ Parent | on HN ]

Re:Advanced Python Mastery – A Course by David Beazley by philsnow on Wednesday July 19, 2023 @03:06PM (#36791592)

> No need to read the lines all into memory first.

It looks like that code does read the whole file:

(with a foo.csv that is 350955 bytes long:)

  % python -V
  Python 3.11.4
  % python
  >>> f = open("foo.csv")
  >>> f.tell()
  0
  >>> header, *records = [row.strip().split(',') for row in f]
  >>> f.tell()
  350955

I thought that using a list comprehension to bind header and records was eagerly consuming the file, so I changed it to a generator comprehension with

  >>> f.close()
  >>> f.open("foo.csv")
  >>> header, *records = (row.strip().split(',') for row in f)
  >>> f.tell()
  350955

nope, I guess the destructuring bind does it?

  >>> f.close()
  >>> f.open("foo.csv")
  >>> headers, records = f.readline().strip().split(','), (row.strip().split(',') for row in f)
  >>> f.tell()
  125

not as neat, though. Is there a golf-ier way to do it?*

[ Parent | on HN ]

Re:Advanced Python Mastery – A Course by David Beazley by me-vs-cat on Wednesday July 19, 2023 @08:21PM (#36795392)

The parent poster was pointing out that this requires having two in-memory complete copies of the file:

    [... for row in open(filename).readlines()]

The readlines return value is one copy, and the list comprehension is another copy. However, that first copy can be avoided with:

    [... for row in open(filename)]

The entire file must still be read to evaluate the list comprehension.

Additionally, this doesn't do what you think it does:

    >>> header, *records = (row.strip().split(',') for row in f)

Compare to this, using a variable for clarity:

    >>> gen = (row.strip().split(',') for row in f)
    >>> header, *records = next(gen)

[ Parent | on HN ]