Re:Let's Write an LLVM Specializer for Python
by halflings on Thursday January 22, 2015 @09:42AM
(#8929330)
The example you gave works perfectly in Python 2.7 (would also be a generator, and you're not using map filter or else); but I agree: those should've been generators from day 1, especially zip and enumerate since they make more elegant code but often come with a performance overhead in Python 2.7
Re:Let's Write an LLVM Specializer for Python
by maxerickson on Thursday January 22, 2015 @09:58AM
(#8929424)
Python 2 enumerate returns an 'enumerate' object that is more or less a light weight wrapper of the sequence that was passed in.
Generators provide a convenient syntax to implement that sort of object.
Re:Let's Write an LLVM Specializer for Python
by exDM69 on Thursday January 22, 2015 @10:06AM
(#8929483)
D'oh. I put in a map(), that returns a list.
Re:Let's Write an LLVM Specializer for Python
by danohuiginn on Thursday January 22, 2015 @09:56AM
(#8929410)
Turning them into generators would have broken a lot of existing code, though, so it's reasonable to leave them until a major version change.
Rather, these are the kind of small, obviously-useful changes that should have come immediately in 3.0, giving people some encouragement to switch.
Re:Let's Write an LLVM Specializer for Python
by exDM69 on Thursday January 22, 2015 @10:06AM
(#8929487)
Most of the breaking changes (including making map, zip, filter, etc lazy) were done in Python 3.0.
Re:Let's Write an LLVM Specializer for Python
by exDM69 on Thursday January 22, 2015 @09:52AM
(#8929386)
Ah, you're absolutely right. for line in open(filename) returns a generator. I added map(str.upper, xxx) there to make it more it make sense.
But the point should be obvious, without generators that would potentially consume a lot of memory.
Re:Let's Write an LLVM Specializer for Python
by danohuiginn on Thursday January 22, 2015 @09:45AM
(#8929355)
(process(line) for line in open('giantfile.txt') if line.lstrip()[0] != '#')
Re:Let's Write an LLVM Specializer for Python
by exDM69 on Thursday January 22, 2015 @09:53AM
(#8929395)
No, it wasn't. I added map(str.upper, xxx), now it does.
Re:Let's Write an LLVM Specializer for Python
by masklinn on Thursday January 22, 2015 @12:56PM
(#8930559)
Then again a lazy map is just an import away in P2. It removes pitfalls from Python but the improvement is⦠very limited (as opposed to e.g. `yield from` which is a big convenience, or for much more specialised uses the ellipsis literal being universal)
Re:Let's Write an LLVM Specializer for Python
by dalke on Thursday January 22, 2015 @08:19PM
(#8932890)
(process(line.upper()) for line in open('giantfile.txt') if line.lstrip()[0] != '#')
Re:Let's Write an LLVM Specializer for Python
by exDM69 on Friday January 23, 2015 @02:54AM
(#8933843)
That's besides the point. It's trivial to write that as a for loop or in a million different ways to avoid the issue. It's a contrived example written to demonstrate a difference.
Here's another one you can't change that easily:
tests_pass = all(process(input) == output for (input, output) in zip(open('inputs.txt'), open('outputs.txt'))
Re:Let's Write an LLVM Specializer for Python
by andreasvc on Friday January 23, 2015 @08:35AM
(#8934712)
You can change that easily, with izip from itertools.
The fact that a bunch of builtins and the values/items methods of dictionaries have become iterators is not very siginificant IMHO. Python 2 code could already be written to use iterators or generator expressions, so in the parts where it was crucial it was already done. In this regard Python 3 has not added new functionality but only changed defaults.
The unicode change is the big one.
Re:Let's Write an LLVM Specializer for Python
by dalke on Friday January 23, 2015 @03:33PM
(#8937093)
In this case (checking that process(input) == output), itertools.izip_longest is probably that right solution, unless there's an out-of-band way to know that inputs.txt is the same length as outputs.txt.
Re:Let's Write an LLVM Specializer for Python
by dalke on Friday January 23, 2015 @03:07PM
(#8936947)
You felt the need to correct yourself earlier, so I think your "besides the point" should be directed to yourself. I was pointing out that your correction wasn't persuasive.
Re:Let's Write an LLVM Specializer for Python
by smazga on Thursday January 22, 2015 @11:51AM
(#8930144)
We went to Python 3 for the multiprocessing module. At the time it was 3.3, but now 3.4 has all that async magic. I wish I still worked on that project.
Re:Let's Write an LLVM Specializer for Python
by chrisheller on Thursday January 22, 2015 @12:40PM
(#8930452)
You'll get an IndexError exception on that if there are any blank lines in the file.
Changing that to line.lstrip().startswith('#') would be an alternate approach.
To me, the killer feature is better lazy evaluation (generators). In particular, important builtins like map, filter, zip, enumerate, etc are generators, instead of returning lists. This makes it feasible to write things like
Some of the above can also be done with itertools package in Python 2, but not everything.Python 3.4 changelog is here, it contains e.g. asynchronous io facilities (asyncio module): https://www.python.org/downloads/release/python-342/
edit: added enumerate() in the example above, for line in open(filename) returns a generator in Python 2.x too.
edit2: enumerate is lazy in python2, I replaced it with map(str.upper)