News for hackers. Stuff that matters.
Re:Widely useful Python utilities by Peter Norvig by madars on Monday September 14, 2009 @03:21PM (#822403)
Newer version of these can be found on AIMA-Python site, which is an excellent read alone: http://code.google.com/p/aima-python/source/browse/trunk/uti...
Re:Widely useful Python utilities by Peter Norvig by masklinn on Monday September 14, 2009 @03:54PM (#822477)
Note that more of them have been merged to recent Pythons than just the top: the `dict` constructor can now take kwargs (and a first dict argument, which means you can create an extended copy of dict d with `dict(d, foo='bar', baz='buzz')`), there's a `collections.defaultdict` in 2.5, `namedtuple` in 2.6 is pretty similar to his Struct (though not exactly, `namedtuple` is a type builder, Norvig's `struct` is basically a dict with args instead of items).

`every` and `some` can be emulated by composing `map` (or a listcomp) with the builtin `any` and `all` (which, for some reason, don't take an optional predicate)

Re:Widely useful Python utilities by Peter Norvig by smanek on Monday September 14, 2009 @04:12PM (#822511)
`every` and `some` can be emulated by composing `map` (or a listcomp) with the builtin `any` and `all` (which, for some reason, don't take an optional predicate)

No they can't - this version wouldn't short circuit evaluation, which is completely the wrong behavior. Map would have to apply the predicate to every element of the list before any/all kicked in.

As a trivial example, consider I had a million item list and want to find out if every element is prime. If 8 is the first element, 'every' would return false after looking at the first item - while your composition would apply the prime predicate to all million elements (which would be ridiculously slow ...) before even starting to apply 'all' (which would short circuit, after seeing the first false).

Re:Widely useful Python utilities by Peter Norvig by knome on Monday September 14, 2009 @04:41PM (#822569)
If you have enough elements to desire short circuit evaluation you can, as `masklinn` mentioned, use a list comprehension. Specifically a generator list comprehension.

http://www.python.org/dev/peps/pep-0289/

    big_list = make_big_list()
    if any( not prime( a ) for a in big_list ) :
      do_thing()
When you surround the list comprehension syntax with parenthesis instead of square brackets, either as a lone argument to a function or independently, you create a generator that will iterate once over the given data.

This allows for functions accepting sequences to short circuit without completing the function mapping.

Re:Widely useful Python utilities by Peter Norvig by masklinn on Monday September 14, 2009 @05:12PM (#822630)
> No they can't - this version wouldn't short circuit evaluation, which is completely the wrong behavior. Map would have to apply the predicate to every element of the list before any/all kicked in.

Just use the lazy map itertools.imap (or Python 3, where `map` is lazy) or replace listcomps by gencomps, any & all take any iterable if you're worried about that. Stop creating problems where there is none.

Re:Widely useful Python utilities by Peter Norvig by mccutchen on Monday September 14, 2009 @05:56PM (#822696)
"(which, for some reason, don't take an optional predicate)"

This kills me pretty much every time I use those functions. It seems like 9 times out of 10, I need the ability to pass in a predicate function.

This always struck me as a major oversight... does anyone know if this design decision was made for a particular reason?

Re:Widely useful Python utilities by Peter Norvig by kragen on Monday September 14, 2009 @06:12PM (#822718)
You want to be able to say all(items, lambda x: x.is_cool()) instead of all(item.is_cool() for item in items)? The second one seems clearer to me and isn't substantially longer.
Re:Widely useful Python utilities by Peter Norvig by mccutchen on Monday September 14, 2009 @06:45PM (#822788)
Good point. I guess usually crops up when I've already got a predicate function handy and I want to say `all(items, f)`. I always just expect that to work for some reason. Using a generator expression works just as well, though, and is probably clearer in the long run.

Besides, I guess if `any` had a signature like

    def any(xs, pred=lambda x: bool(x))
or something, that would reverse the signatures of map, filter, reduce, etc.

On that note, I always want the arguments of filter and reduce, but not those of map, to be reversed for some reason.

Re:Widely useful Python utilities by Peter Norvig by euccastro on Monday September 14, 2009 @09:14PM (#823014)
If you already have a predicate function, it is not hard to say

  all(filter(f, items))
Composing calls like this looks cleaner than incorporating into the function interface all list manipulations that the implementor expects you to want.
Re:Widely useful Python utilities by Peter Norvig by mattj on Tuesday September 15, 2009 @02:46AM (#823415)
Said before, but any(pred(x) for x in xs) is too complex?
Re:Widely useful Python utilities by Peter Norvig by mccutchen on Wednesday September 16, 2009 @12:16AM (#825314)
Quoting myself, now: "Using a generator expression works just as well, though, and is probably clearer in the long run."

It's just that my first instinct is always to use all(xs, pred).

Re:Widely useful Python utilities by Peter Norvig by yummyfajitas on Monday September 14, 2009 @03:40PM (#822443)
A useful utility I've rarely seen included in such a list is inverse_image(collection, function) which computes the inverse image of the function restricted to the collection.

A simple example:

    students_by_grade = inverse_image(students, lambda s: s.grade)
Then students_by_gender["A"] contains a list of all the A students (and so on).
Re:Widely useful Python utilities by Peter Norvig by jongraehl on Monday September 14, 2009 @07:26PM (#822871)
Agreed - sometimes known as "group by".
Re:Widely useful Python utilities by Peter Norvig by amackera on Monday September 14, 2009 @03:29PM (#822415)
On a semi-related note, the code for AIMA taught me a lot about Python, just from reading through it at trying to understand.

http://code.google.com/p/aima-python/source/browse/#svn/trun...

Re:Widely useful Python utilities by Peter Norvig by peripatetic on Tuesday September 15, 2009 @12:50PM (#824185)
http://urchin.earth.li/~twic/twic.py

excellent. and great comment: """This is Tom's personal Python subroutine library. You are not expected to understand this.

Warning: this file contains HIGH MAGIC and DEEP VOODOO, as well as quite a bit of QUESTIONABLE CODE and very few COMMENTS. Use with caution.

ha!

Re:Widely useful Python utilities by Peter Norvig by diN0bot on Tuesday September 15, 2009 @04:42PM (#824709)
so....what's excellent about it.... or do you mean you find it useful to use?

i'm personally very cautious about introducing "magic" and "voodoo" into my repositories. i many times have to tweak libraries (add features, fix bugs), so it's worth them making sense without me having to refactor them first (i did this with qunit, tho...it's hard to generalize, eh?).

more importantly, it's important to me that new developers ramp up fast. there's something about collaboration that many lone programmers don't grok. similarly, the code must make sense to me next month. extensibility and maintainability are almost inversely related to voodoo and magic.

good abstraction, handling complexity through elegance, now these are worth having.

it can be quite hard separating good programming from bad. it is even harder when the work itself is complex in nature, although that is when good programming really shines.

Re:Widely useful Python utilities by Peter Norvig by th0ma5 on Monday September 14, 2009 @03:15PM (#822393)
These are pretty good, some common sense or helper like things that I have implemented here or there at one time or another. Granted, some default programming styles may alleviate the need for some of these things, or ultimately you may want something different in your specific application. Good to see!
Re:Widely useful Python utilities by Peter Norvig by arithmetic on Monday September 14, 2009 @03:21PM (#822401)
These are useful! It always helps to take the monotony out of writing code (especially with methods that check boundary conditions and overflows) and spend more time on the actual design.
Re:Widely useful Python utilities by Peter Norvig by spolsky on Monday September 14, 2009 @09:27PM (#823030)
somebody want to explain this?

infinity = 1.0e400

Re:Widely useful Python utilities by Peter Norvig by nearestneighbor on Monday September 14, 2009 @10:57PM (#823180)
the largest representable float in Python is around 10^300 something or so. Anything bigger is considered infinite. Peter Norvig wanted a name for the float bit pattern that represents infinity.
Re:Widely useful Python utilities by Peter Norvig by mattj on Tuesday September 15, 2009 @02:47AM (#823418)
float("inf")?
Re:Widely useful Python utilities by Peter Norvig by beza1e1 on Tuesday September 15, 2009 @05:52AM (#823553)

    >>> 1.0e400 == float("inf")
    True
I don't know, if there is a difference. Maybe the interpreter can evaluate 1.0e400 a little faster?