When using a for-loop over an iterator, the iterator protocol in Python says to keep returning elements until you run out, at which point you throw an exception. So every loop over an iterator or iterable object in python throws an exception when it is done. https://docs.python.org/3/library/stdtypes.html#iterator.__n...

While that's true at the Python language level, there are already special optimizations for this in CPython: `tp_iternext` is not required to set an exception. If it returns NULL without setting an exception, that's taken to be the end of iteration.

If you call `next()` in Python, this special case is translated to a `StopIteration` exception. But if you use a for-loop, it can directly stop iterating without ever materializing the `StopIteration` exception. So the overhead of Python raise/try-except is already irrelevant for the for-loop.

TIL, thanks for explaining, indeed would have expected exceptions to be pretty cheap also if I knew this.

This covers it very well: https://devblogs.microsoft.com/python/idiomatic-python-eafp-...

In particular, this is not idiomatic python:

    if "key" in dict_:
        value += dict_["key"]

But this is:

    try:
        value += dict_["key"]
    except KeyError:
        pass

I too hate using the exception handling in this way, and if you aren't careful, you end up papering over other unexpected exceptions in your code, so you have to be (A) very specific in the exception you catch, and (B) keep it in as small a portion of code as possible.

I just think it makes for clumsy code - which of the two look better:

    try:
        value = dict_["key"]
    except KeyError:
        pass
    else:
        do_something(value)

OR

    if "key" in dict_:
        do_something(dict["key"])

But it might just be me.

The difference in your "which of the two look better" example is:

(a) has one dict operation plus an exception which rarely occurs and is nearly zero cost if it doesn't.

versus

(b) has nearly always two dict operations, plus a possibly incorrect assumption that the dict will not be mutated between the "if key in dict" and "dict[key]" operations.

Doesn't Python pretty much guarantee that since it's single-threaded?

No, you can have multiple threads in a python app. And while it promises to keep many things atomic, code like that can be interrupted between each line at minium.

If you don't care about a key existing, then this works.

    do_something(dict_.get("key", None))

I use that a lot for data parsing. Passing the exception is not very clean IMHO. I stick to d[key] nomenclature when I need assurance that all the keys are present in the dictionary, and .get(key,None) when I don't.

.get("key") is enough, as the default is already None.

And if you care about the default value being a particular type, when there may also be None in the input stream, do something like:

x.get("key") or []

or

str(x.get("key") or "") # Guarantee strings and avoid "None"!

The second 'key in dict' option is not thread safe if dict_ is not local (another thread may delete the entry between the existence check and use). This is a good reason why the exception handling approach is a better idiom.

The exception handling approach may also be faster if your code cares (may, because I don't know if the cost of calculating the hash of "key" twice is cheaper than the exception handling overhead).

If I find if's and try's looking ugly for a particular use case I try to figure out how to get rid of them. For your first example I would do this, assuming value is a number.

  value += dict_.get('key',0)

Though I agree with using an if in the second example, if there isn't a better way to do the iteration to avoid looking up keys that don't exist.

> In particular, this is not idiomatic python

According to this article, I think his case is rather weak. The conclusion does not seem to follow from the premise to me.

Nearly all loops are terminated by raising an exception.

https://docs.python.org/3/library/exceptions.html#StopIterat...

The first that came to mind is how `get()` is handled in Django's ORM. The idiomatic way to look for a single object is to use `get`, then catch a `DoesNotExist` exception:

From https://docs.djangoproject.com/en/3.2/ref/models/querysets/#...

  from django.core.exceptions import ObjectDoesNotExist

  try:
      blog = Blog.objects.get(id=1)
      entry = Entry.objects.get(blog=blog, entry_number=1)
  except ObjectDoesNotExist:
      print("Either the blog or entry doesn't exist.")

Right, but the better way to actually write this is something like

  entry = Entry.objects.filter(blog__id=1, 
  entry_number=1).first()
  if entry is None:
    # deal with does not exist

Maybe it's my scala/Java background shining through, but we are big Django users and we ban the "catch exceptions as standard" workflow, because there is almost always a cleaner way...

This fails to raise an error if there is more than one object matching the given filters

Presumably entry_number is unique_together with blog_id. Otherwise the original code is also not handling the MultipleObjectsReturned exception.

Generally speaking, I tend toward the cleanest code being:

    blog = Blog.objects.get(id=1)
    entry = blog.entry_set.filter(entry_number=1).first()
    if entry is None:
        handle_missing_entry()
    handle_entry(entry)

But it does suffer from having the extra DB query in there, which may or may not be helpful, depending on the surrounding code (and whether or not you'll be using the blog instance anywhere else).

> Right, but the better way to actually write this is something like

Maybe for some cases, but it does not do the exact same thing, see the comment here: https://stackoverflow.com/a/29455777/1598080

And if we are talking about idiomatic, I think it is maybe a stretch to count this as idiomatic for Python, but given it is documented for Django I think it is fair to call it idiomatic for Django.

I wouldn't make it an if statement unless it's going to be a part of the standard flow. I think the catchphrase is "leap before you look". Though your right that a single query is better.

Honestly, I normally just use get and let the exception fly. If it's a celery task I'll see the stack trace in flower, or right in the output if it's dev with debug on. Then I would go out of my way to make sure there was never a circumstance where a user requests something that doesn't exist.

That is the sort of method that I prefer if it's available. I'm a C# dev so this way also feels far cleaner to me.

What are you suggesting is better about that way? More readable?

In python, it's normal to use exceptions in place of type checking, e.g. in polymorphic functions.

But that would not be for control flow.

On a smaller scale, it is.

To me sending wrong arguments is an error condition, not control flow, see this for more info: https://softwareengineering.stackexchange.com/questions/1892...

It's not about sending the wrong arguments. In Python a function may support a variety of types in a single argument. But instead of querying the type and switching based on that, you often want to try to call a method and if you get an exception (because that method does not exist), try to call a different method that gets you what you need.

The advantage to this over checking the type is that you are still use duck typing. If you check the type then you can only support a specific set of classes.

Thanks for the clarification, I have actually seen this now that I think of it and this indeed is closer to control flow than error flow.

This is also known in Python as the "ask for forgiveness not permission" idiom.