News for hackers. Stuff that matters.
Re:What's New in Python 3.12 by NoboruWataya on Wednesday October 18, 2023 @01:39PM (#37932035)
I suspect people will use these comments to share their wishlist of Python features, so let me add that I really wish Python had a safe navigation operator, for when you are dealing with nested objects that could be None. I have been trying to parse a lot of XML and JSON in Python lately and a feature like that could really help reduce boilerplate checks.

https://en.wikipedia.org/wiki/Safe_navigation_operator

Re:What's New in Python 3.12 by nullwarp on Wednesday October 18, 2023 @01:40PM (#37932059)
While not as short as a proper operator .get("key", {}) does work.
Re:What's New in Python 3.12 by woolion on Wednesday October 18, 2023 @01:55PM (#37932262)
Of course this only works at one level and not arbitrarily deeply, but you still need to check for None with this code; it may actually be better to write `x.get("key") or {}` so that you always get an empty dict.

I write 'may' because the difference between None and an empty dict may be very subtle and rarely does the API specify with enough precision what are supposed to be the semantics of each case.

Re:What's New in Python 3.12 by ddejohn on Wednesday October 18, 2023 @02:15PM (#37932523)
> Of course this only works at one level and not arbitrarily deeply

It does though, you can chain as many of these as you want:

    some_dict.get("level_1_key", {}).get("level_2_key", {}).get("level_3_key", {})...
edit:

>> but you still need to check for None with this code

> Not sure what you mean here. You only have to check for None if you use `.get("key")` and don't provide a fallback value.

GP was talking about `{"foo": None}` and trying to drill deeper, which I misunderstood. Still, a simple try/except allows you to short-circuit the deeply nested access.

Re:What's New in Python 3.12 by smilliken on Wednesday October 18, 2023 @02:21PM (#37932595)
The issue is if the dictionary has a key defined and the value is None. Your get expression will return None, causing your next access to raise an error.
Re:What's New in Python 3.12 by ddejohn on Wednesday October 18, 2023 @02:32PM (#37932731)
Ah, I see. Still, I'd just catch that exception and move on because it's obvious you won't have anything more deeply nested anyway. Blowing up on `None` is an easy short-circuit.
Re:What's New in Python 3.12 by kh_hk on Wednesday October 18, 2023 @02:27PM (#37932668)

    >>> foo = {"foo": None}
    >>> print(foo.get('foo', {}).get('bar'))
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    AttributeError: 'NoneType' object has no attribute 'get'
    >>> print((foo.get('foo') or {}).get('bar'))
    None
Re:What's New in Python 3.12 by ddejohn on Wednesday October 18, 2023 @02:33PM (#37932751)
Right, see my other comment:

> Blowing up on `None` is an easy short-circuit.

The thing being discussed is attempting to access deeply nested values, so short-circuiting here is a win-win. I.e., you wouldn't want to unnecessarily traverse tons of empty dictionaries using the `.get() or {}` trick.

Re:What's New in Python 3.12 by JimDabell on Wednesday October 18, 2023 @02:32PM (#37932738)
> x.get("key") or {}

This has the side effect of replacing falsy values like False or 0 with an empty dictionary instead of giving you the actual value. For the exact intended behaviour, you do need to explicitly check for None instead of using a short circuit trick with or.

Re:What's New in Python 3.12 by ddejohn on Wednesday October 18, 2023 @02:39PM (#37932857)
Yeah, when trying to drill down into deeply nested structures, it'd be a waste to just blindly nullish-coalesce any None objects into empty dicts instead of just immediately short-circuiting out of the deep access. It's a cute trick but would not pass code review in my shop.
Re:What's New in Python 3.12 by woolion on Wednesday October 18, 2023 @05:38PM (#37935125)
The problem was underspecified, but I suggested that under the assumption that the value would be an Optional[dict]. If you have another falsy value, it means either Optional[Any] or something like Optional[Union[dict, list]] (I put list as an example, but you get the idea).

I would say the only reasonable choice is Optional[dict], in which case it should be sufficient, but Python being Python, it could be anything. And in these cases you need to handle all cases way more carefully.

Re:What's New in Python 3.12 by ivoflipse on Wednesday October 18, 2023 @01:53PM (#37932231)
You can participate in the latest iteration of discussing this feature here: https://discuss.python.org/t/introducing-a-safe-navigation-o...
Re:What's New in Python 3.12 by GolDDranks on Wednesday October 18, 2023 @01:56PM (#37932273)
Python, prouding (← is that a word?笑) itself to be the foremost language of duck typing, I'd really want to see that too.
Re:What's New in Python 3.12 by fritzo on Wednesday October 18, 2023 @02:01PM (#37932339)
'priding' is idiomatic English
Re:What's New in Python 3.12 by tipsytoad on Wednesday October 18, 2023 @02:05PM (#37932386)
Priding ;)
Re:What's New in Python 3.12 by esafak on Wednesday October 18, 2023 @04:31PM (#37934381)
touting
Re:What's New in Python 3.12 by cameroncairns on Wednesday October 18, 2023 @02:01PM (#37932330)
The syntax has a bit of a learning curve, but there's a library called glom that can be helpful in parsing heterogeneous JSON. Specifically it has a coalesce operator that let's you define multiple paths to get the data you are trying to get at, with a default value if none of the paths are valid.

https://glom.readthedocs.io/en/latest/api.html#defaults-with...

e.g.

    target = [{"a": {"b": "c"}}, {"a": {"c": "e"}}, {"a": {}}]
    results = glom(target, [Coalesce("a.b", "a.c", default="")]) # -> ["c", "e", ""]
Re:What's New in Python 3.12 by ricree on Wednesday October 18, 2023 @02:20PM (#37932575)
I believe PEP505 should cover what you're asking, but from what I can tell it seems stalled out.

https://peps.python.org/pep-0505/

Re:What's New in Python 3.12 by wodenokoto on Wednesday October 18, 2023 @02:29PM (#37932707)

    def safe_navigation(obj, path: List[str], default=None):
        try:
            for key in path:
                obj = obj[key]
            return obj
        except KeyError:
            return default 
Yeah. I thought it would just be a try-except, but that turned out pretty ugly.

But really, how do you know if your path is missing or the object at the end of the path is None, without using exceptions for communicating this?

Re:What's New in Python 3.12 by o11c on Wednesday October 18, 2023 @02:47PM (#37932969)
I would write:

    def safe_navigation(obj: Any, *path: str, default: Any = None, strict: bool = False) -> Any:
        sentinel = object() if strict else None
        for key in path:
            obj = obj.get(key, sentinel)
            if obj is sentinel:
                return default
        return obj
The only new disadvantage of this is that it only works on mappings and no longer works on sequences.

But I really don't like having this much `Any`; it's generally a sign of poor design.

Re:What's New in Python 3.12 by ddejohn on Wednesday October 18, 2023 @06:39PM (#37935753)
I wouldn't call a function that mutates the input "safe navigation", personally :)
Re:What's New in Python 3.12 by o11c on Wednesday October 18, 2023 @08:26PM (#37936678)
Er, it doesn't? `.get()` (or `[]` in the original) doesn't change the object, and `obj =` only changes which object the local variable refers to?

This isn't like C++ with its intrusive `=`.

Re:What's New in Python 3.12 by ddejohn on Thursday October 19, 2023 @05:14PM (#37948813)
You're right. Python is pass by reference and it's the reference being changed, not the object. This is what happens when you read code on the internet after drinks.
Re:What's New in Python 3.12 by ddejohn on Wednesday October 18, 2023 @06:38PM (#37935742)
There's certainly humor in a function called "safe navigation" mutating the data structure I'm supposedly safely navigating through :D
Re:What's New in Python 3.12 by dmr_92 on Wednesday October 18, 2023 @09:31PM (#37937137)
I don't think this is correct . The name obj gets rebound, but the dictionary being traversed isn't mutated in any way.
Re:What's New in Python 3.12 by Austizzle on Thursday October 19, 2023 @12:47AM (#37938569)
Yeah I agree, no mutation here as far as I can tell
Re:What's New in Python 3.12 by ddejohn on Thursday October 19, 2023 @05:14PM (#37948820)
Correct. I was drunk.
Re:What's New in Python 3.12 by wodenokoto on Thursday October 19, 2023 @02:28AM (#37939237)
I did type it on my phone, so I’m sure it needs a little correction (particularly the choice of exceptions is not optimal, I think) but feel free to run it in any recent python interpreter and return back with any mutation you see.
Re:What's New in Python 3.12 by wodenokoto on Thursday October 19, 2023 @04:35AM (#37940128)

    $ python3
    Python 3.11.6 (main, Oct  2 2023, 20:46:14) [Clang 14.0.3 (clang-1403.0.22.14.1)] on darwin
    Type "help", "copyright", "credits" or "license" for more information.
    >>> from typing import List
    >>> def safe_navigation(obj, path: List[str], default=None):
    ...     try:
    ...         for key in path:
    ...             obj = obj[key]
    ...         return obj
    ...     except KeyError:
    ...         return default 
    ... 
    >>> obj = {1: {1: {1: 2}}}
    >>> safe_navigation(obj, [1,1,1])
    2
    >>> obj
    {1: {1: {1: 2}}}
Re:What's New in Python 3.12 by EdwardDiego on Wednesday October 18, 2023 @02:33PM (#37932753)
I use Pydantic for this for JSON.
Re:What's New in Python 3.12 by graton on Wednesday October 18, 2023 @02:43PM (#37932916)
For JSON there is the `jmespath` library which might help.

https://github.com/jmespath/jmespath.py

    print(jmespath.search(
        expression="some.deep.nested.value",
        data={"some": {"deep": {"nested": {"value": 2}}}},
    ))
    prints 2

    print(jmespath.search(
        expression="some.deep.nested.value2",
        data={"some": {"deep": {"nested": {"value": 2}}}},
    ))
    prints None
Re:What's New in Python 3.12 by ziml77 on Wednesday October 18, 2023 @03:13PM (#37933341)
That's something I've wished for a long time that it had. I know you can use exceptions to get similar behavior but it's really ugly and I just generally don't like raising exceptions for cases that aren't actually exceptional (even though I know Python uses exceptions for normal flow with StopIteration).

What's actually a bit odd about Python missing that operator is that its `or` operator acts as a rough equivalent of C#'s null-coalescing (`??`) operator. To me the safe navigation operator goes hand-in-hand with that.

Re:What's New in Python 3.12 by dragonwriter on Wednesday October 18, 2023 @05:06PM (#37934772)
> What's actually a bit odd about Python missing that operator is that its `or` operator acts as a rough equivalent of C#'s null-coalescing (`??`) operator.

Python `or` is C# `||` plus type coercion. It is not even roughly `??` because important non-None Python values are falsey (0, False, empty collections).

Re:What's New in Python 3.12 by ziml77 on Wednesday October 18, 2023 @05:29PM (#37935034)

    > python3 -c 'print(None or "hello")'
    hello
It evaluates to the left if that's truthy otherwise it evaluates to the right, similar to how `??` evaluates to the left if that's non-null otherwise the right.

In C# even if `||` automatically coerced the types, the output will always be a bool.

Re:What's New in Python 3.12 by movpasd on Wednesday October 18, 2023 @03:25PM (#37933518)
It's been stuck in discussions for a long time, unfortunately. The main reason that I remember gleaning from discussions is it's unclear just how "special" None should be treated as, and whether having special syntax for it is excessively "special".

I saw some suggestions for overridable behaviour in the data model for None-like objects, but didn't scrutinise them closely. Either way, it seems controversial. On the one hand, the ability to override operators is really key to Python's design, but on the other it could lead to rather surprising behaviour from overeager overriding. (But to be fair, this is true of other, more common operators too.)

So, for now we're stuck with `if _ is not None else`, I suppose!

Re:What's New in Python 3.12 by rightbyte on Wednesday October 18, 2023 @03:44PM (#37933775)
On a library level you could return an "empty" object which you could call methods on, returning empty objects and so on, for chaining.
Re:What's New in Python 3.12 by turboponyy on Friday October 20, 2023 @06:22AM (#37954337)
It seems that every language, given enough time, wants to tend towards Haskell!
Re:What's New in Python 3.12 by nonameiguess on Wednesday October 18, 2023 @04:45PM (#37934533)
I haven't even used Python for like four years now, but all of these suggestions to do try except and get.key() or {} seem to be ignoring that Python has had a defaultdict in the standard library for, I don't even know, at least the last decade and a half, as long as I've been paying attention. It's even written in C: https://github.com/python/cpython/blob/d9246c7b734b8958da034....
Re:What's New in Python 3.12 by rwestergren on Thursday October 19, 2023 @06:18AM (#37940816)
defaultdict doesn't seem to easily solve for the main topic of deeply nested values.
Re:What's New in Python 3.12 by malodyets on Wednesday October 18, 2023 @04:48PM (#37934566)
I created a "NoDict" class that does this awhile back: https://github.com/BlackEarth/bl/blob/main/bl/no.py