While I like all these proposed changes individually and PEPs are always well intentioned, my concern is the the Python syntax is getting too “big”.

Pythons massive advantage and the reason it’s so successful is it’s simplicity. Adding syntactic features only adds more surface area for people to learn when coming to the language and barriers to entry for people looking to contribute to an unfamiliar codebase.

There are a few over the last few years I truly love, such as “f strings”, the typing syntax though I’m less convinced by (and I really like TypeScript).

The recently added “walrus operator” is a good example of a syntactic addition that is both brilliant for experienced Python coders but also adds friction to new people learning the language. It’s presence in a codebase could confuse people.

Python is supposed to “favour explicit over implicit” and has always been more “wordy”, symbol based operators that aren’t inherently explicit in their meaning (like this proposed =>) are the type I’m particularly worried about.

Edit:

I just want to add, I don’t think these additions “damage” the language, it’s just the additional friction for people to pick it up or approach an unfamiliar codebase that worries me (maybe unduly). I love Python and these don’t put me off wanting to continue to use it as my primary language. In many ways I look forward to using them as an experienced Python developer.

Python probably passed that point about 10 years ago. I'm not trying to be cynical, I've thought this for a while. At this point, I just wish the best to Python's new features because the language is well past "simple" and it's been many years since that was a thing that was even possible to save.

I've watched new and inexperienced programmers trying to learn full, modern Python up close and personal. (That is, they were trying to get into real code bases that were using the many features in Python, not that we were in a teaching space incrementally introducing things.) Anyone who has done that in the last ten years knows it's not simple anymore.

Personally I consider 3.2 to be the first usable version of Python 3, though that obviously already had a very large and complex data and language model.

New in 3.3:

- yield from: I'm okay with this

- PEP 420 / namespace packages: I think this is/was the best compromise, but this remains a fundamentally ugly topic for something that is sort-of necessary to have

- raise from None: This is okay for library code

- __qualname__: It's a mess. It's not like I'm against having it, but this is a corner of the language that practically invites you to hang yourself one way or another.

- importlib: It's better than the previous iteration but this package was and still is a mess. Again, lots of this is due to legacy aka "the other packaging packages".

New in 3.4

- asyncio beta (no coroutine syntax yet): While coroutines are highly useful, asyncio and Python coroutines simply are not good. This is the beginning of the badness.

- pathlib: Ok!

- enum: Ok!

New in 3.5

- async def, await: This is the release that really set colored functions in stone for Python.

- "@" operator: I've never used this once I think, but that's because I don't write math code in Python. If I did, I think I'd like it, so I'm okay with this.

- type hints: Another oof moment

New in 3.6

- f-strings: Yay!

- dict now ordered: Yay!

- async generators and such: Again, I think colored functions are a big blunder, but if you have them, you need this.

- Variable annotations

New in 3.7

- PEP 563: Type annotations can be turned into quoted annotations without a syntactic change through __future__ import.

- contextvars: "thread-locals for coroutines"

- PEP 562: Added the Python 2 misfeature back where Python 2 interpreted __get__ and __set__ on modules

- Typing becomes part of the core language, I think this is when you could start writing things like "list[int]" (instead of "typing.List[int]").

New in 3.8

- := Yuck

- "Positional-only parameters" because the simplified CPython API for defining functions has this, so ... Python code needs that as well? Wot?

- More typing

New in 3.9

- More typing

New in 3.10

- And even more typing

- Pattern matching

"I'm okay with this"

I pulled this out of your post because it's something I think about a lot of the individual changes. Individually, they generally make sense.

It's just that the sum total of all the individual sensible things is a large mess, getting larger every release.

I first used Python in the 20th century, and have clocked many thousands of hours with it overall. Now I mostly program in Go. (Actually, even after ~7 years part-time professional with Go, I'm not sure which I've used more yet! I may still technically have more time with Python. If I do, though, I'm close to the crossover.) Occasionally, I miss iterators, and with Go getting generics it's probably my #1 ask in Go. Other than that, though... I just can't help but notice that I don't actually miss that many features of Python very often. I miss Haskell features, I sometimes miss Rust features, every once in a while I'll miss something more exotic like an Erlang feature, but I almost never think "gee, I really miss this Python feature". It isn't that I can't see where I'd use them if they exist, I just don't sit there pining for them all the time. I consider this a big data point. I should also point out I'm talking about subconscious inclinations, it's not like I'm setting out and daily disciplining myself to not miss Python features. It's just a thing that happens or doesn't. YMMV. I'm probably more often grateful for a missing Python feature than I'm missing it (e.g., it's actually kinda nice after a certain scale to know what "x.y" does, rather than have to go resolving down an arbitrarily-complicated chain of classes, inheritance, method overrides, etc.).

its quite fun to see python api generated from C/C++ libraries. They always go ham on the type specification, making the resulting code unrecognizable as Python.

>Pythons massive advantage and the reason it’s so successful is it’s simplicity.

Yet, if you use it more than few years, you see how it's just a disguise.

Underneath superficial simplicity there's ton of leaking abstractions, semantics which designers designed themself into the corner, and gigantic C API that blocks any performance improvements.

Yes. And currently it has also split into several sub-languages almost now. Numpy/Scipy is one, then there is enterprise Python, and the ML Python has its own conventions. That's how big and complex Python today is.

Also: there used to be one and best way to do a thing. Now there's multiple ways to do a half-assed job.

There's always been more than one effective way to do things in Python. The contention otherwise is just some rosy thinking. Heck, decorators were introduced almost 20 years ago and they're basically just additional syntax for

def foo():

    return "hi"

foo = my_decorator(foo)

Similarly for context managers and many other features.

We can go back much farther than that even: why have lists and tuples, when they're basically interchangeable in most cases? Why have list comprehensions when a loop does the job just fine and might arguably be clearer?

Could you give some examples?

The walrus operator, the `match` statement, the f-strings etc. these seem like just syntactic sugar. Also note the type hints are optional. So these changes are opt in: you can still write the Python of yesterday. But, if you can afford to take it up a notch, you can write the Python of today. I don't see Python getting harder for the newcomers, instead I find it getting better for the ones who have invested deeply into it, and I quite enjoy it.

> these changes are opt in

Only for personal projects, with no outside contributors, no external dependencies, etc.

If you want to read someone else's code, e.g. to learn more, or see how one of your dependencies works, etc. then you may need to face these features.

If you want to allow external contributions to your project, then you'll have to face these features (even if that just means an informal coding standard that says "don't use XYZ").

If you want to work more closely/collaboratively with others, e.g. in an organisation, then you'll have to face these features (even if it's just a discussion about whether to allow them).

If you want to write tooling for the language, e.g. parsers, doc generators, linters, formatters, diffing tool, etc. then you'll need to face them.

Finally, if you're writing library code that's intended for use by others, you need to be aware of such language features, in case they break any of your assumptions. For example, a library might rely on the fact that lambdas can't re-bind variables (they can mutate the content of an object, but they can't change the reference, since '=' is a statement rather than an expression); the introduction of ':=' expressions violates that assumption.

In the real world, I'm still nagging people to use docstrings and type hints.

It is really not my experience that people read PEPs and try to shoehorn in language tricks they don't need into their random scripts at $DAYJOB.

I would actually be thrilled to see a walrus operator show up in a PR from one of my coworkers. But I'd have to shoot it down because we need to be compatible with 3.6.

What if a newcomer wants to make sense of a large codebase that uses all the new features?

It used to be that Python was mostly similar to pseudocode, easy to understand and reason about. Now all this "syntactic sugar" just makes it harder to read IMO.

I completely agree, one of the advantages of python is that it reads like pseudo-code for people not familiar with the language. Some of the proposed syntax changes are not that intuitive and require knowing how they work beforehand.

There were already a lot of features like that:

- context manager

- decorators

- inline conditions

- else keywords in loops, which nobody knows even exist.

- raise from (empty, or with params).

- yield from (espacially with send())

- comprehension lists

- advanced unpacking. This is valid: (a, (b, *c)) = [1, {*range(4)}]

- advanced slicing. This is valid: foo[2:8:2] = bar[:4]

Most of which existed in Python 2 already !

But it has never been a problem before. Why ?

Because the community doesn't abuse features, it's part of the creed. But also because half of the python coders don't even know they exist (they are just trying to solve their problem, but dev is not their job), and are productive without it, so they just don't use it.

In fact, a lot of tutorials on the internet never even mention those.

Since the walrus came out, I used it once, and never saw it on a client code base.

But when the community do use the features, they usually make things better: decorators made flask/click what it is, slicing made numpy/pandas what it is, type hints made fastapi/pydantic what they are.

Those worries are mostly that: worries. They don't reflect what happens in practice.

You are right. Let's be real, if you're teaching Python to kids and bootcampers, you're going to only cover a little bit of what's in Python 2, and that's fine.

The other stuff is mostly designed by framework authors to by used by framework authors. And having nice libraries and frameworks is what really makes this language so useful for everyone.

It's trendy to claim Python is meant to be some kind of "pure environment" where the source code of Flask or SQLAlchemy or NumPy should just like a tutorial, but Python has never been that in the 10+ years I've been getting paid to use it.

I thought that was the aspiration of Go. As far as I could tell, Hacker News commenters hated Go and now it's mostly ignored and forgotten here.

I wanted to joke about still waiting for COME FROM to be added to Python, then I found it that someone already did that...

while dynlang add more syntax and types, static verbose ones are trimming them off, isn't that curious :)

I do worry Python will head down the path of C++ and turn into a monstrosity.

Can't we ever exercise some discipline and keep things simple in software? More features isn't strictly better, there's a hidden cost that adds up over time.

I'd welcome mere perf / memory improvements, security patches, and ancient feature removals for the next 5 years or so.

Interestingly Go is very disciplined about not adding new features in general - and it gets a lot of criticism for that!

Adding loads of new syntax is easy. Now removing the GIL on the other hand...

Maybe we should flip this around and start removing enhancements that never caught on?

After the backlash to Python 3's breaking changes, I doubt Python will ever do that again, unfortunately.

Well can I ask for interpreter warnings when it detects the new syntax? “Using these features makes your code harder to understand and maintain …”

I actually think either a new PEP-8 or an opinionated linter would be one way to test the waters. Black[1] seems to be gaining popularity by enforcing strong opinions. I remember reading large companies' (like Google's) coding styles.

[1] https://pypi.org/project/black/

you can start by writing a "Python: The Good Parts" book

> Well, functions in Python can take other functions as arguments. There's however no good way to set default value for such arguments

I don't understand what is special about functions? Why can't you set the default value for an argument called `foo` to be a function `bar` by writing `foo=bar`?

There isn't anything special about functions; the original article does not describe this correctly. Rather, the big conceptual difference is the point at which the expression is evaluated – once for the whole program (`=`), vs. at each call site (`=>`).

I presume this got accepted because it fixes a well-known gotcha with default parameters in Python due to early evaluation, where, for instance, the dictionary instance in `def fun(args={}): …` would be shared between all invocations, leading to all sorts of fun bugs. This is especially pernicious as most Python programmers will know other languages as well, where this tends to be handled much more sensibly (e.g. in C++, D, …) and default arguments are evaluated at each call site.

I think this could have been fixed by always evaluating default args on each invocation. There is no reason why a default arg of all things should carry any mutable state. The people who would complain probably already had bugs in their code

This change will enable to have default values that are created at function call, not at function definition.

For example I can define:

  def f(x, l=>len(x)):
      print(l)

where the default value of l depends on the other variable x.

My immediate response to this is: WTF.

Why not just do `l = len(x) if l is None else l`?

With the new syntax can you define defaults computed from other defaults? What about side effects? Why is computation happening in function args at all?!

Having the default value be None isn't great for documentation, and it isn't great for static typing, because now your argument has to be of type Union[Foo, None]. I've written functions where there are reasonably long chains of stuff like

if foo is None:

    foo = []

and it'd be nice to be done with that.

I see what you're saying about typing. As for the reasonably long chains of default initialization code. Well now its in the definition. You could have functions with more code defining the parameters than in body.

LWN had a good article on this: https://lwn.net/Articles/875441/

Short answer is these downsides were discussed but the proposal went ahead anyway.

Imo I'm with you. This type of syntax has a lot of implicit behavior where it's going to be difficult to find/fix bugs related to it.

That was a great article, thank you for the link!

> Why not just do `l = len(x) if l is None else l`?

Because dev tooling can “see” and present information from the argument list much more easily than analyzing downstream behavior.

> What about side effects?

Those happen whether or not the computation is visible in the signature (though its more obvious to the programmer of the consuming code if it is visible.)

I've never really seen arg1 depend on arg0 like that in Python.

However this comes up a lot for new engineers during programming interviews. It's very common to see recursive algos use empty lists as default/start-case args:

    def traverse(graph, seen_so_far=[]):  # Incorrect
        ....
        seen_so_far.append(node)

Subsequent calls to `traverse` will have state accumulated from earlier calls. The right solution is:

    def traverse(graph, seen_so_far=None):
        if not seen_so_far:
            seen_so_far = []
        .....

This isn't obvious if you're new to Python. This PEP makes this (arguably) clearer with:

    def traverse(graph, seen_so_far=>[]):

But how does => make it more obvious to someone who's new to python?

> This PEP makes this (arguably) clearer

It's more concise when you are familiar with it, and it improves introspection and dev tooling when functions use it, but it is not fundamentally clearer.

Oh my, that’s just terrible. Really hoping this pep wont make it all the way. 100% unreadable.

I’m not sure it’s unreadable to an experienced coder, someone learning the language though will be confused about the difference between = and => in the function argument definition.

Also the “arrow” is clearly pointing the wrong way, it should be “<=“.

It adds up. It's right that we have gotten used to all of the other stuff, but it adds up. It's fair to be very critical of new additions.

It's fair to be critical, but calling this flat out awful and unreadable is way overselling it. Whether it helps add up to something less readable than preferable is arguable, but taken on its own this change is minimally invasive and frankly pretty readable.

It seems perfectly readable to me, honestly. It might be confusing to newcomers, but I don't see a problem with it.

The author of the article missed the point entirely. The feature allows default arguments to be set to the _result_ of a function call (that may or may not depend on other arguments).

Not sure I understand what he's going for here. The big issue that that PEP addresses is mutable default arguments, nothing about functions as arguments.

>>> import this

The Zen of Python, by Tim Peters

[...]

There should be one-- and preferably only one --obvious way to do it.

[...]

The usage, and resulting culture, of Python has changed a lot since 1999 when that was published. I think the language and ecosystem could have been improved without abandoning this principle with specific efforts, but I'm not sure those efforts would have been helpful overall. They'd have likely impacted the growth/success of it.

Strict adherence to this principle would not allow the language to evolve and improve.

I've always interpreted this as to establish Python as being antithetical to the Perl idea of having a hundred ways to do something and trying to do it in the most clever way, readability be damned.

Yuck, python is losing its zen and gets uglier and uglier.

And that's assuming it ever had any kind of "zen" in the first place. Its package management system looks like from something from Dante's "Inferno" rather than any kind of Oriental wisdom.

Agreed.

It's weird, I am usually a very late adopter of technology. Being early here has allowed me to watch Python mutate into one of the things I fled from. When I heard someone say that the standard library is where code goes to die, I felt my heart break a little bit.

Most of these changes directly address friction I've experienced. I like them.

Not sure I'm getting why we need a => for a default function argument instead of just reusing the = symbol...

Because that would make the change backwards incompatible. Some code out there could be (ab)using the fact that default arguments are bound at function definition time, and depend on that behavior to stay as it is. So this new behavior must be behind a new syntax.

But said old code is using an older version of Python, which does not support this. The compiler could issue a warning for newer version until the switchover, and therefore save us from this "special case".

Unfortunately, the author used some unnecessary words.

>Function parameters can have default values which are calculated during function definition and saved. This proposal introduces a new form of argument default, defined by an expression to be evaluated at function call time

So it’s not specific to functions as arguments? A long-time gripe I’ve had about default parameters in Python is that they are static from function definition time instead of dynamic at call time. A consequence of that is that if you have a function like

    def foo(bar, baz=[]):
        baz.append(bar)
        return baz

baz will be reused for the duration of the process kind of like a global variable.

If this new syntax fixes that, I see the value.

> If this new syntax fixes that, I see the value.

It does. The old syntax creates and assigns the variable at compile-time. => does it at execution time.

So this code:

    def foo(bar, baz=[]):
        baz.append(bar)
        return baz
    foo(1)
    foo(2)
    print(foo(3))

would print '[1, 2, 3]'

But this code:

    def foo(bar, baz=>[]):
        baz.append(bar)
        return baz
    foo(1)
    foo(2)
    print(foo(3))

would print '[3]'

That's much clearer. Thanks

This blog post does not do a good job of communicating the status of these PEPs, for example the first one "PEP 671" is in Draft status and it has not been accepted by the Steering Council.

In fact I would be surprised if it did get accepted as many members of the Core Dev / Steering Council have similar reservations about extending Python's Syntax as others have expressed in comments here. As demonstrated by their recent rejection of PEP 677 (Callable Type Syntax).

<sarcasm> Yeah, great, let's add more syntax to python. Because a cake always tastes much better, the more ingredients it has. Just ask C++! </sarcasm>

Sorry but did I miss something?

When exactly did "this saves 2 lines in the code!" become reason enough to introduce new syntactic elements?

> When exactly did "this saves 2 lines in the code!" become reason enough to introduce new syntactic elements?

My first thought is decorators. They were added in 2.4, back in 2003: https://www.python.org/dev/peps/pep-0318/

All they are is syntactic sugar for:

    def decorate(func):
        doStuff()
        func()
        return
    myFunc = decorate(myFunc)

List comprehensions are just a syntactic sugar for a for loop with a single line that appends something.

There are plenty of syntactic elements that only exist to save a line or two of code. I much prefer the walrus operator over:

    x = GetNextObject()
    while x:
        doStuff()
        x = GetNextObject()

> List comprehensions are just a syntactic sugar for a for loop with a single line that appends something.

They can be that.

They can also be syntactic sugar obscuring a construct that would be much more readable if it were written as some nested loops and if statements.

Problem with all things that fall under the category of syntactic sugar: The small gain in readbility in some cases is more than offset by the LOSS of readbility when they are overused.

Just as an Example: Do I think Decorators are neat? Sure. When they are used in moderation, they absolutely are. However, when codebases start to have decorators everywhere, use multiple Decorators on functions, with some Decs with Args thrown in for good measure, things start to get tricky. And latest when they are used to do something completely different from their original intent (act as syntactic sugar for wrapper functions), what began as a nice idea, can quickly escalate into a debugging nightmare.

IMO, Language design needs to be much more careful about what is added to a language than what is left out.

> They can also be syntactic sugar obscuring a construct that would be much more readable if it were written as some nested loops and if statements.

For sure. I once wrote a list comprehension that was nearly 200 characters wide. I thought "eh...this is extremely unreadable" and rewrote it into several lines. I figured a rewrite into a few lines of code was the better choice than adding a comment, especially if I decided I needed to change it at some point. There was a performance drop, but it was only about a second, and only when the program was starting up.

Every language feature beyond basic binary operators can be abused, but the answer isn't to remove (or refuse to implement) them, but to smack engineers and tell them a firm "NO!" when they abuse them.

I have yet to see decorators being abused (Not denying it's happened, but I just haven't seen it myself yet), but I've seen someone abuse operator overloading. Someone had decided to write their own TCP socket class, and decided that "sock += 'some data'" was a great way to send data over the socket. The scapy library kind of abuses overloading the / operator in order to build network packets with multiple layers, but I think it works in that case because anything else might actually be harder to read.

Sometimes it even requires MORE lines of code than before, check out the last "improvement" from the article, the sentinels

+1, I find it bizarre that a new language feature is being added to save two lines in a corner case.

God, I would love for there to be a standardization of package management and project structure in python from the top down.

What a great article! Love to read this kind of content where the author clearly has put effort to summarize things.

It's sad to say, but I think I'm over Python. It's not fun or pretty anymore, with all of the versioning headaches, the nightmare pit of dependency management, different runtime versions shipping on different machines. It's like they keep adding all of these features I don't want so it seems more like a 'big boy' language, but without any benefits in runtime performance.

What I really liked Python for was as a _productive_, high velocity language - when I just want to bang an idea or a prototype out. Now there are all these design considerations about what version to target, what language features to use.. blah. I just wanted 2.8+ with sane unicode, and that's it. Grumble grumble.

I’m just starting to like Python. Worked with it off and on for years but Perl was my goto scripting language. I prefer static typing and thought Python was too slow, which it sort of is. Also thought a decade a foot dragging to get everyone to 3.x was too much. Glad that’s over and i don’t have to worry about legacy code.

Now, Python, with Pandas, NumPy, Matplotlib, etc, provide the best “get shit done” language. Throw in Jupyter Lab and i find myself extremely productive. might try type hints next.

It's still really easy to bang out prototypes; you don't need to use all the bells and whistles. But since Python is also at the heart of huge companies, it needs to be able to work well for large, mature codebases, too. These PEPs exist because the community is clamoring for them. This blog post highlights some improvements that directly address friction I've faced

I'm with you. I used to be a huge Python fan boy and I used it professionally for many years. With the 2->3 transition and all the new baloney it seems to me that it's being "improved" to death. For backend web stuff Erlang/Elixer makes way more sense, for scientific stuff Julia makes more sense, for rapid application development (RAD) Nim makes more sense, and with Go and Rust and Zig and D and so on there are just better options out there for most tasks.

I really loved learning Python maybe ~20 years ago. At the time, I had been super into Perl and was also learning a bit about PHP. Python was such a breath of fresh air. I use Python some in my day job, and it just can't get over how painful it is. Far too many production bugs that aren't caught by linting, far too slow processing for big data work.

Despite my dislike, it certainly does excel at being a glue language, hence why all of our data scientists use it with numpy and various ML libraries.

I've been feeling this lately. I have been working on large python code bases the last couple years, and its been a pretty awful from a maintenance perspective. For my next position I want to avoid it.

Switch to Nim and programming will be fun again.

I have been working in Elixir lately and enjoying it quite a lot! Keeping an eye on Nim as well..

Elixir is pretty good too. Elixir, Phoenix and Ecto have the best docs I have ever seen.