News for hackers. Stuff that matters.
Re:What’s New in Python 3.8 by paulddraper on Monday October 14, 2019 @07:38PM (#21254049)
> As a developer who has primarily developed applications in Python for his entire professional career, I can't say I'm especially excited about any of the "headlining" features of 3.8.

Python is a fairly old, mature language.

What features would you have been especially excited about?

Re:What’s New in Python 3.8 by CivBase on Monday October 14, 2019 @08:00PM (#21254200)
The f-strings from 3.6 are a (relatively) recent feature that I have absolutely loved. I'd go so far as to say they are my favorite feature introduced by Python 3.

I'm also looking forward to PEP-554 [0], which allows for "subinterpreters" for running concurrent code without removing the GIL or incurring the overhead of subprocesses.

[0] https://www.python.org/dev/peps/pep-0554/

Re:What’s New in Python 3.8 by hannibalhorn on Tuesday October 15, 2019 @08:09AM (#21257605)
Sounds like PEP-544 is actually included in 3.8 on a "provisional" basis, which I guess just means they reserve the right to change the API. That's a killer feature for me, thanks for pointing it out!
Re:What’s New in Python 3.8 by hannibalhorn on Wednesday October 16, 2019 @09:46AM (#21270246)
Correction: So the PEP said it'll be in 3.8, but apparently it's been postponed to 3.9.
Re:What’s New in Python 3.8 by dgrunwald on Wednesday October 16, 2019 @04:36PM (#21275270)
As someone familiar with the Python C extension API, I somewhat doubt that it'll make 3.9 either. Disclaimer: I'm only familiar with CPython as a user of the extension API; I have no idea on how the Python team plans to address the challenges I'm mentioning here, and what their current progress is.

The current extension module API encourages global state, e.g. types (and objects too) are allocated statically in a global C variable. For example, there is a global C variable `_Py_NoneStruct` that is the Python `None` value, and extension modules are accessing this variable directly.

Every use of this object needs to adjust its reference count, and that reference count is directly stored within the C global variable.

`_Py_NoneStruct` is currently even exposed in the PEP 384 stable ABI. Existing extension module binaries are commonly directly touching `_Py_NoneStruct.ob_refcnt` without any synchronization. Breaking the PEP 384 compatibility promise is fundamentally unavoidable here.

One of two things must happen: Alternative one: All refcount operations must be made atomic for thread-safety. These are really really common in the Python interpreter, but atomic operations are expensive on modern CPUs (especially if there's contention). But multiple threads using the value `None` would be quite common in Python code, so I doubt you'd gain any speed even at today's core counts -- in fact I'd expect the constant inter-CPU-communication for the refcounts to make everything slower than just using a single core with today's Python!

So alternative two, ensure no Python objects are shared between the subinterpreters. That's the plan. But that also means it's a breaking change for extension modules. And it's not just an ABI change (which would be handled by merely recompiling against the new headers). Any extension modules that do not yet support PEP 489 are already incompatible with subinterpreters, so that will take quite a bit of work until the ecosystem is upgraded. But there will probably also be some other breaking API changes. I think type objects are currently shared across subinterpreters, and those are frequently defined as a `Py_TypeObject` global variable in extension module code. Also, if every subinterpreter has its own GIL, extension modules calling `PyGILState_Ensure()` will have to specify which subinterpreter they will be using, so that the appropriate lock can be acquired.

My prediction: 3.9 may have the basic functionality, but it still won't be able to run on multiple cores concurrently. That will take a bunch of more work and breaking changes, and will likely be released as Python 4.0.

There will be another slow upgrade process ("my dependencies must upgrade before I can") until the Python ecosystem is multi-subinterpreter-compatible. But at least this one only affects extension modules.

Re:What’s New in Python 3.8 by enriquto on Tuesday October 15, 2019 @03:11AM (#21256210)
f-strings are great. Much nicer than ".format". I hope in a next iteration of the language all strings will be f-strings by default, avoiding the need to prefix them by a silly "f".
Re:What’s New in Python 3.8 by rolux on Tuesday October 15, 2019 @03:40AM (#21256345)
That would of course be an absolute disaster, given that any user input could easily leak internal state and/or break your program.
Re:What’s New in Python 3.8 by enriquto on Tuesday October 15, 2019 @04:22AM (#21256526)
I meant for inline strings (i.e., typed by the programmer). This is an inoffensive change. The only possible ``accident'' is when the programmer wants to write "{x}" instead of the value of x. This is such and exceptional case that it may be best treated by forcing to escape the curly brackets.

If anything, user-input strings must be treated as tainted whatever the case.

Re:What’s New in Python 3.8 by eesmith on Tuesday October 15, 2019 @06:44AM (#21257214)
A quick look at the Python stdlib gives shows some of the breakage that would occur:

1) existing uses of .format() would break:

   aifc.py: raise Error('marker {0!r} does not exist'.format(id))
2) existing uses of "%" formatting would break:

    argparse.py: result = '{%s}' % ','.join(choice_strs)
3) many regular expressions would break:

    uuid.py: if re.fullmatch('(?:[0-9a-f][0-9a-f]-){5}[0-9a-f][0-9a-f]', value):
4) existing strings which contain uuids would break:

    uuid.py: >>> x = uuid.UUID('{00010203-0405-0607-0809-0a0b0c0d0e0f}')
Re:What’s New in Python 3.8 by ptx on Tuesday October 15, 2019 @07:57AM (#21257555)
Then you would run into the same problem Kotlin has with its raw strings – the brackets need to be escaped, but raw strings disable escaping, so you'd have to write r"{}" as r"{'\{\}'}" which is quite ugly and not very raw at all.
Re:What’s New in Python 3.8 by Grue3 on Tuesday October 15, 2019 @09:16AM (#21257997)
It's not an exceptional use case at all. f-strings are pretty much useless for any non-trivial formatting:

1) customizable templates (can't allow the user to access arbitrary variables)

2) i18n (can't allow the translator to access arbitrary variables)

3) complex values being passed to .format() instead of some variable

Re:What’s New in Python 3.8 by duckerude on Tuesday October 15, 2019 @06:35AM (#21257174)
That would break any existing strings that use curly braces, including most legacy use of str.format and docstrings that include code examples with dictionaries. It would be a big backward compatibility issue.
Re:What’s New in Python 3.8 by BerislavLopac on Tuesday October 15, 2019 @06:02AM (#21257022)
Explicit is better than implicit.
Re:What’s New in Python 3.8 by enriquto on Tuesday October 15, 2019 @06:04AM (#21257031)
Sure, the curly brackets are explicit enough. What do you mean?
Re:What’s New in Python 3.8 by glic3rinu on Tuesday October 15, 2019 @04:17AM (#21256508)
I wouldn’t mind cpython performance improvements over language features. Specifically startup time. Py cli tools can easily take up to 300-600ms to start. Now, try to use them for scripting. A single script doing a bunch of calls to python clis has already several seconds of runtime penalty.
Re:What’s New in Python 3.8 by paulddraper on Tuesday October 15, 2019 @12:44PM (#21260503)
I completely agree, but you'll notice similar startup overheads in almost all non-native languages: Java, Ruby, Node.js.

I wonder if there is anything that can really be done, outside of something like Cython.

Re:What’s New in Python 3.8 by Rotareti on Tuesday October 15, 2019 @04:51PM (#21263591)
> I wonder if there is anything that can really be done, outside of something like Cython.

I compile my cli tools with nuitka [0], the resulting binaries take half the time to start. I find the difference quite notable.

[0]: https://github.com/Nuitka/Nuitka

Re:What’s New in Python 3.8 by glic3rinu on Tuesday October 15, 2019 @05:10PM (#21263821)
The interpreter loads reasonably fast, takes around 20-40ms. And it's even faster if "import site on initialization" is disabled.

But startup time increases an order of magnitude once you start importing 'requests' and other common packages.

Perhaps an option to turn imports lazy...

Re:What’s New in Python 3.8 by levi_n on Monday October 14, 2019 @10:21PM (#21255029)
Though not a language feature per say, I would love if the language would take on the packaging ecosystem and deliver a ground up approach that isn't just a kludge on a kludge on a kludge.
Re:What’s New in Python 3.8 by takeda on Tuesday October 15, 2019 @01:59AM (#21255890)
It's not great, but also it isn't terrible, the problem with Python packaging is that there were many ways to do it and people are confused. What's worse, there are tons of articles (in including from PyPA) that are providing bad information.

If you use setuptools and place all your configuration declaratively in setup.cfg it is not that bad.

Re:What’s New in Python 3.8 by xapata on Monday October 14, 2019 @11:31PM (#21255322)
Unfortunately, that'd mean a package manager for C, Fortran, etc. as well, since so many Python packages have non-Python dependencies.

I've started using Docker for ensuring prod behaves the same as dev. Going from Mac dev to Linux prod would otherwise cause trouble.

Re:What’s New in Python 3.8 by j88439h84 on Tuesday October 15, 2019 @05:45AM (#21256933)
Sounds like Conda.
Re:What’s New in Python 3.8 by xapata on Wednesday October 16, 2019 @04:31PM (#21275193)
Conda can't fix some problems, like Python 3.7 wanting ncurses >6, but a dependency wanting ncurses 5.9
Re:What’s New in Python 3.8 by shaklee3 on Tuesday October 15, 2019 @12:44AM (#21255631)
I've heard people recommend poetry on here as something that is good for package management, but I haven't tried it. Does anyone know of something like that, but also can build your application into a Docker container as well?
Re:What’s New in Python 3.8 by penagwin on Tuesday October 15, 2019 @01:12AM (#21255728)
Into a docker container as well? The package manager isn't really related to docker though?

Does your container expose ports or need volumes? Does it need gunicorn or uwsgi in front of it? What about system packages? None of those (except maybe the last one) are really in the scope of the package manager.

Re:What’s New in Python 3.8 by shaklee3 on Tuesday October 15, 2019 @09:16AM (#21258000)
The main use case is I have a project with a setup.py that has the normal stuff in it, and there's a main entry point in one of the files. I then make a dockerfile that installs that package in a container and runs the main file as the entry point. Ignoring things like ports, it would be nice to emit a standard dockerfile like that, since it's very common.
Re:What’s New in Python 3.8 by jakeogh on Monday October 14, 2019 @10:45PM (#21255138)
You might like portage. We have USE flags (which are ./configure --stuff), slots (so you can install multiple versions in a clean way), and any kind of dependency tree you can think of. When I am thinking of upgrading the system python, I can enable the next version and let things simmer for testing (building for py3.7 and 3.8 for example) before I actually throw the switch (eselect python set N).
Re:What’s New in Python 3.8 by jhanschoo on Tuesday October 15, 2019 @12:05AM (#21255466)
While not official, doesn't conda solve this problem? (Except for not containing the latest versions).
Re:What’s New in Python 3.8 by edflsafoiewq on Monday October 14, 2019 @10:14PM (#21254985)
I'd really like a way to refer to a variable in a outer scope (like global... but not just for global variables).
Re:What’s New in Python 3.8 by paultudorjones on Monday October 14, 2019 @10:26PM (#21255047)
Re:What’s New in Python 3.8 by eru on Tuesday October 15, 2019 @02:16AM (#21255961)
In the same vein, they could add syntax to distinguish introducing a variable from setting the value of an existing variable.

Python's heuristics for that are pretty annoying.

Re:What’s New in Python 3.8 by arnsholt on Tuesday October 15, 2019 @04:19AM (#21256518)
I think that feature has exactly zero chances of getting adopted, but that's probably one of my top Python annoyances too.

If it's any consolation, the scope of a variable isn't determined by heuristics, it's just that the rules are (IMO) kinda bad. Basically, if a variable is assigned (in addition to `=` and friends, `import`, `class` and `def` are also assignments in disguise) to in what Python calls a "block" (which is not like a C-style block; rather `class` body, `def` body, or top-level) it is considered to belong to that block (with the notable exception of the name for a caught exception in a `except Exception as e: ...`, see [1]). If you want it to refer to a variable outside the block, you need to use `global` or `nonlocal` as appropriate. The full gory details are in [2]

1: https://docs.python.org/3/reference/compound_stmts.html#the-...

2: https://docs.python.org/3/reference/executionmodel.html#nami...

Re:What’s New in Python 3.8 by eru on Tuesday October 15, 2019 @06:12AM (#21257071)
Yes, I meant the rules are bad. I called them heuristics, because I assume they came up with the rules as a heuristic to what would be the most useful behaviour.

Python didn't use to have lexical scoping, so the rules are retrofitting around that.

Re:What’s New in Python 3.8 by arnsholt on Tuesday October 15, 2019 @07:54AM (#21257540)
Yeah, it all seems a bit hacky. Especially the bit where a caught exception bound to a variable is cleared at the end of the `except` clause. I'm pretty sure it's because if you didn't you'd get a reference cycle (exception object -> stack trace -> function invocation record -> function locals -> exception object), which makes CPython's GC sad.
Re:What’s New in Python 3.8 by alanfranz on Tuesday October 15, 2019 @02:46AM (#21256107)
... and what prevents you from doing that? You mean you'd want to reassign them?
Re:What’s New in Python 3.8 by manojlds on Tuesday October 15, 2019 @01:11AM (#21255725)
Yeah the fact that a new release is boring is a good thing.
Re:What’s New in Python 3.8 by Grue3 on Tuesday October 15, 2019 @09:20AM (#21258037)
Multi-line/multi-expression lambdas.

"end" statement which would enable automatic indentation.

"switch" statement instead of "if/elif/.../else" hell.

Re:What’s New in Python 3.8 by ktpsns on Tuesday October 15, 2019 @09:33AM (#21258155)
Oh yes, too all of them!

Correctly done multi-line lambdas/expressions could bring Python's expressivitiy to where ECMAScript is nowadays, where you can write both foo = function(...) or function foo().

Perl 6 has a great switch statement, called "given" https://docs.perl6.org/language/control#given

Re:What’s New in Python 3.8 by Franciscouzo on Tuesday October 15, 2019 @12:26AM (#21255559)
A null coalescing operator
Re:What’s New in Python 3.8 by j88439h84 on Tuesday October 15, 2019 @05:48AM (#21256948)
There was a now-deferred PEP for 3.8 https://www.python.org/dev/peps/pep-0505/
Re:What’s New in Python 3.8 by metalforever on Tuesday October 15, 2019 @09:31AM (#21258136)
I would honestly like an option to "compile" python to check for typing inconsistencies with the type hinting introduced in Python 3. Also, the multithreading story is kinda shitty
Re:What’s New in Python 3.8 by Rotareti on Tuesday October 15, 2019 @05:28PM (#21264032)
> I would honestly like an option to "compile" python to check for typing inconsistencies

You can use mypy for that: https://github.com/python/mypy

Re:What’s New in Python 3.8 by j88439h84 on Tuesday October 15, 2019 @12:49AM (#21255654)
Pattern matching. Quasiliterals. PSF support for Poetry or PyPy.
Re:What’s New in Python 3.8 by jp_sc on Monday October 14, 2019 @10:34PM (#21255086)
I would love if they added "?" to the list of valid characters for names, so I could write something like `valid?()` instead of ``is_valid()`.
Re:What’s New in Python 3.8 by bandrami on Tuesday October 15, 2019 @02:31AM (#21256035)
just use "_P", a fairly-widely-recognized substitute from the Lisp world.
Re:What’s New in Python 3.8 by maksimum on Tuesday October 15, 2019 @02:39AM (#21256075)
As a python programmer I would have no idea what `valid_P` is supposed to mean. Valid Pascals?
Re:What’s New in Python 3.8 by bandrami on Tuesday October 15, 2019 @02:52AM (#21256134)
P is shaped like a question mark, though I think it was retronymed to stand for "Predicate"
Re:What’s New in Python 3.8 by NateEag on Tuesday October 15, 2019 @07:29AM (#21257418)
I think "predicate" is the original meaning of that postfix.

Emacs Lisp, at least, can use question marks in names without batting an eyelash.

I see no reason that would not have been true for Lisps generally since their birth.

Re:What’s New in Python 3.8 by ptx on Tuesday October 15, 2019 @08:05AM (#21257592)
The suffix is just "p" in Common Lisp[1], so I think the "predicate" explanation is more likely to be correct than the visual analogy to the question mark.

[1] http://www.lispworks.com/documentation/HyperSpec/Body/f_list...

Re:What’s New in Python 3.8 by cutler on Tuesday October 15, 2019 @07:54AM (#21257543)
Or even better, just `valid?` Oh, wait .... that's Ruby.
Re:What’s New in Python 3.8 by alanfranz on Tuesday October 15, 2019 @02:45AM (#21256101)
Ruby to the rescue!
Re:What’s New in Python 3.8 by themeiguoren on Wednesday October 16, 2019 @06:51AM (#21268987)
I want to build executables that I can pass around without also sending an entire environment. Crazy that’s not a native feature.
Re:What’s New in Python 3.8 by bouncing on Tuesday October 15, 2019 @05:58AM (#21257003)
Honestly? Performance and the possibility of making Pypy the reference implementation.
Re:What’s New in Python 3.8 by KaiserPro on Tuesday October 15, 2019 @09:09AM (#21257949)
proper threads.