News for hackers. Stuff that matters.
Re:Things to use in Python 3 by DonaldPShimoda on Wednesday May 15, 2019 @12:34AM (#19916434)
> Type hinting (3.5+)

> Static vs dynamic typing is a spicy topic in software engineering and almost everyone has an opinion on it. I will let the reader decide when they should write types, but I think you should at least know that Python 3 supports type hints.

I think this gives readers the impression that type hints have anything to do with dynamic vs. static type systems, which isn't true. These are merely annotations that are attached to values. In fact, it's legal to use full strings as type annotations — they have no inherent semantic value.

You can gain semantic value by using a tool like MyPy or an IDE like PyCharm, but the type annotations do not in themselves do anything. I think it could be worth clarifying this for readers who are unaware.

Re:Things to use in Python 3 by BerislavLopac on Wednesday May 15, 2019 @04:43AM (#19917475)
The most underrated part of type annotations are so-called stub files [0], which allow you to keep your annotations separate from the actual code, thus reducing its size while still benefiting from the static analysis with mypy.

Unfortunately the IDE support for those files (.pyi extension) is quite limited -- I have even suggested to the PyCharm team to implement an option to automatically save all annotations to stub files, while still displaying them alongside the code in the editor.

[0] https://www.python.org/dev/peps/pep-0484/#stub-files

Re:Things to use in Python 3 by NelsonMinar on Wednesday May 15, 2019 @10:22AM (#19919638)
I'm sort of sad that Python has reinvented .h files.
Re:Things to use in Python 3 by ASalazarMX on Wednesday May 15, 2019 @01:32PM (#19921763)
Same here. Fortunately those files are entirely optional and apparently seldomly used, judging by the limited IDE support after 5 years.
Re:Things to use in Python 3 by DonaldPShimoda on Wednesday May 15, 2019 @12:44PM (#19921207)
Whoa, that's neat! I had no idea!

Does PyCharm support these? Like if I write the stub files manually, will type hints and autocompletion work correctly in the main file? This could be a great way to reduce the amount of syntactic overhead, at least in function definitions and top-level code.

Re:Things to use in Python 3 by BerislavLopac on Thursday May 16, 2019 @07:23AM (#19927952)
At the moment, you can click on a (say) class definition, and, if it has the corresponding entry in the pyi file, it will open it. But I would like it to be a bit more integrated.

Another advantage is that you can use pyi files for static checking, but remove them when deploying the application.

Re:Things to use in Python 3 by tlavoie on Thursday May 16, 2019 @12:24PM (#19930394)
Sounds sort of like the .mli files in Ocaml. As I understand it, the compiler uses (and enforces) the types found therein, though can infer a lot from the code in .ml files as well.
Re:Things to use in Python 3 by ehsankia on Wednesday May 15, 2019 @03:04AM (#19917044)
Yeah, I'm pretty sure Python itself explicitly uses the term "annotations" everywhere instead of "type hints". The point is that they can be used for anything, typing being one such use.

As for libraries, other than MyPy, there's also Pyre by Facebook and Pytype by Google. I'm definitely excited to see more advanced static analysis being implemented beyond simple type checking. Some of these libraries are starting to explore that.

Re:Things to use in Python 3 by Sharlin on Wednesday May 15, 2019 @04:30AM (#19917414)
Permitting arbitrary annotations is a rather nice feature that, besides static analysis, opens the door for expressive runtime pre/postcondition checking functionality like the Clojure spec library[1]. Does Python have something similar?

[1] https://clojure.org/guides/spec

Re:Things to use in Python 3 by auscompgeek on Wednesday May 15, 2019 @08:20AM (#19918656)
Unfortunately PEP 563 [1] means that any non-type-hint usage of annotations is considered deprecated. You could still do things with decorators (like mentioned in a sibling comment), although I suppose that doesn't offer such an elegant syntax.

[1]: https://www.python.org/dev/peps/pep-0563/#non-typing-usage-o...

Re:Things to use in Python 3 by DonaldPShimoda on Wednesday May 15, 2019 @12:47PM (#19921234)
Although not really what they were intended for, somebody used annotations to provide a macro system [0].

[0] https://tinkering.xyz/abusing-type-annotations/

Re:Things to use in Python 3 by z_mitchell on Wednesday May 15, 2019 @12:56PM (#19921356)
Author here. Be warned, this was just a proof of concept that I did because I thought it would be fun/weird/interesting!
Re:Things to use in Python 3 by neoncontrails on Friday May 17, 2019 @02:28PM (#19942229)
Oh my goodness. I've committed some naughty deeds with the ast module before, but this is truly, wonderfully, heinous. You push this idea so far that I wouldn't even recognize some of these examples as syntactically valid, let alone predict their effects. I'm awestruck and a little frightened of you.
Re:Things to use in Python 3 by jacobush on Wednesday May 15, 2019 @05:12AM (#19917626)
https://andreacensi.github.io/contracts/tour.html#quick-tour looks to be a stable solution for that.
Re:Things to use in Python 3 by ieie92ndjrj on Wednesday May 15, 2019 @09:57AM (#19919396)
The docs at python.org refer to them as “type hints” ️

https://docs.python.org/3/library/typing.html

Re:Things to use in Python 3 by ehsankia on Wednesday May 15, 2019 @10:56PM (#19925765)
As mentioned in the other comment, this is just a library used to give type hints as annotations. The feature itself is called annotations, this is one specific use-case.

The reason this library exists in stdlib is so that all the different type hint libraries can interoperate. Otherwise if they each brought their own typing, you couldn't switch between different static analysis libraries.

Re:Things to use in Python 3 by uranusjr on Wednesday May 15, 2019 @10:06PM (#19925559)
`typing` is the module that uses annotations as type hints. Parent is correct; Python documentation calls the syntax annotations.
Re:Things to use in Python 3 by ieie92ndjrj on Thursday May 16, 2019 @02:17PM (#19931399)
So Python docs refer to both depending on the context.
Re:Things to use in Python 3 by BlackFly on Wednesday May 15, 2019 @04:37AM (#19917442)
They are available in the runtime, so you can do runtime reflection on the annotations and perform dependency injection if that is your thing. In that sense they do something: they provide information to the runtime which you can use if you want.
Re:Things to use in Python 3 by DonaldPShimoda on Wednesday May 15, 2019 @12:49PM (#19921263)
I think this is just a semantic argument at this point, but to me what you mention isn't evidence of the annotations doing anything. They make it possible for things to be done, just as I said.

The distinction I'm drawing is to statically-typed languages where simply adding annotations will actually change semantics of the program (e.g., by forcing a downcast or something).

But you're absolutely right that annotations can be useful for many things!

Re:Things to use in Python 3 by xtf on Wednesday May 15, 2019 @02:39AM (#19916943)
Except further optimisation in Cython like http://docs.cython.org/en/latest/src/tutorial/pure.html#stat...
Re:Things to use in Python 3 by DonaldPShimoda on Wednesday May 15, 2019 @02:47AM (#19916971)
I think this falls under the realm of "gain[ing] semantic value by using a tool like MyPy". The implementation of static typing by way of processing type annotations is an implementation choice, and not a part of the Python language specification.
Re:Things to use in Python 3 by Animats on Wednesday May 15, 2019 @03:47AM (#19917217)
The implementation of static typing by way of processing type annotations is an implementation choice, and not a part of the Python language specification.

That comes close to being the worst idea in the history of programming languages. Types aren't checked, you can't trust the annotations, and you give up a big optimization opportunity. But Guido's naive interpreter will still work.

If you say something is a float, the compiler needs to both enforce that and use that. Then maybe the Python crowd wouldn't need to call out to C code whenever they needed to do some number crunching.

Re:Things to use in Python 3 by shakna on Wednesday May 15, 2019 @05:59AM (#19917850)
> Then maybe the Python crowd wouldn't need to call out to C code whenever they needed to do some number crunching.

Numpy and Scipy use a lot of Fortran at the backend, and unfortunately, beating Fortran when it comes to number crunching is insanely difficult, not just because of the language, but because those libraries like LAPACK have spent decades being refined into incredibly fast and accurate systems.

Python's ability to talk to those libraries is a bonus. But Python itself could never compete directly with them. Pure-python code isn't capable of some of the insane speed those libraries can pull off.

Re:Things to use in Python 3 by whatshisface on Wednesday May 15, 2019 @07:54AM (#19918495)
>because those libraries like LAPACK have spent decades being refined into incredibly fast and accurate systems.

In addition, due to their use in benchmarks, x86 has in part been designed to make LAPACK fast.

Re:Things to use in Python 3 by coldtea on Wednesday May 15, 2019 @04:08AM (#19917311)
>That comes close to being the worst idea in the history of programming languages. Types aren't checked, you can't trust the annotations

As for types not being checked, that's not a problem. JITs do extra specialization on assumed types even without annotations all the time, and just drop them at runtime when they are not valid anymore (e.g. a variable that only pointed to integers now stores a string). Still, this runtime overhead of dropping specialized code aside, those optimizations make e.g. v8 hella fast.

(And of course you could also just enable optimizations after you've statically checked the whole program types with something like mypy -- this is even easier and less dynamic than what v8 does).

>and you give up a big optimization opportunity

You don't give anything up, since an implementation can take advantage of this "big optimization opportunity" if it wants and has the manpower to add it.

>But Guido's naive interpreter will still work.

It's also less work, which unless we have a volunteer here, was and remains the intention.

Re:Things to use in Python 3 by orf on Wednesday May 15, 2019 @04:06AM (#19917297)
Why is calling out to C code via something like numpy a bad thing?

Python is never going to be as fast as C. Ever. Even if you somehow managed to decouple numeric types from PyObject in a backwards compatible way it would not be as fast.

Type Hints can’t be used as an optimization in the interpreter due to the way they are resolved. And in any case, it would not be safe to do so and there are very few cases where optimizations make sense.

Re:Things to use in Python 3 by coldtea on Wednesday May 15, 2019 @04:02AM (#19917279)
Though nothing prevents it from becoming "part of the Python language specification" -- except the backend implementation (besides Python doesn't have a specification, it's what the CPython does).
Re:Things to use in Python 3 by DonaldPShimoda on Wednesday May 15, 2019 @12:52PM (#19921304)
Well I think you could argue that Python is specified in part by CPython, but also in part by the PEPs. As static type checking is inherently opposed to the Python philosophy, it would take a PEP to force CPython to change its implementation accordingly.
Re:Things to use in Python 3 by seertaak on Wednesday May 15, 2019 @10:13AM (#19919552)
Thanks for pointing that out! -- I've been missing a feature like that in Numba (which is, otherwise, really cool).
Re:Things to use in Python 3 by bitcurious on Wednesday May 15, 2019 @10:15AM (#19919571)
What's the best practice for type hinting a class that's not being imported?

I.e. A module contains a function that accepts a pandas.DataFrame. This module doesn't import pandas.

Re:Things to use in Python 3 by jimbo1qaz on Wednesday May 15, 2019 @10:49AM (#19919920)
from typing import TYPE_CHECKING

if TYPE_CHECKING:

  import whatever
def foo(param: 'whatever'): ...
Re:Things to use in Python 3 by bitcurious on Thursday May 16, 2019 @03:19PM (#19932122)
Thanks! :)
Re:Things to use in Python 3 by mrfusion on Wednesday May 15, 2019 @09:07AM (#19918985)
Are there any tools that can use the annotations and watch a running script and warn you if they’re violated? Maybe during your test suite.

Or linters could do some basic analysis.

Re:Things to use in Python 3 by uryga on Wednesday May 15, 2019 @09:23AM (#19919120)
off the top of my head there's pytypes¹ and typeguard² – both look at type annotations and enforce them at runtime.

¹ https://github.com/Stewori/pytypes/

² https://github.com/agronholm/typeguard

Re:Things to use in Python 3 by xjlin0 on Wednesday May 15, 2019 @01:23AM (#19916639)
How to do Type hinting with argument default values?
Re:Things to use in Python 3 by DonaldPShimoda on Wednesday May 15, 2019 @01:26AM (#19916649)
Pretty straightforwardly:

    x: int = 0
This makes sense when you see that the variable production in the Python grammar [0] has been updated to essentially read "either a lone variable or a variable with a type annotation". (I highlighted the relevant line of the grammar for you in the link.)

[0] https://github.com/python/cpython/blob/master/Grammar/Gramma...