News for hackers. Stuff that matters.
Re:Adoption of Mypy for Python type checking: 45% already us... by skohan on Sunday March 21, 2021 @10:25AM (#26530687)
When I read the twitter replies, it doesn't make me optimistic for the future of typed python. I don't work with Python often, so I don't know what the "truth on the ground" is, but it seems like having a language divided between several different type-checking implementations is going to be a nightmare for things like interoperability of frameworks.

I'm imagining a future 5 years down the road where some poor python developer has to use some "polyfill" solution because they want to import a library which was developed with a type-checking solution which is no longer main-stream, or because the type-checkin solution they're using has breaking changes with respect to previous versions.

Re:Adoption of Mypy for Python type checking: 45% already us... by philwelch on Sunday March 21, 2021 @10:50AM (#26530861)
Python 3 type annotations are part of the language spec. Whether you use mypy or a different library for type checking shouldn’t necessarily make a difference to the code.
Re:Adoption of Mypy for Python type checking: 45% already us... by ali_m on Sunday March 21, 2021 @11:23AM (#26531113)
Type annotations may be part of the language spec, but the way they are enforced is not. Different type checking libraries have made different design choices (leniency vs strictness, type inference vs gradual typing etc.). Although these differences don't matter at runtime, they do mean that code which passes one particular type checker might fail with another.
Re:Adoption of Mypy for Python type checking: 45% already us... by throwaway81523 on Sunday March 21, 2021 @03:54PM (#26533370)
There actually seems to be no story about inductive types (e.g. trees) in Python and this appears due to well known theoretical difficulties combining subtyping (required for OO) and type inference (?). I.e. it is a hard problem. There are a few long term open mypy github issues about it, and some hacky workarounds, but no real answer. This is also why languages like Haskell (with good support for inductive types) tend to not have OO. I'm not exactly familiar with the issues, but there is some PLT literature about it, by Anton Setzer and others.
Re:Adoption of Mypy for Python type checking: 45% already us... by joshuamorton on Monday March 22, 2021 @12:13PM (#26542829)
Can you elaborate on this? Forward refs and recursive types are accepted on most places that isn't NewType, so you can make them, but not in a one liner.
Re:Adoption of Mypy for Python type checking: 45% already us... by throwaway81523 on Sunday March 28, 2021 @02:04PM (#26612596)
Sorry about the slow reply but I just noticed your question. Haskell supports recursive types just fine. The issue I'm mentioning is that it is hard to express those types in MyPy, which is a static typechecking add-on for Python, not Haskell.
Re:Adoption of Mypy for Python type checking: 45% already us... by davnicwil on Sunday March 21, 2021 @12:09PM (#26531434)
Is the solution here just to skip type checking the code of dependencies, and just check the types at the interface with your own code using your tool of choice? Or is it more complex than that?
Re:Adoption of Mypy for Python type checking: 45% already us... by neilparikh on Sunday March 21, 2021 @02:47PM (#26532810)
I think that mostly works, but it's still possible that different tools assign different semantics for the same syntax, which could then interact in a weird way if the library author wrote the interface annotations with Tool A's semantics in mind, while you ended up checking them against your code with Tool B.

I think the type systems are simple enough that this is probably a theoretical concern, and won't come up often in practice.

Re:Adoption of Mypy for Python type checking: 45% already us... by jraph on Sunday March 21, 2021 @01:40PM (#26532232)
I think so. When typing checking a project, its dependencies don't get type checked as well as far as I know. Type errors in dependencies are kind of not relevant for you, and many dependencies are not type-annotated anyway.
Re:Adoption of Mypy for Python type checking: 45% already us... by ali_m on Sunday March 21, 2021 @02:29PM (#26532653)
Well you probably do still want the type checker to at least look at the public interfaces of any libraries you depend on, since you want it to enforce that you're calling into them correctly.
Re:Adoption of Mypy for Python type checking: 45% already us... by jraph on Sunday March 21, 2021 @02:38PM (#26532735)
Absolutely. It should probably not report errors that are not in your code, by default anyway, but still extract information about typing relevant to you as much as it can.
Re:Adoption of Mypy for Python type checking: 45% already us... by heavyset_go on Sunday March 21, 2021 @03:36PM (#26533206)
There is no semantic meaning given to type annotations by the interpreter. There's no need for polyfill nor are there issues with interoperability of frameworks because, again, type annotations have no semantic meaning.

They're like comments, they're only there for developer convenience and can be completely ignored.

Re:Adoption of Mypy for Python type checking: 45% already us... by midrus on Sunday March 21, 2021 @10:55AM (#26530897)
The ecosystem is already pretty split on several fronts, not just typing. Python 2 vs 3, async vs blocking, packaging tools, etc.

I left python 3 or 4 years ago and not planning to go back (basically the company I was working for mandated they were not going to ever upgrade to python 3 because of the effort required and thus stopped all migration related projects).

The language is nice, and frameworks such as Django are great, but the mess the ecosystem has become is not something I miss.

Re:Adoption of Mypy for Python type checking: 45% already us... by johnnycerberus on Sunday March 21, 2021 @12:18PM (#26531497)
I think your comment is a little bit alarmist.

* Python 2 is dead.

* pip is the most popular in the industry, conda serves a different purpose if your focus is mainly on scientific computing

* what does async vs blocking even mean in this context? all languages have the same problem, just use the libraries that are non-blocking.

* mypy is the static type checker that is the most popular (28.3k repositories, 1.9k packages), if you are using other type checkers then congrats, you will just be the CoffeeScript/Reason user of the JS world, refusing to believe that TypeScript won the mindshare

Re:Adoption of Mypy for Python type checking: 45% already us... by dragonwriter on Sunday March 21, 2021 @07:25PM (#26535085)
> mypy is the static type checker that is the most popular (28.3k repositories, 1.9k packages), if you are using other type checkers then congrats, you will just be the CoffeeScript/Reason user of the JS world, refusing to believe that TypeScript won the mindshare

Not really. TypeScript is a JS superset (and TypeScript tooling works with unadorned JS, too), but Coffee script/Reason are incompatible languages that happen to compile to JS and have static typing. All of the Python typecheckers are more like TypeScript, except that even the type annotations and much of the type system is also standardized as part of the core language. You can literally use any of them on the same code base, and get largely the same results (some will catch some type errors that others miss, some will flag some correct code as possibly erroneous that others would leave alone, but the differences tend to be minor.)

Re:Adoption of Mypy for Python type checking: 45% already us... by joshuamorton on Sunday March 21, 2021 @02:03PM (#26532436)
Pytype (Google) pyre (facebook) and pyright (microsoft) all have more institutional backing than mypy. They aren't going away anytime soon.

Otoh, development of all the major type checkers is fairly collaborative. Mypy usually gets features first, but not always.

Re:Adoption of Mypy for Python type checking: 45% already us... by spicybright on Sunday March 21, 2021 @06:44PM (#26534857)
Why not just adopt one into the standard library, seems like a very obvious battery you'd want to include.
Re:Adoption of Mypy for Python type checking: 45% already us... by joshuamorton on Sunday March 21, 2021 @07:16PM (#26535029)
They have different goals. Pytype does inference, which allows easier bootstrapping. Pyre is fast and supports some nonstandard control flow and taint flow stuff. Pyright supports partial validation for syntactically broken files, for use as part of an ide.
Re:Adoption of Mypy for Python type checking: 45% already us... by dragonwriter on Sunday March 21, 2021 @07:29PM (#26535103)
> Why not just adopt one into the standard library, seems like a very obvious battery you'd want to include.

Instead, the approach seems to be adopting a standardized system of type annotations into the standard library. While a typechecker may expose a library interface, it's main use is as an executable tool, not a library (unlike annotations themselves), so it's not super critical to have one in the standard library. It might be convenient to have it in the standard distribution, but as a practical matter it's not like there is a problem getting at least one of them that does a suitable job with standard type annotations anywhere you can get Python.

Re:Adoption of Mypy for Python type checking: 45% already us... by heavyset_go on Sunday March 21, 2021 @07:37PM (#26535152)
Check out the typing module[1] from the Python standard library. A lot of it was adopted directly from Mypy.

[1] https://docs.python.org/3/library/typing.html

Re:Adoption of Mypy for Python type checking: 45% already us... by maple3142 on Sunday March 21, 2021 @12:48PM (#26531761)
> what does async vs blocking even mean in this context? all languages have the same problem, just use the libraries that are non-blocking.

Maybe Node.js? Although it does have some sync function to do io, most of them are still unblocking, and the ecosystem is unblocking by default too. It is harder to find a blocking library than finding a non-blocking ones.

Re:Adoption of Mypy for Python type checking: 45% already us... by Izkata on Monday March 22, 2021 @08:37PM (#26549459)
> * Python 2 is dead.

Officially yes, but in terms of actual use? Not really. Plenty of legacy codebases are still around that don't have the resources to upgrade.

Re:Adoption of Mypy for Python type checking: 45% already us... by elcritch on Sunday March 21, 2021 @10:34PM (#26536257)
Async vs blocking functions aren't a thing in Erlang and Elixir because everything (almost) is non-blocking.
Re:Adoption of Mypy for Python type checking: 45% already us... by guitarbill on Sunday March 21, 2021 @12:12PM (#26531450)
I agree that the ecosystem is a bit messy, but it's more of a function of how long Python has been around, and how successful it has been. Newer languages have the benefit of hindsight.

Take `go fmt` or `cargo fmt`. black is the de facto autoformatter, but it's still an optional dependency. Maybe like virtualenv it'll eventually be integrated into the standard library/distro. That would help. With other stuff, the split you describe could be a real problem, although for the majority of Python users, using mypy and poetry is fine.

> basically the company I was working for mandated they were not going to ever upgrade to python 3 because of the effort required and thus stopped all migration related projects

If you're actually serious, this is intriguing to me. Business-wise, I have never seen the math shake out this way. When we've done sizings/estimated developer hours and ROI, it's always been more economic to slowly migrate to Python 3 vs a rewrite, even if the dev team is also already using and well-versed in another language. If you're able to provide more details, please do. It's so rare to have a high-quality discussion about this.

Re:Adoption of Mypy for Python type checking: 45% already us... by AlphaSite on Sunday March 21, 2021 @11:35AM (#26531200)
Python 2 is pretty dead at this point.
Re:Adoption of Mypy for Python type checking: 45% already us... by e3k on Sunday March 21, 2021 @07:18PM (#26535043)
2.x dead? Not really: lots of 10 year old projects are in maintenance or even actively developed.

Migrating several hundred thousand lines of code bases with few people and many dependencies (some of them not maintained, not ported to 3) are not trivial.

These will stay for years still. Actually it is very sad, that significant new tools are basically useless on 2.x (e.g. vscode: lots of waiting on completion, or jumping to definition - if it works at all).

Re:Adoption of Mypy for Python type checking: 45% already us... by heavyset_go on Sunday March 21, 2021 @04:33PM (#26533723)
> Python 2 vs 3

This isn't an issue in 2021. Python 2 reached its EOL over a year ago, and those who own old Python codebases have had known for literally almost 13 years now that Python 2 would be EOL'd when Python 3 came out in 2008.

Re:Adoption of Mypy for Python type checking: 45% already us... by canjobear on Sunday March 21, 2021 @11:07AM (#26531000)
It’s always possible to run code ignoring all type annotations.