This seems like an inaccurate and condescending put-down. What is the definition of "educated" in this context? Is there a book or generally well-known resource?
The author is clearly thoughtful and curious. Near the top of the post he says "what I’m hoping for is that someone will come along and tell me that I got it all wrong. “When you do it as follows, the system works: $explanation” Because, you know, I’d really like to have static typing in Python."
> 1. If you're using a dynamic language, then _by definition_ the language will not enforce your static hints at runtime. However, good news! Python has always been strongly typed, and _does_ enforce types at runtime!
And yet, the author points out that this is a working program:
foo: int = 'hello'
print(foo)
The Any type is to allow incremental typing. You start out with dynamic code, and progressively 'typify' it by adding more and more annotations, with Any working as a stopgap where for the moment no valid type exists, until you finally have a fully typed program. Inserting Anys because you're lazy is like casting things to Object in Java because you're lazy, abusing it like that is just incredibly sloppy and bad code.
If someone is really struggling with this, simply use a linting rule to ban Any in 'mature' code.
There are lint rules to avoid typing as `any`.
Python is strongly typed. It respects types at runtime e.g., `"1"+1` causes TypeError. Python is not statically typed.
For comparison, C is statically typed but it is weakly typed e.g., printf function has no idea about actual types of its arguments (in particular, errors in the format arg may produce non-sensical results silently).
Can a string be printed? Yes! Then Python is (correctly) allowing typesafe behavior here. Your (incorrect) annotation does not in any way contradict the typesafe-ness of printing a string (or an int). They're both equally printable.
This is just a very, very bad example, plain and simple. It 'looks' bad to a superficial reading, but in practice it demonstrates only what is already known about static typing in Python - it is enforced (or not) separately from the interpreter.
Am I the only one baffled by this way of thinking?
Static type annotations by definition are not enforced at runtime. This has been true of every language that has ever used static typing, including C, C++, Java, Rust, Typescript... you name it, statically typed languages only enforce their static typing at compile/analysis time.
Yes, it is possible to annotate types in Python incorrectly. It's possible to do this in all other languages that allow type-unsafe behavior (whether natively or via reflection, etc). This may be less common in some languages than in others, but it is fundamentally possible in the vast majority of languages that perform static typing, because those types are fundamentally enforced at analysis time, not runtime.
The author of the article seems to be unfamiliar with these distinctions, and maybe you are too. It's fair to complain that these distinctions are confusing and make things harder for programmers. Nevertheless, very few languages have ever been designed with a type system that acts exactly the same at analysis time and runtime. It's a very difficult problem, partly because these are all abstractions from the perspective of the underlying computer, which only understands boolean logic and integer/floating point math, and has virtually no other notion of types in any formal sense.
Type systems are a complex topic, and as someone who has been paying attention to them for a while, it's frustrating to see uninformed discussion of them show up on Hacker News. That said, it's perfectly understandable that people are confused by these distinctions, and rest assured there's a lot of effort going into developing better languages that suffer _less_ from these issues.
For the everyday working programmer, however, there are currently lots of tradeoffs to be made, and Python's approach to static+dynamic typing is actually pretty usable, all things considered, which is why the community overall has embraced the new static typing despite its blemishes.
Also, some languages like Java do actually enforce types at runtime. If you've ever called a Java method dynamically with the wrong types, you'd know. You can run into ClassCastException, or worse (ClassNotFoundException if you have the wrong classloader.) This is a runtime error.
A type system is just a set of rules that must be followed. The rules can be as weird or as simple as you want. Just because <int> + <string> is allowed doesn't mean it's not enforcing types.
Not really a criticism, just trying to push to expand your idea of what a type system is.
It is pretty damn hard to make this mistake in languages that enforce static typing. I mean, you would have to go out of your way to do so. In Python, however, it is trivial to write the wrong type signature or to modify the body of a function so that it no longer matches the signature.
"Fundamentally possible but terribly unlikely, as opposed to the Python way of doing it" would be a more accurate description.
Since this argument is not meaningfully different from "comments that are lies considered harmful", it seems fair to expect reasonable people to dismiss it as uninteresting.
However, there are other aspects to the article’s argument, such as the fact that it’s easy to end up with type annotations going silently unenforced even if you do run the checker.
Do you think that's a fair treatment of someone who disagrees with you but has been respectful of your opinion so far?
I realize proglang debates are flamewar territory. But I have been very careful to state I find Python type hints puzzling and less useful than they should be. It is obviously an interesting opinion shared by many others, not just me or the article's author.
As a fan of statically typed languages, I do indeed find some Python programmers engage in a form of Stockholm's syndrome. It usually takes the form of the assertion "I never found a bug that was a type error". Have you or anyone you know ever said this?
In my mind, the opposite statement of the one you attribute to me would be "wishful thinking and a positive attitude is enough to catch mistakes, it's not necessary to have help from automated tooling". In this day and age, I cannot disagree more.
It's precisely because I do like and use them that I find Python's static type hints to be extremely useful for large Python codebases.
What I hear you saying is "Python does not offer these things". What I hope you hear me saying is "Python does, in fact, offer these things".
> I think your (and the article's) argument could be summarized as "static type annotations without automated enforcement by actually running a type checker considered harmful"
It illustrates your mindset, in my opinion. I think type annotations without checking them (or without a satisfying implementation of said type checking) are mostly pointless.
You also left out the part where I remarked on the dismissive tone of your reply.
Is it "legally" baffling, i.e., do I understand why this behaves in the way it does purely mechanically, and do I understand the arguments the gymnasts are making? Yes.
But from an idiomatic, colloquial, linguistic, syntactic, programming-historical, or programming-cultural standpoint? Baffled.
Even calling it gradual typing is baffling. It's just adding metadata-that-doesn't-look-like-metadata that sophisticated programs that aren't part of the Python bytecode compiler can use. The definition I know is the same one from Wikipedia:
> Gradual typing is a type system in which some variables and expressions may be given types and the correctness of the typing is checked at compile time [...]
This is definitely not what Python is doing.
Maybe not by default but there is tooling that makes it work that way — that’s exactly how the python build system works at my company. I don’t know the details but when I “build” python code (creating bytecode pyc files) it produces compiler errors and fails to build if I e.g. try to pass an int to an f(x: str). This has usefully caught bugs before I ran some expensive/time-consuming scripts.
I guess you could say this doesn’t count as it’s a separate program doing the type checking but IMO there’s not a clear distinction between that and e.g. having a first type checking pass in the “same” compiler program.
> While these [type] annotations are available at runtime through the usual __annotations__ attribute, no type checking happens at runtime. Instead, the proposal assumes the existence of a separate off-line type checker which users can run over their source code voluntarily.
When I read "compile time", I mentally replace it with "before runtime". There isn't a real compilation step in Python, so it's reasonable to accept static code analysis as part of "compile time" in that definition.
The point is that type annotations will let a static analysis tool make sure that the program is correctly typed. This can be enabled easily in most of the modern IDEs and editors. You could just decide to block the code's deployment in CI/CD if the type checker raises issues, for example.
E.g. this kind of code: https://play.rust-lang.org/?version=stable&mode=debug&editio...
typically works fine (or worse, works fine in debug mode and causes unpredictable UB-related bugs in release mode).
std::mem::transmute here is a no-op at runtime; all it does is subvert the static type checker.
The difference between statically-typed languages and things like Python with type hints is _not_ that the former has any runtime type checking; it’s that subverting the (static!) type system is more difficult and requires more obvious ceremony, and that static type checking is required whereas with Python it’s optional.
If you hacked rustc to disable static typechecking you would get a similar experience to Python with type hints but without any mypy-like tools. (Of course, this would be hard to do in practice, because Rust, like most statically-typed languages, uses types not just for typechecking but also for code generation and method dispatch.)
This said, I find the linting/type-hint-checking stage of Python cumbersome and confusing :(
When Python "enforces types at runtime" this is the actual types, not type hints. Type hints are not a runtime artifact. The "actual type" here is "string", and enforcing it means Python would not allow invalid operations on it without error'ing. A programming language that will let you do basically anything to any value because it doesn't enforce type safety is of course C. Python is safer than C.
*(unsigned long*)0xFFFFFF14 = 0x749235f8;
*0xFFFFFF14 = 0x749235f8;
char* s = "abc";
*(unsigned long*)0xFFFFFF14 = s;
Best of all, all of these are enforced at compile time.
Now, C absolutely will let you convert an int to a pointer, or a char to an int, or an array to a pointer, or... well, it will let you convert lots of things to lots of things. Some of those things are unsafe unless you are quite sure what you're doing.
> A programming language that will let you do basically anything to any value because it doesn't enforce type safety is of course C.
False on several grounds. You can't call something that isn't a function. You can't call a function on something that doesn't have it. You can't use an int as a pointer or an array without casting it. You can't use an int as a dict (not that C has any built-in idea of what a dict is...)
> Python is safer than C.
Depends on what you're doing, and what kinds of errors you are more prone to.
Sure you can. That's exactly what you do when you call a function returned by `dlopen` -- it might be a function, but it might also just be garbage. It might even be garbage that's coincidentally marked as executable and does some stuff before eventually crashing!
> You can't call a function on something that doesn't have it.
This is also very easy to do -- you can default-initialize a structure containing function pointers, and then call one of them. You'll be calling some random crap on the stack, which might or might not do anything of interest. But either way, C will happily let you do it.
C isn't completely untyped (there are, as you've observed, things it will forbid at compile time), but it's about as weak as a static typing system can be.
In both examples, you have tell the compiler exactly what the types are. In the first one you have to cast the return value of `dlsym` to a function pointer, and at that point, as far as the compiler is concerned, it's a function. In the second example you have to explicitly declare the struct containing function pointers, so of course you can call its members.
So it's strange to suggest that C is somewhat untyped; it's not untyped at all. C just makes it very easy to force the compiler to assume different type for a value (that's necessary for low-level code like the examples in the comment you replied to; and of course that makes it very easy to do unsafe things, I don't think anyone disputes that).
And about the "weak typing system" comment, note that nobody agrees on what exactly that means. Just as an example, see how this page[1] defines "weaker" vs "stronger" types, and how it's completely different from the way you're using here. In general, I find people call type systems that don't do what they expect or want "weak", but that's not an useful discriminator: at that point you might just call it "bad typing" to dispel any illusion that it has any objective meaning.
[1] http://book.realworldhaskell.org/read/types-and-functions.ht...
Just because "weak typing" means something different in Haskell doesn't mean it can't be used meaningfully (and beyond "bad") in the context of C. C's typing is historically referred to as "weak" because C's notion of casting doesn't distinguish between type and value transmutation: pointer-to-pointer casts don't convert the referent (because they can't), which in turn gives the C compiler very little leeway in proving that the program's types as declared have any particular meaning at runtime.
Compare this to Python, which is "strongly" typed in the same sense: doing `y = str(x)` on `x` means that `type(y)` actually is `str`, and not merely a promise to the runtime. It's enforced, which makes it strong.
That's why it's useless to say things like "it's about as weak as a static typing system can be" and "has been historically considered weak".
If you had explained your objection to C's type system (like you have now), I wouldn't have said anything. And I've been on the Internet long enough to have seen C's typing is historically referred to as "weak" because X for *many* values of X, so I disagree that that's the only or even main objection.
You know what I meant and what error I was clarifying for the comment I was replying to. Yet you went out of your way to point all sorts of irrelevant mistakes in my post, when the gist of it was right.
Do you feel it was more important to correct me on these trivial things, or was my reply more or less correct when fixing the conceptual error in this sentence?:
> "In what reasonable sense can Python be said to "enforce types at runtime" here?"
Here's a nitpick of my own to your post:
> "Some of those things are unsafe unless you are quite sure what you're doing."
Wrong. Type (un)safety does not depend on you "being quite sure of what you're doing".
PS: assume I know C and that I've programmed complex things using it. No need to catch yourself with statements like "not that C has any idea of what a dict is". Assume we are both C programmers. It will make you sound more polite.
2. Your last two sentences were, I think, not needed for the point you were making. They were also somewhere between gross generalizations and flat-out wrong, and felt like a gratuitous slam on a language that wasn't even the topic. I thought that deserved a response, even if it wasn't your main point.
3. I may assume that you know C. I don't assume that everyone reading this exchange knows C.
Pointless correction. Gross generalizations are not wrong in this particular situation, which is not meant to teach anyone C.
Let me try again.
Do you think that your correction was more important than helping the poster of this comment correct their conceptual mistake?
> "In what reasonable sense can Python be said to "enforce types at runtime" here?"
Yes, but is it caught at static analyzer run? Yes, I typed it into vi and ran mypy on it and it said "error: Incompatible types in assignment (expression has type "str", variable has type "int")."
But I didn't have to run mypy on it, my editor told me: "Expression of type "Literal['hello']" cannot be assigned to declared type "int"".
Maybe the word "educated" wasn't the right choice in this case, can you offer a better one? I honestly couldn't make it through the entire article, despite several attempts. It is full of inconsistencies and what seem like wilful misunderstandings to justify the authors conclusion.
Particularly as the author starts off saying "run a static analysis tool" as if knowing that's how you go about it, and then saying "it doesn't catch it at runtime".
It does: in form of TypeError exceptions. It doesn't enforce type hints, sure, but that's why they're called hints and not declarations.
Python is both strongly typed and protocol-typed. In the case of `print(...)`, the protocol is that the received parameter responds to `str(...)` (i.e., has `__str__`).
(What Python isn't is statically typed. But "strong" and "static" are completely different typing dimensions.)
> In what reasonable sense can Python be said to "enforce statically declared types at runtime" here?
which is certainly an extremely pertinent question for a language that allows declaring supposedly static qualities of a program.
It can't, because Python doesn't have statically declared types. It's a strong, dynamically typed language.
Python also doesn't allow you to declare static types. It allows you to declare type hints, which are a sort of adjacent and potentially unrealistic (or just plain incorrect) typechecking proof that doesn't correspond at all to runtime behavior.
Python has the capacity to have some form of gradual typing built-in at the language level, but it does not, which has led to the enormous confusion we have before us. The SBCL implementation of Common Lisp is a great example of a strong, dynamically typed language that is able to, when possible, detect errors statically at compile time. That tech is 30 years old now.
There is no reason why the Python bytecode compiler could not statically or dynamically detect this programmer error at either compile-time or run-time present in this statement:
x: int = "x"
(declaim (type integer x))
(defvar x "oops")
; ==>
Unhandled SIMPLE-TYPE-ERROR in thread #<SB-THREAD:THREAD "main thread" RUNNING {10005D05B3}>:
Cannot set SYMBOL-VALUE of X to "oops", not of type INTEGER.
quitting
compilation unit aborted
caught 1 fatal ERROR condition
I don't see why that would be the case. The value of Mypy is that it's essentially a proof language over something that resembles Python[1]: it's a way to do some amount of static typing without having to maintain a separate copy and representation of the program.
And sure: Python could detect inconsistent type hints at runtime. But why would it? It's already strongly typed, and the presence of type hints usually means that someone is already running Mypy or another checker during development. It's not clear that there's a significant advantage to be gained, particularly one that justifies the additional overhead.
[1]: "Resembles Python" because mypy does not actually evaluate any Python. It doesn't know what types your program has at runtime; it only knows type hints and a few small rules (for things like string literals) and trusts the developer to reconcile those rules with Python's runtime behavior.
Because the programmer explicitly intended a contract which is being violated, which would in turn indicate a bug.
Anecdotally, my code in python is functionally fully typechecked and runtime type checking would solely penalize me by slowing down my code. There's no need for it because the type checker correctly proves the soundness of the program's types, in exactly the same way that Java's or C++'s do.
The point is precisely that you don't need to suffer the costs of runtime type checking if you have a solid typechecker. Lisp's approach here is worse than the one adopted by python/js/ts/go/etc.
I believe this is the relavent passage in the article the GP is referencing, which is incorrect if you take "runtime" to mean something like when the line is executed: "Python is a dynamically typed language. By definition, you don’t know the real types of variables at runtime."
It _is_ however correct to say you don't know the real type _before_ runtime, at least python does not.
In [1]: foo: int = 'hello'
In [2]: foo + 1
TypeError
Input In [2], in <cell line: 1>()
----> 1 foo + 1
TypeError: can only concatenate str (not "int") to str >>> foo: int = 'hello'
>>> type(foo)
<class 'str'>
But we have (to quote another commenter) "uneducated" people making a supposedly boneheaded mistake to assume
x: T = y
Instead, this line is more properly interpreted as "documentation reasonably believed to represent a type saying something about values that x can be assigned at runtime, that's not in a docstring, that some programs may query."
I can totally forgive anybody who believes the former over the latter, and expresses their belief in a statement like that which you've quoted.
Absent of active machine checking (e.g., via mypy), type hints should be considered exactly what you said: another form of documentation, one that might just be wrong.
But in Python, it's not (and documented in a PEP as not), and it causes massive confusion.
[1] https://devblogs.microsoft.com/typescript/a-proposal-for-typ...
As far as I remember, the "__str__" method is present on all objects in some form, so print will happily take any type.
> Python is a dynamically typed language. By definition, you don’t know the real types of variables at runtime. This is a feature.
In "you don’t know the real types of variables at runtime", this is just flat wrong. One doesn't know the real type of a variable until runtime. It seems to me the author maybe has a bit of confusion between weak typing and dynamic typing here.
I read the sentence as
"you don't know the real types of variable at [the start of] runtime"
"you don't know the real types of variable *during* runtimes"
In the program
def x(f):
return f(3)
If your answer is that `f` has type "function-like", then that's a much weaker kind of a type than even type hints offer, let alone a real type system.
We can tell by looking at what python triggers TypeErrors for.
>If your answer is that `f` has type "function-like", then that's a much weaker kind of a type than even type hints offer, let alone a real type system.
f implements the Callable interface - meaning it's either a function/method or an instance of a class with a `__call__` method.
So you can pass e.g. `print` or an object with a `__call__` method. If you try to do `x(5)` that's an immediate runtime TypeError because `5` can't be called.
f must also accept one int argument. It can have additional optional arguments, and it can also accept other types (e.g. `f("foo")` can also work!), but it must work on one int. The one int can already be optional - the function can also work with zero arguments.
Calling e.g. `"foo".join(5)` fails with a TypeError because 5 isn't Iterable, and calling `"foo".join([1], [2])` fails with a TypeError because it gets more arguments than expected.
So PHP is not a dynamically typed programming language according to you? It enforces type hints at runtime just fine.
There is nothing in the definition of dynamically typed languages that says they can't use type hints to perform runtime checks.
Most people I know writing Python don't use type hints because they are too much of a hurdle for very little payoff. The tooling that pays attention to type hints is slow as molasses or difficult to use or understand. The type hints themselves are of dubious use.
As a fan and advocate of static typing, I find myself advocating for type hints anyway, but I must agree I often can't reply anything to my coworkers' objections, because Python type hints are truly not that useful.
I use mypy daily on big codebases, it is fine, not fast, but fine.
> The type hints themselves are of dubious use.
They tell you when you make type errors, they help you understand what type things should be. The same as in literally every other language with static typing. There is nothing special here, nothing different. python with a static type checker running in strict mode is not fundementally different from Java's static type checking.
> As a fan and advocate of static typing, I find myself advocating for type hints anyway, but I must agree I often can't reply anything to my coworkers' objections, because Python type hints are truly not that useful.
What do they lack that would make them useful?
No, obviously not the same, otherwise I wouldn't be complaining. They are not even on par with Typescript, which I'm not a fan of either.
> [type hints] tell you when you make type errors
Not according to other comments I seem to be getting here. Other people are arguing type hints are not primarily for checking, but a form of notation for documentation. Seems wasteful, and I wish the Python community and tooling decided instead that they are for actually checking them.
> What do they lack that would make them useful?
Standardized, go-to tools that work in the build pipeline and that catch most errors without taking a long time to do so.
I haven't found mypy to fill these requirements. It's so bad I cannot convince my coworkers to make the effort to write more type hints.
What is the difference?
> They are not even on par with Typescript, which I'm not a fan of either.
Go is not on par with Typescript or Python, I still don't think it is okay for people to just say fuckit and `interface{}` it all and it is still shit to work with code that does use `interface{}`. At least Python with mypy has null safety, something that Java and Go does not have. There are some places it is worse than other statically typed languages, others where it is better.
> Standardized, go-to tools that work in the build pipeline and that catch most errors without taking a long time to do so.
It is mypy. What actual type errors does mypy not catch that you want it to catch? Why can't you use it in the build pipeline? I do it every day, it catches all the errors it should. The one complaint I can maybe see is the duct type compatibility complaint, and it is not really something that comes up that often for me, and definitely not something I would say invalidates the whole concept.
I'm not going to repeat myself, I already told you.
I find mypy slow, unsatisfying, inconsistent, and it fails to catch many type errors. No, I'm not going to go look in my work laptop to give you an example.
> There are some places it is worse than other statically typed languages, others where it is better.
In most places it is way worse, and I'll find it very hard to find common ground with anyone who disagrees on this.
Feel free to disagree, but I don't find this conversation useful.
Compare that to a c++ project where it’s not unusual to have 10, 15, 30 min build times.
Or even worse, the time to wait for discovering the problem runtime in prod.
The type hints also help in IDE navigation, where there is zero time to wait, pycharm can go to definition or find references in under a second.
That's locally on a fast (ish) laptop. GH CI takes 10 minutes to run tests, so I'm a little hesitant to see how long mypy takes to run.
in my experience it's a far better type checker than mypy, which tends to silently not check things without you ever realising
Counterexample: PHP is a dynamic language which enforces static type declarations at runtime.
The point is that it is not expected by definition, since by definition, static != runtime.
However, even in Typescript it is nontrivial to _annotate_ these complex types, and in most cases it's still possible to do in Python - you just need to commit to using something like dataclasses rather than pretending that what you have is a `dict`.
Basically, to get the most out of static typing in the world of Python, you do have to write more boilerplate than you would in Typescript, and in particular you need to avoid dicts and prefer various sorts of class-based data containers (again, dataclasses, attrs, namedtuples, etc).
The only one which stands out to me as non-trivial in TypeScript is thrown exceptions… and even then only because you can only type them in a catch clause, and only with unknown, and they’re invisible to callers. But then this is why people who would care for checked exceptions quite reasonably just use something like a Result type.
But I agree with your conclusion completely, and it’s basically the same one I came to reading the post.
I was curious about how the current state of things compares to TS, as I may soon have a foray back into Python soon, and would be delighted to bring some static types into the mix. Pretty much as I expected, it’ll probably be more effort/less valuable than TS, but more valuable to me than to the author.
— Consider using Pydantic (either its drop-in dataclasses replacement, or its models). It has issues and documentation is lacking in places, but in many use cases it’s a boon when it comes to being confident about data you read and/or pass around.
— Don’t use TypedDict if you foresee needing anything like optional keys, especially with defaults. I went with TypedDict and it came back to bite me later (though was relatively easy to refactor).
— Keep an eye on https://pypi.org/project/typing-extensions/, it has some useful utility types. I wish I knew about this package a couple of months back.
— If you use VS Code (which I started after switching to TS), there’re some shenanigans to be aware of. Unlike TypeScript’s elegant approach where you’re in control of installing typing versions, VS Code’s official Python extension will force-install unvetted third-party typings, not caring if they don’t match library versions in your use and not allowing to opt out. In later versions you can unset mypy in Python extension settings, and use a separate Mypy extension that doesn’t do this behind your back and doesn’t send you deep into some rando’s typings when you hit F12 to jump to definition.
— That aside, VS Code can provide development experience is not that far from TypeScript. You can keep a virtualenv nearby and select its Python bin as interpreter in your workspace, giving Mypy access to dependencies (and if VS Code’s interpreter selection GUI resolves symlinks, you can enter interpreter path by hand so that it doesn‘t).
— Sphinx documentation is good at picking up typing hints and works reasonably well with Pydantic’s models, too.
I don't think this follows by definition. It's common not to check the static hints at runtime, but a dynamic language could choose to treat them as contracts or assertions. Over in Lisp-land, SBCL will enforce static type declarations at runtime [1], contrary to most other Lisp compilers, at least under the default compilation settings (you can force it to skip the checks by compiling with a low "safety" level). If the compiler can prove that a static type declaration always holds, it will omit the runtime check; otherwise it will compile it into a runtime assertion.
What definition is that? Raku is a dynamically language with gradual typing which will enforce those types at runtime (if it cannot do so at compile time).
What statically compiled language doesn't provide a way (sometimes clunky) to have heterogeneous values in a dict? You can do that with void* in C, even
This isn't really true. For example, in Java it's Map<K,Object>. For languages without subtyping, existential types are often used. It's quite common to see in infrastructural code, e.g. caching.
Duck typing and heterogeneous dictionaries are (and have been) standard Python.
Adding a type system which doesn’t respect duck typing by trying to access the member even when the types don’t match or which can’t express standard idioms used in Python seems a poor architectural choice.
It’s not saying that Python’s type system is “too much discipline”, but rather that the type system doesn’t encode typical Pythonisms.
and heterogeneously typed ducts: https://peps.python.org/pep-0589/
Traditionally, duck typing is used to inject types into a library that isn’t expecting extension at that particular point — eg, substituting a test class for a real class in a data object that normally wouldn’t be a protocol.
I’m not seeing how that PEP addresses that use case.
- - - -
Similarly, the heading on the dictionary says “for a fixed set of keys” — but what if I want a dynamic heterodox dict? Eg, unpacking JSON.
You just end up shoving “any” all over the place. At which point, are the types helping?
Though, protocols do help the dict case for typing:
dict : Hashable -> Any
https://peps.python.org/pep-0544/#:~:text=Structural%20subty....
> substituted-for class to be defined as a protocol.
... which is false
> Similarly, the heading on the dictionary says “for a fixed set of keys” — but what if I want a dynamic heterodox dict? Eg, unpacking JSON.
You can quite obviously decode to recursive types (e.g. using pydantic), not sure what the problem is.
This section seems to agree with me, where it explains why normal classes can’t be subclassed to protocols:
> Now, C is a subtype of Proto, and Proto is a subtype of Base. But C cannot be a subtype of Base (since the latter is not a protocol). This situation would be really weird. In addition, there is an ambiguity about whether attributes of Base should become protocol members of Proto.
https://peps.python.org/pep-0544/#protocols-subclassing-norm...
- - - -
I’m not sure why you think it’s “obvious” that you can use a third party library to solve the problem — or why that addresses my complaint that the built-in type system doesn’t work for that.
If anything, the existence of a third party library hints the standard library doesn’t cover the use case.
edit: To be more precise: The article specifically mentions that accessing invalid members give you a type error, but posits that people will probably not bother and just use 'any' instead. How is that not saying 'too much discipline'?
I’m also generally pro-types, but I think it’s worth having a discussion about this type system in the context of Pythonisms.
- - - - -
If you’re using “any” for most of your types, then it’s not providing value — an untyped statement implicitly has the type “any”.
There’s a reason I brought up the JSON example: loading and manipulating JSON of varying structure is something I do a lot at work.
>There’s a reason I brought up the JSON example: loading and manipulating JSON of varying structure is something I do a lot at work.
Do you have a specific example that you find troublesome? For JSON with a fixed, regular structure (i.e. no subtyping) something like Pydantic works well, otherwise using unions + protocols with custom validation code has covered even my most esoteric use cases so far.
If we're talking about unstable JSON, a recursive union type works well enough in my experience, though defining that type gets a bit ugly if your type checker doesn't directly support recursive types.
Python types don’t support a common Pythonism used frequently in my work.
In fact, you admit that even fixed JSON can be difficult without a 3rd party library.
You’re lecturing me like I don’t understand types without realizing that you repeated my point about a gap in the current type system.
JSON = Union[
None,
bool,
int,
float,
str,
List['JSON'],
Dict[str, 'JSON'],
]
>In fact, you admit that even fixed JSON can be difficult without a 3rd party library.
Sure, but that's just because the standard library (for now?) doesn't offer deserialization with runtime validation. Java doesn't have that either, if you want to parse JSON you'll have to either roll your own or rely on a third party library.
Don't get me wrong, the situation isn't perfect, but especially if you use the 'right' tooling, you can in my experience already get a perfectly serviceable experience, which to me is indicative that the type system itself is fine and pythonic, it's just the tooling that is lagging a bit behind at the moment.
> Sure, but that's just because the standard library (for now?) doesn't offer deserialization with runtime validation.
> it's just the tooling that is lagging a bit behind at the moment
So… the standard library has exactly the problem I articulated, and you’re violently agreeing while speaking down to me.
I think that reflects poorly on you.
If you think that's violently agreeing and talking down to you, then there must be some communication barrier that I don't know how to overcome.
This does not define the type of a valid json object. (fe None is not a valid json object, neither is '')
A few concrete criticisms:
1. If you're using a dynamic language, then _by definition_ the language will not enforce your static hints at runtime. However, good news! Python has always been strongly typed, and _does_ enforce types at runtime!
2. A number of the examples given would be _impossible_ to statically type in most compiled languages (those without dependent types). It's hard to know where to go with a critique saying that you can't fully represent heterogeneous values in a dict, given that you can't do this _at all_ in many statically compiled languages.
It seems to fall back on saying "use of the type system requires too much discipline to be useful". This might be an interesting criticism on its own; however many current users of Python can say, from experience, that the required discipline is not a high enough hurdle that it prevents us from using types consistently and correctly.
There is plenty to critique about Python's static typing, but this article would have been better titled "I'm confused about Python's static type system and want somebody to show me how it's actually used in real-world scenarios".