(where "not otherwise typecheckable" means types that can't be expressed with stubs - e.g., Django, dataclasses pre-PEP-681, pytest fixtures, etc.)
(It's also more difficult to support plugins effectively in a type checker like ty, than in a linter like ruff, since any non-trivial use case would likely require deep changes to how we represent types and how we implement type inference. That's not something that lends itself to a couple of simple hook extension points.)
Considering how fast uv and ruff took off, I am sure you are aware of the impact your project could have. I understand that supporting plugins is hard. However, if you are considering adding support for some popular libraries, IMHO, it would be really beneficial for the community if you could evaluate the feasibility of implementing things in a somewhat generic way, which could be then maybe leveraged by third-party authors.
In any case, thanks for all the amazing work.
I don’t have any such experience (short of a macro system, which requires code generation or runtime support) and it always makes me curious when people ask for type system plugins whether this is a standard feature in a type system I’ve never used.
So if we were to do this for ty, we would have to carefully design the internal data types and algorithms that we use to model Python code, so that they're extensible in a robust way.
But we would also have to decide what kind of Rust plugin architecture to use. (Embed a Lua interpreter? dlopen plugins at runtime? Sidecar process communication over stdin/stdout?)
Solvable problems, to be sure, but it adds to the amount of work that's needed to support this well — which in turn affects our decisions about whether/when to prioritize this relative to other features.
For years I pushed black for formatting code. Once formatting was baked into ruff I ditched black. Having fewer dependencies to track and update simplifies my life and shortens my dependabot queue.
Is this all based off a spec that python provides? If so, what does that look like?
Or do you "recode" the python language in rust, then use rust features to parse the python files?
Regardless of how it's done - This is a really fascinating project, and I'm really glad you guys are doing it!
https://github.com/python/cpython/blob/main/Parser/Python.as...
ty uses the same AST and parser as ruff. We don't use the ASDL grammar directly, because we store a few syntax nodes differently internally than how they're represented upstream. Our parser is hand-written in Rust. At first, our AST was also entirely hand-written, though we're moving in the direction of auto-generating more of it from a declarative grammar.
https://github.com/astral-sh/ruff/issues/15655
https://github.com/astral-sh/ruff/tree/main/crates/ruff_pyth...
https://github.com/astral-sh/ruff/blob/main/crates/ruff_pyth...
CPython uses a generated parser. The grammar is defined in https://github.com/python/cpython/blob/main/Grammar/python.g... which is used to generate the specification at https://docs.python.org/3/reference/grammar.html#full-gramma...
We use a hand-written parser, in Rust, based on the specification. We've written that previously at https://astral.sh/blog/ruff-v0.4.0#a-hand-written-parser
No, they are correctly using semantic versioning to indicate pre-alpha releases. https://github.com/astral-sh/ty/releases https://semver.org/
What am I missing here?
For example, as my recent struggles showed, SQLAlchemy breaks `pyright` in all kinds of ways. Compared with how other 'dynamic' ORMs like Prisma interact with types, it's just a disaster and makes type checking applications that use it almost pointless.
How does Ty play with SQLAlchemy?
If you know the SQL you want it's just a matter of writing it in SQLAlchemy's query language which is quite close to SQL. Should just be a matter of practice to become fluent in it. "Complex queries" usually turn up when you're doing something like rendering a table or report or something. You don't need the ORM for this kind of thing, just write a query.
An ORM is useful when you want to write domain logic to do read/write operations against domain entities and persist them back to a database. IMO people get hung up on ORMs and think if they're using one then they have to use it for everything then do the most horrible contortions that should have just been db queries. SQLAlchemy allows you to use the ORM judiciously.
Good documentation should absolutely provide a usable reference to quickly look up common ways to solve common problems. Even the PHP docs got that right twenty years ago.
Also, I disagree: A library should be as self-evident and incrementally understandable as possible, not require reading a full tome and grow a grey beard before being accessible.
> "Complex queries" usually turn up when you're doing something like rendering a table or report or something. You don't need the ORM for this kind of thing, just write a query.
Or, when building generic filtering/sorting/pagination logic for a bog-standard CRUD app. Or to do full-text search. Or when doing lateral joins to minimize queries. Or to iterate over a huge table. There's lots of cases where I want the ergonomics and malleability of ORM query instances even when working with complex queries.
In defense of OP, a particular frustration I have with SQLAlchemy is that I understand SQL just fine, but the ways in which I translate my SQL knowledge into SQLAlchemy incantations is often pretty obscure. I think I deserve "quick answers to do very specific things" because I already have the exact form of the SQL solution in my head. That it then takes 20 minutes of digging through docs or ChatGPT is annoying.
Ty could solve this if they rebel and decide to ignore the Python typing standards, which I honestly would appreciate, but if they take the sensible approach and follow the standards, it won't change anything.
I've been typing them with TypedDict for a while now and it's been fine. What can't you do?
I loathe the Python convention of just using kwargs instead of clearly annotated parameters; most libraries don't even have doc comments in the code, so you're really required to look up the documentation, hope that it actually describes the method you're interested in and contains more than stuff like "foo: the foo to use"—or fall back to rummaging in the library intestines to figure out how it works.
It's pathetic.
Maybe you're in a niche spot, or using scientist-based code. I've seen plenty of trainwrecks in 'conda-only' ""libraries"" done by scientists. Maybe that's the niche you're at?
On a more serious note, I can't even blame library devs as long as they try. Type "hints" often are anything but _just_ hints. Some are expected to be statically checked; some may alter runtime behavior (e.g. the @overload decorator). It's like the anti-pattern of TypeScript's enums laid out here and there, and it's even harder to notice such side-effects in Python.
I don’t know about SQLAlchemy, but for libraries like pandas I just don’t see how it can be done, and so people are actively replacing them with modern typed alternatives
I love how Python makes me so much faster due to its dynamic nature! Move fast, break things!
Pure Python that is.
Looks like you found the not-so-secret repository we're using to prepare for a broader announcement :)
Please be aware this is pre-alpha software. The current version is 0.0.0a6 and the releases so far are all in service of validating our release process. We're excited to get this in people's hands, but want to set the expectation that we still have a lot of work left to do before this is production ready.
Stay tuned for more for news in the near future!
(... I work at Astral)