My team started with an untyped codebase a year ago. I added a rule to CI that every pr that touches files with type errors must decrease the type errors in touched files by at least 5. When we started we had like 30k type errors. A year later and it’s about 5k left. This was a small wrapper script over mypy/pyright that just called them twice once on master and once on your branch to compare error counts.

I did occasionally get pushback on it but stricter checks will be inconvenience at first and I argued it’d help over time. Now that we’ve had it a year it’s pretty well accepted in same way many would accept run formatter.

The problem with these sorts of "rules" is they create needless changes outside of the PR's primary focus. The PR's purpose is to do X: add a feature, fix a bug, whatever. Now because I've done X, you want me to do Y, a set of totally unrelated changes that ultimately are a distraction.

Why wouldn’t you submit two different branches to review? One where you fix errors for types and another for the feature with the fixes:

    (git main) > git checkout -b abc-1234/type-fixes
    <do work on types and commit it>
    (git abc-1234/type-fixes) > git checkout -b abc-1234/feature-branch
    <do work on feature and commit it>

Then you can submit the feature branch as a PR on top of the type fixing branch and a PR for that branch to main. You only see the type fixing changes on that PR and only see the feature changes on its PR.

Seems fine to me, but in my experience these “rules” are often tied to a single PR and various automated CI checks.

Mypy can be configured to ignore modules based on their names.

Though honestly I think it’s worth it to just bite the bullet and spend a full day or two to go through and fix every single error. After this you will have a much easier time navigating code base. I mean, your “new code” is probably modifications to your old code or calling your old code right? So you want to have at least the boundary between new and old typed. It’s quite satisfying anyway, as you will likely learn and discover tons of bugs on the way.

With any large legacy Python codebase, that's usually a month-long undertaking for one engineer, if not longer!

Python by design encourages duck typing, which means that you'll have plenty of places that simply take multiple different types, and just slapping a union is usually non-trival and not very beneficial either (eg. you'd be randomly patching things with fakes and mocks in tests).

If you think it's a single day job, I am sure my company (and plenty others) would pay you gladly your single daily rate to get our codebase migrated in a day :-D

> day or two

Your code must be quite small.

For any large code base, it'll go on for months, since it gets done between higher priority work. You'll discover some problems, sure. But if they were really important, you'd have hit them earlier. My experience is these sorts of efforts create a lot of busy work and not much more.

You can pipe the diff into specific flake8 checkers. Disable the check by default so it's not run over the entire codebase, and have a separate line for running specific checks on just the diff. Eg:

git diff -U0 --relative origin/master... | flake8 --diff --select <custom-codes>

I'll leave the flake8 plugin as an exercise to the reader as ours is intertwined with code I can't share right now.

The same (and only) way you’d reasonably be able to do it in any other language that uses a series of text files for an input.

File by file.

I generally force it on a per-file basis by adding

     # mypy: disallow-untyped-defs 

on top of new python files.

You can use flakehell for that.