News for hackers. Stuff that matters.
Re:How to create a Python package in 2022 by globular-toast on Thursday July 28, 2022 @06:49AM (#32262717)
Our CI pipelines invoke pre-commit. That way it's trivial to run the exact same tools locally as would be run in CI.

Running the tools locally is basically about tightening the development loop. Many of the commonly used tools (e.g. black, isort etc.) actually make the changes to the files so you'll never even commit failing versions. Do you really want to push changes to some remote CI system only to be told it's failed some boring QA check? There's nothing at all stopping you from doing that. Pre-commit is completely optional for each developer. I would just recommend it for sanity reasons.

Re:How to create a Python package in 2022 by wyuenho on Thursday July 28, 2022 @07:50AM (#32263087)
> Do you really want to push changes to some remote CI system only to be told it's failed some boring QA check?

Yes, that's what CI/CD is for.

> Pre-commit is completely optional for each developer.

No it is not, it's installed at your git pre-commit hook, and it gets run every fucking time I commit, even if I've set up my editor correctly to auto-format and lint everything at I develop, meaning 99.9999% of the time sure the code I commit will pass all these linter checks. I can use git commit --no-verify to bypass pre-commit, but then again, what's the point of using pre-commit in the first place if you need to bypass it? There is absolutely no point of linting twice locally and thrice in total just to hit the first stage of your deployment.

Re:How to create a Python package in 2022 by chucksmash on Thursday July 28, 2022 @08:18AM (#32263294)

  rm ./.git/hooks/pre-commit
Or just do not install the hook in the first place. Choice is still the developer's unless your team is doing something else here that limits your ability to delete/rename files locally.
Re:How to create a Python package in 2022 by wyuenho on Thursday July 28, 2022 @08:37AM (#32263463)
That does not alleviate the problem with difficulty having your editor to use the correct version and settings for the linters and formatters. This approach also does not address the issue of having 2 separate sets of configuration when you are trying address problem number 1 above.
Re:How to create a Python package in 2022 by globular-toast on Thursday July 28, 2022 @09:05AM (#32263739)
> Yes, that's what CI/CD is for.

Not really. CI/CD is a methodology where a team immediately integrates new changes into a trunk/release. The CI/CD pipeline is there to give the team the confidence to merge your stuff. When I see people constantly pushing breaking code into a CI pipeline I see an incredible amount of wasted time and shared computing resource. Especially if it's some trivial formatting check that you should have already done locally. You do what works for you, but tools like pre-commit were invented to save time and effort and they work well.

> I've set up my editor correctly to auto-format and lint everything at I develop, meaning 99.9999% of the time sure the code I commit will pass all these linter checks.

Then what is the problem? Have your team installed hooks that take a long time to run? Even on larger codebases pre-commit adds a negligible amount of time to each commit, unless perhaps you've touched every file in the codebase or something. Honestly your gripe with pre-commit seems mostly irrational.

Re:How to create a Python package in 2022 by wyuenho on Thursday July 28, 2022 @09:10AM (#32263782)
> The CI/CD pipeline is there to give the team the confidence to merge your stuff.

That's why linting is a part of every CI stage. Linters check your code for bugs.

> Have your team installed hooks that take a long time to run?

Yes, they are called tests.

Re:How to create a Python package in 2022 by globular-toast on Thursday July 28, 2022 @09:29AM (#32264005)
Wait, you're not supposed to run tests with pre-commit. I can see why that would be frustrating. If that's the case, your team is doing it wrong.
Re:How to create a Python package in 2022 by wyuenho on Thursday July 28, 2022 @10:07AM (#32264478)
Perhaps, but it's not just one team, and if you are doing code quality checks on CI anyway, I don't see the point of pre-commit (for most people).
Re:How to create a Python package in 2022 by doliveira on Thursday July 28, 2022 @08:25AM (#32263365)
How is it mandatory? You can just not install the hooks
Re:How to create a Python package in 2022 by wyuenho on Thursday July 28, 2022 @08:35AM (#32263442)
1. Because it defeats the purpose of pre-commit.

2. pre-commit install installs the hook by default.

3. If you happen to fail linting a few times a year, there's always an anal coworker telling your boss in a 1-on-1 you are not following some BS ways of working.

Re:How to create a Python package in 2022 by doliveira on Thursday July 28, 2022 @08:59AM (#32263678)
> 1. Because it defeats the purpose of pre-commit.

No, it doesn't. Pre-commits are just a DX bonus so you won't have to wait for the CI/CD server.

2. pre-commit install installs the hook by default.

Am I missing something? If you run "$ pre-commit install" of course it will install the hooks. Do you mean that poetry install, setup.py themselves are making the magic of installing the hooks themselves? If so, it's indeed a bad practice.

3. If you happen to fail linting a few times a year, there's always an anal coworker telling your boss in a 1-on-1 you are not following some BS ways of working.

I don't get why anyone would care, unless you're polluting the CI/CD history with tons of tiny commits all day along just to run linting.

Re:How to create a Python package in 2022 by fnord123 on Friday July 29, 2022 @04:19AM (#32274355)
Filling the CI/CD pipeline with tiny commits to fix linting when it should be done before commit or push is why pre-commit exists. It's not BS, clogging the job queue shits in everyone else.

Of course I don't know GPs specific context so they may have other details, but generally it's

Re:How to create a Python package in 2022 by wyuenho on Friday July 29, 2022 @12:46PM (#32279191)
Except this problem rarely comes up. Everyone has their favorite way to set up their editors to lint and format automatically using the project's config.

Over my career, I see that every company has some version of:

1. New guy joins a team, wants to push code ASAP

2. He sets up his favorite editor and punt configuring linters and formatters

3. Code pushed, CI fails at linting

4. Some well-meaning coworker or new guy suggests some variations of husky/pre-commit/fancy git hook scripts

5. Team agrees and put that into the repo

6. 6 months to 1 year later the entire team realizes its benefit does not out-weight the cost, and unanimously agree to rip it out unceremoniously.

Re:How to create a Python package in 2022 by fnord123 on Friday July 29, 2022 @05:01PM (#32281732)
Well I have different experiences where no one would ever go back to living without pre-commit, black, autoflake, etc. It moves the fixes to before they get to CI. There's literally no downside.

Maybe you have experience with some weird tool? Or running black and autoflake messed around with emacs and it needs to reload all the buffers?

Re:How to create a Python package in 2022 by wyuenho on Friday July 29, 2022 @05:29PM (#32281990)
I initially had some problems, then I've written an emacs package[1] to fix all of them. And it's due to the course of writing this package made me realize just how bad pre-commit is from its UX, design to its entire premise.

[1]: https://github.com/wyuenho/emacs-python-exec-find

Re:How to create a Python package in 2022 by fnord123 on Saturday July 30, 2022 @04:58AM (#32285348)
Ok then I think the issue is not so much pre-commit but that python tooling in general should not be written in python. It's a minority opinion but the fact that there is a tooling python and an application python can mess with things as you're experiencing.
Re:How to create a Python package in 2022 by doliveira on Friday July 29, 2022 @08:01AM (#32275843)
Yeah, I agree. It's just that they can just run pre-commit manually if the hooks bother them that much
Re:How to create a Python package in 2022 by fnord123 on Thursday July 28, 2022 @09:02AM (#32263717)
Tell me you use Emacs without telling me you use Emacs.
Re:How to create a Python package in 2022 by doliveira on Thursday July 28, 2022 @07:26PM (#32271287)
Curious, how does using Emacs affect that much the workflow to the point of pre-commit being so annoying?
Re:How to create a Python package in 2022 by globular-toast on Thursday July 28, 2022 @09:28AM (#32263993)
Not sure why, though? I've used emacs for 15 years and pre-commit for probably 5 years at this point.
Re:How to create a Python package in 2022 by fnord123 on Saturday July 30, 2022 @05:57AM (#32285546)
Are you using git inside Emacs or from the command line? It seems GP is doing everything in Emacs and application python resolution is tripping up with tooling python.

If you have it all sorted out from Emacs maybe check the repo they posted which tries to do all this.

Re:How to create a Python package in 2022 by wyuenho on Thursday July 28, 2022 @09:16AM (#32263860)
Guilty
Re:How to create a Python package in 2022 by Too on Friday July 29, 2022 @02:24AM (#32273777)
Does it matter if it’s run twice? Usually it’s so fast you hardly even notice it.

If your pre-commit takes more than 3 seconds to run it’s set up incorrectly IMO, and should belong to a more manually (and CI of course) invoked test suite instead.