This is a solved problem across a variety of popular and mainstream programming languages. I don't mean to suggest that the problem isn't complicated, but this isn't a problem that doesn't have a wealth of previously written solutions to look at for inspiration.

My Java app declares its dependencies in a build.sbt file using Scala syntax and has them cached in an Ivy directory. Yours declares them in a pom file using XML syntax and has them cached in a Maven directory. Neither of us even tries to do lockfiles and instead just has the CI server build an "uber jar" with all the dependencies bundled in.

I get why Python needs lockfiles, but goddamn, that need is a symptom of the mess of managing Python dependencies.

There's still a long way to go - I use Airflow for ETL management, and I'm using pipenv to manage it - except Pipenv can't create a lockfile because a dependency of a dependency of Airflow requires in flask-wtf <= 0.7, while another dependency of a dependency requires ==0.8

In a Maven pom.xml I can easily exclude or override the conflicting dependency manually if needed, but I can't in a Pipfile.

Well, that seems naive. Almost 1/4th of Maven Central libraries broke binary compatibility in patch updates: https://avandeursen.com/2014/10/09/semantic-versioning-in-ma...

Netflix, at least, doesn't agree that Java doesn't need lockfiles: https://github.com/nebula-plugins/gradle-dependency-lock-plu...

Yes, which is part of the reason why lots of Java projects almost never update their dependencies, because no one remembers why that version was chosen. Splitting what you want from what you have is important to communicate this.

> There's still a long way to go - I use Airflow for ETL management, and I'm using pipenv to manage it - except Pipenv can't create a lockfile because a dependency of a dependency of Airflow requires in flask-wtf <= 0.7, while another dependency of a dependency requires ==0.8

I'm not sure what you expect it would be able to do here...

Most shops I know use one of maven or gradle. I can’t think of any other serious contenders in Java (if you still use ant: please consider alternatives).

Poetry looks like it did just that though and I'm warming up to it at a very high speed.

Boot, and Lein. I'm partial to boot lately.

You wouldn't write your application to support 5 versions of Django, but you _probably_ would do so for a library.

That said, I do basically agree about `pip` existing already. We could have built a tool to manipulate `requirements.txt` files instead of introducing another format and a toolchain that is _much_ slower and brittler. Though ultimately at this point Python packaging woes feel like they are at a much lower level (the fact that libraries end up being "installed" mean that preparing all your dependencies to go out to multiple servers is a mess).

There's pip-compile from https://pypi.org/project/pip-tools/ that does exactly that. Pipenv uses its resolving mechanism if I'm not mistaken. It produces standard req.txt file with versions pinned and supports separate dev requirements. It had some bug with upgrades last time I checked though, not sure whether it's resolved, currently considering using it for projects at work.

Since its all just requirements files with pip-tools, its been fairly commitment free, in that I haven't had to substantially rework any major bits of workflow to use it - nor would I to remove it. Not sure I could say the same thing about pipenv and/or poetry.

The "library" workflow works for applications too. Put your direct dependencies in setup.py. Build wheels of everything and upload them to an internal PyPI server. Pin everything in requirements.txt.

That would be weird. Perhaps you mean 'flaunting'?