The README.rst documents the `egg` subcommand like this:

> Creates a new Python project. Think of an "egg" as a new idea.

How does this differ from the egg distribution format [1] introduced by setuptools, which wheel aims to replace? If it's completely different, couldn't a distinct term be used? Python packaging is confusing enough as it is!

[1] - https://packaging.python.org/glossary/#term-egg

From what I can tell, the egg subcommand doesn't create any kind of egg file, just creates a new project.

But I agree. Creating a new distinct keyword for that command (why not just `hatch new my-app`?) would be far better. I've tried to get into Python packaging, and it's the biggest PITA I've dealt with as a dev.

I can recommend this great video from PyCon 2017 which demystifies a lot of Python's packaging process: [1]

But it's still a mess only Rube Goldberg could love.

[1] - https://www.youtube.com/watch?v=qOH-h-EKKac

0.4.0 changed the name from `egg` to `new` https://github.com/ofek/hatch/commit/5649b3f6c452b670d919fae...

Thanks ofek, seemingly small things like this make a huge difference.

I truly appreciate the feedback :)

I agree. It is definitely a completely different thing and even though 'egg' is a cool word I wish it was just 'hatch create my-app'

I don't want to be suspicious and negative but here is yet another needless dependency without any real value. mkvirtualenv is the best tool to manage venvs, pytest with pytest-cov to test and there is so much benefit to know original instruments but not some inferior wrapper. If you bothered by bugs in pip it's better to fix it in pip. All community will be grateful. But another wrapper is a complete shame.

I don't understand this mindset. There are millions of python developers using this language in for everything from from small shell scripts, web scrapers, to apps and. machine learning models.

Yet there is a monolithic additude towards tooling for languages, so much so that experimentation is actively discouraged. In my experience, there is no better way to level up as a developer than releasing and supporting an actual product that solves what you see as problems. Sure, you can contribute to existing projects, but you're going to learn a lot more if you are the owner. This experience, in and of itself, makes it worth it. I don't understand the existential fear and admonishment since the 'superior' tool will be most widely used. Also, there are many more developers today of all stripes and skill levels, more than enough to support the existing toolset, and others that like to play around and experiment with different projects for fun. This is somewhat how biological evolution works... lots of experiments(mutations), almost all are dead ends, but sometimes, that experiment leads to a wildly successful adaptation.

I only have about 4 years experience, but I've never understood this mindset. I see the value of having a primary toolset for a language, but don't see the value of having onw and only one solution. Can someone please explain to me its value?

I use Python daily and have never heard of mkvirtualenv. I started using pyenv a couple years ago and it worked well enough that I stayed with it. It's really a shame that there is no source of truth for packaging in the Python world.

mkvirtualenv is part of virtualenvmanager. I'm sure other ways are good, it's my favourite (along with it's peers rmvirtualenv lsvirtualenv).

I think you might mean virtualenvwrapper.

Nothing negative in pointing out the constant pattern of reinventing the wheel that has been damaging Python for more than a decade.

I agree as well. Conda tackles a few of the issues that OP is trying to solve.

https://conda.io/docs/

what are the differences between these things:

pip

easy_install

virtualenv

mkvirtualenv

pipenv

pyenv

Found a partial answer here (doesn't go into pip and easy_install, those handle installation, the others handle environment isolation) - https://stackoverflow.com/questions/41573587/what-is-the-dif...

pip: Python's official package manager

easy_install: Forget it existed, substitute with pip

virtualenv: A virtual environment, providing an isolated Python and libraries

mkvirtualenv: A script to create a virtualenv

pipenv: Combines pip and virtualenv into an integrated solution, comes with a Pipfile not unlike Ruby's Gemfile for example

pyenv: Script that helps to install and manage different Python versions on your system, so you can have the full range of Python 2.x and 3.x installed without them getting in the way of each other. Allows you to set local, default and system Pythons allowing you to switch your Python interpreter depending on which project/directory you're in

pyenv also has pyenv-virtualenv...

By itself, pyenv is only responsible for isolating python versions (e.g. 2.6, 2.7, 3.3. this would be like rvm/rbenv/nvm).

pyenv-virtualenv is a wrapper to allow pyenv and virtualenv to work together

I've stopped using any other virtual env managers than asdf[0]- it has one simple API for pretty much everything.

You can define a .tools-versions file in the root of your project, which lets you install the correct versions with a simple `asdf install`. We even use this in CI for zero-config language version upgrades.

[0]: https://github.com/asdf-vm/asdf

I read "asdf" and wondered, "Using Common Lisp to configure Python?" Then I followed the link and am curious whether or not it is a port of the widely used (in) Common Lisp tool.

https://common-lisp.net/project/asdf/

(Author here) I'm happy to answer any questions!

What does it do?

I started reading the readme, and after getting past general "this is awesome", I got to the doc of the config option...

How about showing an example usage, which does something simple and cool?

Agreed, a quick-start example at the top would be both helpful and engaging.

Working on that now!

Basically, it combines the core features of many disparate tools into a handful of commands so users have to type less things.

If I had to say what is truly novel here, it would be the virtual env management and shell support.

This looks interesting and potentially really helpful! I have been keeping my eyes open for something along these lines. As the author, how do you see this project's relation to Conda? And if you see it working alongside of, rather than in place of, Conda, what support does it have for recognizing and using Conda envs?

Thanks for the kind words!

Hatch can replace Conda as a virtual env manager but doesn't really try to compete with it in regards to binary installs.

Hatch does try to detect when a Conda env is in use and will adjust the shell prompt accordingly. https://github.com/ofek/hatch/blob/59cfdd4a929240dd82b2e4e25...

Do you support non-python (especially binary) dependencies? Suppose I have an application that ships binaries but uses python as a scripting language.

install/uninstall/update just uses pip, so yes!

Could I reconfigure it to use conda instead? Conda can install non-python applications but pip isn't very good at that.

Not at the moment, but what an interesting idea! I'll look into doing that.

Nice to see Python 2 compatibility. I started using poet[0] recently, and faced issues once I tried adding Python 2 support.

[0]: https://github.com/sdispater/poet

Unless I'm mistaken, Hatch is only 3.5+, but contains a command line option to specify if a project is 2 compatible.

Yes, the tool itself is 3.5+ but what you are working on matters not!

Oh, that wasn't clear when I originally looked. I'm glad you clarified that.

Thank you for clarifying.

See also: pipenv :)

http://docs.pipenv.org/en/latest/

Very different tools, but they cover the same surface area, in spots.

Friendly competition {^.^}

Well I was in the process of writing my own tool to do this after starting to hit the limits of what I could do with cookiecutter... so I'm hooked.

I expect I'll be writing quite a few pull requests.

This looks cool! Two things that leap to mind as things I'd like to know after glancing over the readme:

- comparison with pipenv

- how it handles things like `pip freeze` for, e.g., Heroku

Thanks and great question! As Hatch does a bunch of different things, I'll just speak to the package management.

More or less, hatch manages packages based on environments, whereas pipenv does so based on projects.

When pip supports Pipfiles, hatch will as well. At that point, project based package management will be supported too.

Cool, thanks!

Fyi, I just significantly uncluttered the readme

Hmm, I'm not sure if I like the idea of a development tool dictating my folder structure.

Go does this.

Looks very interesting! Are there any plans to support conda?

Thanks!

Eventually, yes.

How is this better than using a Docker image?

Developing inside a docker image seems like needless complexity to me, especially when virtualenv takes care of the python environment isolation already without another layer of indirection. Virtualenvs are just file copies and PATH vars with some tooling.

Maybe if what you are building runs comfortably in one app, on one server, with no important discrepancies between your dev system and production.

The environment I work with has docker containers for

    Web App Server
    Background Task Worker Server
    Database
    Redis Node
    Cassandra Node

Getting all of these to work nicely together on linuxes/macOS/Windows in a repeatable fashion across multiple developers is a major headache. Being able to spin the whole cluster up with `docker-compose up -d` turns the whole mess into merely a minor headache.

Most of the time, I only need to work on functionality that depends on DB and WEB nodes, but if I ever need to debug some issue with the worker tasks, it's much easier to have a whole working system available.

Agreed. If you're on Windows/OSX, it also adds a VM between your host and your code and file sharing has terrible performance.

File sharing performance has gotten much better on OS X, at least: https://blog.docker.com/2017/05/user-guided-caching-in-docke...

It's still not /great/. I have had to switch from my mac to a new Win10 laptop for development because the app server in my docker container would take around 2 mins to detect a change and reboot the app server, which then would take a few minutes to reboot. Now it's much closer to realtime. I do hope that docker get's it sorted out soon, as the tooling on windows isn't as good as what I am used to on osx.

And by tooling, I mean command line. I am using Cmder with Linux subsystem for Windows, and it STILL isn't half as good as even the terminal built into osx, let alone iterm.

Docker might replace the need for the virtual environment, but doesn't manage the components of a Python project (building, testing, installing and updating dependencies, etc).

Plus, a virtualenv is a much simpler tool, it's essentially just a way to tell the Python interpreter from which directories to load modules, so it's less likely to give you problems.

It doesn't. Imagine you just want to run some utility written in Python (let's use youtube-dl since it is been on front page lately), why do I need to install Docker to run a simple command? For isolation? No thanks. Virtualenv was created to isolate overwrittkng and installing dependencies.

Also, want to emulate multi-sites on single machine deployment? You need virtualenv.

Docker does solve those problems, even if you and I prefer to use other solutions.

And by the way, you don't actually need virtualenv :) we run multiple webapps on a machine by just installing dependencies in a directory (pip -t) in the git repo, and then adding the appropriate PYTHONPATH in the service's startup config. As long as you don't need multiple versions of the Python interpreter, this works fine.

> Docker does solve those problems

...with a very computationally expensive solution, and without improving other aspects like security.

A systemd unit on a "chroot" can do the same and provide isolation.

Is it really computationally expensive? Containers/jails/zones are essentially chroots "on steroids", isolating things other than the filesystem (networking, hostname, process IDs, users, etc.)

Once built a docker container is not expensive, but the idea with Docker is to create immutable, layered containers and rebuilding every time there is a change. Unlike with nodejs, it's harder to volume mount python packages installed in the docker container and so no one does it.

Change in requirements means rebuilding the container which is a very computationally and IO expensive process. So if you can get away with it virtualenv is better, but putting everything in a container so you can simply ship a container is also useful. It's a compromise.

Rebuilding the container shouldn't be expensive, that's what layers are for.

A layer with your pip requirements being added and then installed is the most expensive part of building a python container so if you change your requirements you invalidate the cache for that layer (and possibly the other layers above).

Looks like it does a bit too much. I like pipenv https://github.com/kennethreitz/pipenv

My thoughts exactly :)

Aside : Can someone explain how HN rises topics to the top? (I'm genuinely curious)

As of writing this comment, this submission has 4 points in 3 hours, and is on the top half of the front page. (Not saying the project doesn't shouldn't be - looks pretty awesome tbh).

Does the reputation of the people upvoting it matter? (still surprising it'd rise given only 4 points in 3 hours). Does the reputation of the person submitting it matter?

Read the FAQ please: https://news.ycombinator.com/newsfaq.html. A quick Google would've gotten you to that answer too.

And don't use random topics to ask questions like these, that's what Ask HN can be used for: https://news.ycombinator.com/ask

Not sure but I feel they've fine tuned it more so that posts like this one don't go by completely unnoticed. I also believe the comments are factored into the ranking as well.

I assume reputation, source, and subject, matter. Though I think they are purposely vague on the details to make it harder for people to game.