With Python, the only sane choice is probably through Docker images :-(

This is the reason why I consciously attempt to move away from Python and choose Go or Rust for new projects if possible. Of course, on existing projects, Python deployment is a pain.

> the only sane choice is probably through Docker images

Its really not. You either tailor your dependencies to match the host (clue, you should be doing this anyway, it makes things much easier in the long run), or use virtual envs.

pyenv also gives you some (non obvious) flexibility as well

However, having said that, the chances are, your go or rust binary is going to be in a docker env as well(so is your python), so its basically all the same.

I'm personally a big fan of (especially) the most recent builds of Python 3, but lately I've been interested in learning Go or Rust just because there's a few things I really like about them both vs some other languages I've looked at.

So, seein' that you've coded in both Go and Rust apparently, my question to you is: Which do you prefer of the two (and why)? I personally lean a bit toward Go, but I haven't learned enough of either to decide absolutely which of the two I should learn first.

I think it is better to not _decide_ which one to learn. They both have their perks and quirks. If you can, I’d recommend to dip your toes on both. For network services, Go feels super nice because of the high quality libraries in its ecosystem. I feel like it is better Java.

Rust on the hand, feels like a better C++... with a steep learning curve, and useful when you can/need to get something correct/efficient/safe at the cost of increased developer time and cognitive load.

They both have their place IMHO.

Nice. Thank you for the info. Do they both have pretty good GUI toolkit support available? I currently use PyQt5 in Python, but I'm not totally against GTK if that's what is easier for Go or Rust. Still think I kinda feel like Go's the one of the two I'm prolly gonna learn first, although you've got me leanin' hard toward also learning Rust in addition to Go.

tarballs, rpms or debs also work.

One should probably run their own package server like https://github.com/pypiserver/pypiserver

All of that said, containers are nice because you have a log of what is running, easy to transport and coordinate.

When you use Go and Rust over Python, does the use of Docker disappear? What replaces it?

> One should probably run their own package server like https://github.com/pypiserver/pypiserver

Never used pypiserver but I’ve had a good experience with https://github.com/devpi/devpi

Why is deploying through docker images not OK? I always thought that's what docker is meant to solve.

Desktop/local usage, especially for non technical users or people working in environments where they can’t install docker (eg locked down corporate machine). There isn’t a great way to pass around a single executable or installer for a Python app. There’s pyinstaller, but it’s finicky and doesn’t quite work cross platform (you have to build the executable on the target OS).

On the other hand, OP’s setup makes it very easy to publish packages. So you can create a tarball and have a user pip install that, then run your app with a simple CLI. Or publish to pypi if you want it public. The downside is you assume the user has the right version of Python and knows how to switch versions if need be and all the weirdness that comes with that. But for web apps, packaging your app also makes it easy to wrap in a simple dockerfile that basically just installs the package and then runs it.

If you can't run docker, you're just as likely not to be able to run arbitrary executables.

In every modern corporate environment I've been in, you either have the ability to run whatever on your box, or it is locked down tight.

Yes, but if you sign the executable with an acceptable certificate (the company’s cert or some other trusted CA), you can install an arbitrary executable usually. Docker would typically be blacklisted, however. In most corporate IT environments, it’s much easier to get a small, focused executable white listed or code signed than to get something with such a huge attack area as docker approved for regular, non-technical users.

> There isn’t a great way to pass around a single executable or installer for a Python app.

Have you tried Nuitka or mypyc? (Haven't tried them myself but I've heard very good things about Nuitka.)

Also Briefcase: https://beeware.org/project/projects/tools/briefcase/. "Convert a Python project into a standalone native application."

Personally I have not. I have only had one particularly weird use case where I really needed to deliver a Python executable alongside an Electron app, and made it work with Pyinstaller. We’re actually reworking things to use Pyodide to compile the Python to WASM so we don’t have to deliver any separate Python code at all. It’s a real Rube Goldberg machine of an application, but we had a lot of weird constraints we had to meet (eg, we couldn’t use sockets to communicate between Python and Node, so had to package the Python as an executable). It works beautifully but it was a mess to get running.

Docker was made for development and testing of code, not production. It has been co-opted for production with good success. It seems fine to do it for hobbyist projects, but still feels dirty in a commercial setting.

The largest internet services on the planet were using LXC containers for production ops before you ever heard of the term 'docker'. In its earliest iterations it was easier to push a container to production than to run it on your laptop.

> using LXC containers for production

which isn't docker.

From what I recall, spotify were the first company with a large footprint to use docker. However for some reason they skipped VMs and went screaming into docker when it was _very_ new. Personally that seemed like a mistake, but you know, each to their own.

If you're wanting to get into a bun fight about containers, then IBM 360 and JCL has some time for you.

I'm using docker in commercial environments for years and cannot complain... But I don't distribute software, which is where the problem is probably more visible.

If we stopped using software in prod because it feels "dirty" we would not be using computers at all.

Why?

Why what?

If you're deploying a REST API with authentication, caching and so on then the KISS solution is python with a framework (such as Django) in a docker container.

I've spent many many hours (years) trying nix, rust, Haskell, go, spring framework and all sorts of other things which are a lot of fun but not so good for getting shit done.

For other domains this doesn't apply of course; lower-level network stuff, portable CLIs, CPU-intensive workloads and so on are much better in go/rust but you can either integrate them with a network call, spawning an OS process or an FFI in the case of rust.

I'm using Pycharm Pro and it can use Docker for code completion. I use Docker AND venv on host filesystem for friends using Pycharm Community and VSCode. Is this still necessary?

Last I've heard, the nix experience for python was kind of lacking, sadly - but I don't have any first-hand experience to go with that statement.

I wouldn’t write off platforms such as Elastic Beanstalk and Heroku for web projects, smaller ones of bog-standard shape anyway

Heroku, yes I would push for that. Its expensive, but it'll save you an entire devop function right up til you hit the 30 people mark.

Elastic beanstalk is just a horrid dev environment. Lots of waiting, lots of non-obvious options, and very little reward. I would personally push for lambda and zappa (https://github.com/zappa/Zappa) for python, as it seems to be much easier to deploy and debug.

If this is a company project, a REST API or task service of some kind, then you are probably using Docker in the year 2021.

If this is anything else, or if you work for a shop that hasn't embraced containerization, then you use PyInstaller (http://www.pyinstaller.org) to bundle your application. Either into a directory that contains your full Python virtual environment (only 5-10 megs!), or into a single executable file.

The latter is most convenient for a Go/Rust type experience. But the former will startup faster, because that single-file executable has to first uncompress itself to the system temp directory.

Most production deployment goes like this:

    1. Put files on a server,
    2. Start a process.

How much ancillary scaffolding you put around that is a matter of taste.

If your Python code has native code under the hood (eg. Numpy) this two-step is very much a 'step1: circle, step 2: draw the rest of owl' description of production deployment.

If you're deploying to a server you'll want to pip install dependencies on the server. If you're using docker you'll want to pip install dependencies in your container. If you're deploying to an end users computer you'll want to use pyinstaller, which admittedly is not trivial to get working in all cases.

This is exactly my point. .venv/ needs to live in the project deployment directory, and needs a wrapper script to use the python binary/symlink therein.

Isolate the project from the python runtime it uses, and you'll always have the right set of packages installed.

None of this is perfect, but an in-tree .venv/ and convenience scripts seems to be the least-worst option.

I was overly brief here.

I meant to say "I look to see how they are going to deploy and use the virtualenv they have created."

Hint: what works in development -- just telling people to type "source .venv/bin/activate.sh" or such, doesn't fly in an unattended environment.

All that it requires, of course, is a bin/venv-python wrapper (bash) script to reference the created .venv/ directory, so this is hardly ground-breaking stuff, but as I mentioned originally, this (crucial) section is missed every time.

I use a minimal-but-complete pairing of venv and pip, and a couple of location-independent wrapper scripts, and I can run things the same across all environments.