News for hackers. Stuff that matters.
Re:Pyinfra: Automate Infrastructure Using Python by posix_monad on Tuesday April 30, 2024 @01:40PM (#40213688)
Python seems like a really poor choice for infrastructure.

- Python is not easy to build into portable binaries

- The package ecosystem is very hard to use in a reproducible way

- The language is not truly typed - types add massive value for infrastructure and scripts because they are less likely to be unit-tested

- The lack of a "let" or "var" keyword makes simple programming errors more likely (again, this code is less likely to be unit-tested)

Maybe I'm missing something? I don't know why I would want to introduce Python in this domain.

Re:Pyinfra: Automate Infrastructure Using Python by exceptione on Tuesday April 30, 2024 @04:03PM (#40215642)
I 100% agree with your points.

No type checking = no serious job. I have learned enough from Ansible to not ever touch that kind of stuff again.

There has been a time that Python was a fringe language, only known by some hardcore nerds. I thought Joel Spolsky had once mentioned that having Python on your resume was a signal of a quality developer, someone who went off the beaten path.

Times have changed. Python is now the MS Excel for developers. It shines for quick and dirty data mangling. Unfortunately, that is how a seizable portion of people approach software engineering. My theory is that for some having to do abstract thinking and perform a dry analysis beforehand is an impediment. They can only discover what they want while banging out something. They fix the runtime errors they could catch, and slap some more features on top.

Types imply a kind of foresight, and that is what some people really have difficulty with.

EDIT: Might sound negative, so I admit that the quick feedback cycle you can get from an interpreter language like php/python is a feature in itself.

Re:Pyinfra: Automate Infrastructure Using Python by DandyDev on Tuesday April 30, 2024 @02:43PM (#40214609)
Why is it important to be able to build into portable binaries? Pyinfra doesn't require running Python on the machines you manage. Pyinfra basically turns your Python code into shell commands which it runs over SSH. So only your development machine has to run Python.

I think there is not a lot of overlap between people who need to automate infrastructure and people who don't know how to install Python on their development machine.

As for your other comments regarding Python as a language: I mostly agree. I have stepped away from Python as a language to develop production software. In Python I miss the confidence I get from static typing. Having said that, for automating infrastructure, you're effectively comparing Pyinfra and Python to bash scripts and YAML (for things like Ansible), which are both orders of magnitude worse if you like static typing or any form of being able to verify what you wrote.

Re:Pyinfra: Automate Infrastructure Using Python by yjftsjthsd-h on Tuesday April 30, 2024 @08:37PM (#40218205)
N=1 I'm capable of handling python on my dev/deploy box(es), but that doesn't mean it's not a pain. In my perfect world, ansible/puppet/chef/whatever would ship as a single static binary even when they mostly ran against remote SSH targets.
Re:Pyinfra: Automate Infrastructure Using Python by BirAdam on Tuesday April 30, 2024 @10:50PM (#40218996)
Right, but this makes me wonder why I can’t just do: ssh user@host “echo ‘Hello World’”

All these kinds of tools essentially just executing commands over SSH… I could just SSH.

Re:Pyinfra: Automate Infrastructure Using Python by fire_lake on Wednesday May 01, 2024 @02:49AM (#40220285)
> So only your development machine has to run Python.

If you have a team of developers and a CI process, then portability is important. There isn’t one development machine.

Re:Pyinfra: Automate Infrastructure Using Python by Fizzadar on Tuesday April 30, 2024 @01:59PM (#40214007)
Extremely aware of this (see pyinstaller attempt): https://github.com/pyinfra-dev/pyinfra/pull/768)

I chose Python because it’s what I was writing all day back in 2015. Which makes me realise pyinfra is almost 10!

Edit: I mostly write Go or YAML (k8s) these days but Python still makes an appearance from time to time (outside of pyinfra dev).

Re:Pyinfra: Automate Infrastructure Using Python by benrutter on Tuesday April 30, 2024 @02:38PM (#40214548)
I think Go would be a logically choice if you're being completely language agnostic, but most teams aren't. If teams are working exclusively in python already for web or data projects, there's a benefit to not introducing a new language just for architecture deployment if that's a small part of a teams function.
Re:Pyinfra: Automate Infrastructure Using Python by aprdm on Tuesday April 30, 2024 @01:49PM (#40213822)
Maybe because Python is already in use by pretty much every company that makes money in this (and others) domain ? Some of what you mention looks like pebkac problems as well.
Re:Pyinfra: Automate Infrastructure Using Python by mhh__ on Tuesday April 30, 2024 @01:57PM (#40213966)
Well so is pretty much any configuration language under the sun, and all the other options that aren't python.
Re:Pyinfra: Automate Infrastructure Using Python by JoBrad on Tuesday April 30, 2024 @02:30PM (#40214444)
What would you have used? All of your issues aside, Python is very approachable to people who are used to managing infrastructure but may not have a strong programming background.
Re:Pyinfra: Automate Infrastructure Using Python by mixmastamyk on Tuesday April 30, 2024 @03:09PM (#40214963)
Sounds like you'ved misjudged the use case. Tools like this do the deployment, they aren't generally deployed themselves.

So a portable binary is not a requirement. Other points like let or types are not an impediment either, there are many quality tools available if you need them (ruff, pyflakes, mypy), and python has been doing this kind of work productively for thirty years now.

Re:Pyinfra: Automate Infrastructure Using Python by posix_monad on Wednesday May 01, 2024 @05:05AM (#40221020)
> Tools like this do the deployment, they aren't generally deployed themselves.

It will have to be executed on many different developer machines (or even your own machine several years in the future) so a simple, reproducible build process, including fetching pip dependencies, is critical.

Re:Pyinfra: Automate Infrastructure Using Python by mixmastamyk on Wednesday May 01, 2024 @10:45AM (#40223994)
Presumably the tool keeps backward compatibility, as did ansible or salt, so this doesn’t seem to be a real-world concern. Very few folks are doing nix-level stuff, yet the world marches on.

There will likely be a security fix in it or a dep at a later point, so you wouldn’t want to use the exact same version anyway.

Re:Pyinfra: Automate Infrastructure Using Python by bborud on Tuesday April 30, 2024 @02:06PM (#40214102)
Python is a nightmare when used for tooling. I’ve wasted so much time wrangling Python tooling for embedded development. Go would be a much better choice.
Re:Pyinfra: Automate Infrastructure Using Python by dboreham on Tuesday April 30, 2024 @04:47PM (#40216136)
Shiv is a decent solution for making a portable package. Single file that only depends on a recent system Python being installed.
Re:Pyinfra: Automate Infrastructure Using Python by IshKebab on Tuesday April 30, 2024 @05:30PM (#40216605)
I agree, Python is a pretty bad choice for all the reasons you mentioned.

That said I think there are precious few good alternatives. I've been using Deno a fair bit for "scripting" and it works pretty well, but I wish there were more options.

Also I have to say if you are using a tool like this to manage thousands of machines you're absolutely doing it wrong. I don't even work in ops/infra but even I know that manually running commands on multiple machines via SSH is asking for trouble.

Re:Pyinfra: Automate Infrastructure Using Python by nijave on Tuesday April 30, 2024 @06:22PM (#40217148)
A decent Python development tool chain handles most of that. Docker, pylint, black, type hints integrated with IDE/editor

Admittedly some languages like Go do a better job integrating all this into the core of the language. However, Go doesn't tend to have as powerful of a stdlib so it tends to be a lot more verbose to achieve the same thing.

Re:Pyinfra: Automate Infrastructure Using Python by fire_lake on Wednesday May 01, 2024 @02:50AM (#40220295)
Docker should not be required for day-to-day development.
Re:Pyinfra: Automate Infrastructure Using Python by nijave on Wednesday May 01, 2024 @08:15AM (#40222158)
Why?
Re:Pyinfra: Automate Infrastructure Using Python by fire_lake on Friday May 03, 2024 @08:02AM (#40246708)
It’s hard to install on some setups. It’s much slower than native. It’s admitting defeat.
Re:Pyinfra: Automate Infrastructure Using Python by mountainriver on Tuesday April 30, 2024 @07:01PM (#40217529)
It’s better than Yaml or HCL though
Re:Pyinfra: Automate Infrastructure Using Python by goodpoint on Wednesday May 01, 2024 @04:13AM (#40220764)
Python is an excellent choice.

> - Python is not easy to build into portable binaries > - The package ecosystem is very hard to use in a reproducible way

People use OS packages since 4 decades.

> - The language is not truly typed

The language IS strictly typed.

> - types add massive value for infrastructure and scripts because they are less likely to be unit-tested

99% of errors in deployment are not solved by typing.

> - The lack of a "let" or "var" keyword makes simple programming errors more likely (again, this code is less likely to be unit-tested)

If your logic is so complex that let/var makes a difference you should be not touching infra.

Re:Pyinfra: Automate Infrastructure Using Python by kortex on Wednesday May 01, 2024 @03:56PM (#40228546)
This might have been true a few years ago, but these are all solved problems in 2024.

> Python is not easy to build into portable binaries

https://pex.readthedocs.io/en/v2.1.40/buildingpex.html

- The package ecosystem is very hard to use in a reproducible way

pip, virtualenv, and requirements.in/txt is extremely reproducible. I will offer that it's not exactly idiot-proof yet and there are tons of stale tutorials out there

> The language is not truly typed - types add massive value for infrastructure and scripts because they are less likely to be unit-tested

Yes it is, if you want it to be. There's nothing stopping someone from using mypy, pyright, or other type tool on the strictest setting, and not passing builds unless you have 100% type coverage.

> The lack of a "let" or "var" keyword makes simple programming errors more likely (again, this code is less likely to be unit-tested)

No, but you get ~95% of the safety guarantees by using immutable-esque objects like @dataclass(frozen=True), pydantic models with the same, or attrs/cattrs with similar setting.