News for hackers. Stuff that matters.
Re:Learning Go as a Python Developer: The Good and the Bad by jeremydw on Monday July 18, 2022 @04:09PM (#32143014)
This comment section itself clearly shows how crazy dependency and environment management is in Python. In this thread alone, we've received instructions to...

- poetry

- "Just pin the dependencies and use Docker"

- pip freeze

- Vendoring in dependency code

- pipreqs

- virtualenv

This is simply a mess and it's handled much better in other languages. I manage a small agency team and there are some weeks where I feel like we need a full-time devops person to just help resolve environment issues with Python projects around the team.

Re:Learning Go as a Python Developer: The Good and the Bad by takeda on Monday July 18, 2022 @05:43PM (#32144295)
Keep in mind that Python is 31 year old (it's even older than Java) it was created around the same time as world wide web. So it started when no one even knew they would need dependency management and evolved over time from people posting packages on their home pages, to a central website to what we now call PyPI. Similarly the tooling and way of packaging the code evolved.

What you described are multiple tools that also target different areas:

> - poetry

from what you listed this seems like the only tool that actually takes care of dependency management

> - "Just pin the dependencies and use Docker"

this is standard fallback for all languages when people are lazy and don't want to figure out how to handle the dependencies

> - pip freeze

all this does it just lists currently installed packages in a form that can be automatically read by pip

> - Vendoring in dependency code

this again is just a way that applies to all languages, and it is still necessary even if there's a robust dependency management as there are some cases where bundling everything together is preferred

> - pipreqs

this is just a tool that scans your code and tells you what dependencies you are using. You are really lost if you need a tool to tell you what packages is your application is using, but I suppose it can be useful for one offs if you inherit some python code that wasn't using any dependence management.

> - virtualenv

this is just a method to have dependencies installed locally in project directory instead per system. This was created especially for development (although it can be used for deployment as well) as people started working on multiple services with different dependencies. It's now included in python so it's more like a feature of the language.

Re:Learning Go as a Python Developer: The Good and the Bad by sph on Monday July 18, 2022 @07:15PM (#32145360)
Being 31 years old doesn't preclude having a decent, official and reproducible way of installing packages in 2022. That's just a bad excuse to justify subpar package managers and terrible governance around this problem.

Package management is pretty much a solved problem, no matter how old is your language. It smells to an outsider like me like a lot of bike-shedding and not enough pragmatism is going on in Python land over this issue. Has a new BDFL stepped up after Guido left?

Re:Learning Go as a Python Developer: The Good and the Bad by takeda on Monday July 18, 2022 @07:47PM (#32145629)
But Python has a decent and a reproducible way of installing packages. The problem python has is that things evolved over time, so you can find on the net plenty of outdated information. There is also a lot of blogs and articles with bad practices, most written by people that got something working.

I think also a lot of issues with packaging is ironically because of PyPA that supposed to work on a standard, but in reality instead of embracing and promoting something that works they just pushes half-assed solutions because author is one of the members. Kind of like they were pushing failed Pipenv "for humans". Seems like Poetry is generally popular and devs are happy with it, so of course PyPA started pushing their own Hatch project, because python packaging was finally getting too straight forward.

I think Python would benefit as a whole if PyPA was just dissolved.

Re:Learning Go as a Python Developer: The Good and the Bad by oarabbus_ on Monday July 18, 2022 @09:10PM (#32146232)
>But Python has a decent and a reproducible way of installing packages.

Reading the comment thread it's not immediately clear what the answer is - it seems implied that the proper way is using Poetry, is that the case?

Re:Learning Go as a Python Developer: The Good and the Bad by stavros on Tuesday July 19, 2022 @04:43AM (#32149016)
To me, yes. Poetry for me has been fairly straightforward, in anything I needed to do (dependency management, package publishing, etc).
Re:Learning Go as a Python Developer: The Good and the Bad by blooalien on Wednesday July 27, 2022 @09:16AM (#32250028)
Poetry has been the one I've settled on as well. It "Just Works™" for everything I've used it for thus far, and it's even been easy to convert older methods I've tried to the Poetry way of doing things.
Re:Learning Go as a Python Developer: The Good and the Bad by Ygg2 on Tuesday July 19, 2022 @12:40AM (#32147612)
It's not.

See https://imgs.xkcd.com/comics/python_environment_2x.png

Or sometime the computer is haunted and my colleague had problems installing tensorflow. To this day he has no working tensorflow.

Re:Learning Go as a Python Developer: The Good and the Bad by takeda on Wednesday July 20, 2022 @01:35AM (#32161827)
I suppose we can follow memes or facts.

Please note I never used tensorflow on that computer in fact I never used tensorflow, but this is my experience: https://gist.github.com/takeda/89ec29501b6e8641415668f22f3e9...

It succeeded after first try.

I do see that in their repo[1] they use a non standard way to build the package. They use Bazel, but that's Google for you. They never do things everyone else is doing. I'm not sure why this is Python problem rather than package problem.

They have tons of open issues around building: [2]

[1] https://github.com/tensorflow/tensorflow

[2] https://github.com/tensorflow/tensorflow/labels/type%3Abuild...

Re:Learning Go as a Python Developer: The Good and the Bad by Ygg2 on Wednesday July 20, 2022 @03:27AM (#32162611)
That's the thing. It's not a meme. It's a joke, that rings true.

It works fine for me.

Not for him. Same package, nearly identical laptops, different outcomes.

And I as newcomer to Python, there are like four (pip, venv, brew, conda) cli APIs that I had to learn just to get working on some Python file.

Re:Learning Go as a Python Developer: The Good and the Bad by leetrout on Monday July 18, 2022 @08:57PM (#32146122)
> I think Python would benefit as a whole if PyPA was just dissolved

STRONG agree. And I hate the cult of personality that was previously (still?) strong.

Re:Learning Go as a Python Developer: The Good and the Bad by akagusu on Monday July 18, 2022 @10:37PM (#32146898)
I think we can get a good package management system for Python if we get rid of PyPA.
Re:Learning Go as a Python Developer: The Good and the Bad by hota_mazi on Monday July 18, 2022 @10:34PM (#32146879)
Yup. Java is 27 years and has a splendid package management system (Maven Central) which is so well designed that you can use it from two different tools that are extremely different (Maven and Gradle).
Re:Learning Go as a Python Developer: The Good and the Bad by FridgeSeal on Monday July 18, 2022 @07:21PM (#32145419)
No but there’s a walrus operator!

Not sure anyone was asking for that, unlike a fix for packaging issues.

Re:Learning Go as a Python Developer: The Good and the Bad by tmh88j on Monday July 18, 2022 @07:52PM (#32145683)
I really like the walrus operator. It's helped me to create more straightforward if/elif blocks and loops. Effective python has some pretty good examples. https://effectivepython.com/2020/02/02/prevent-repetition-wi...
Re:Learning Go as a Python Developer: The Good and the Bad by XorNot on Monday July 18, 2022 @08:38PM (#32145998)
This really isn't a good argument though: it's an extra, extremely specific use case for assignment that looks visually very similar.

And worse, effects code maintainability - if you need that assignment higher up, you're now editing the if statement, adding an assignment, plus whatever your interstitial code is.

Python doesn't have block scoping so the argument for it is weak.

Re:Learning Go as a Python Developer: The Good and the Bad by tmh88j on Monday July 18, 2022 @08:42PM (#32146021)
It's not for extremely specific use-case, unless you consider using variables as a condition of an if statement or loop extremely specific.

> And worse, effects code maintainability - if you need that assignment higher up, you're now editing the if statement, adding an assignment, plus whatever your interstitial code is.

How is that different than variables declared without the walrus operator? If you declare a variable with the walrus operator and decide to move its declaration you can still continue to reference that variable in the same spot, just like any other variable. Do you have an example you can share to demonstrate this? I'm not sure I understand what you mean.

> Python doesn't have block scoping so the argument for it is weak.

The walrus operator another way to define variables, not change how they behave. It's just another addition to the "pythonic" way of coding. It's helped me to write more concise and even clearer code. I suggest reading the Effective Python link I provided for some examples of how you can benefit from it.

Re:Learning Go as a Python Developer: The Good and the Bad by XorNot on Tuesday July 19, 2022 @10:06PM (#32160493)
So by way of example - suppose I have this:

  # some other code
  
  if determined_value := some_function_call():
      do_action(determined_value)
  
and then I change it to this:

  # some other code
  
  determined_value = some_function_call()
  logger.info("Determined value was %s", determined_value)
  validate(determined_value)
  
  if determined_value:
      do_action(determined_value)
  
and determined_value is a reasonably expensive operation (at the very least I would never want to redundantly do it twice) - then in this case my diff for this looks like:

  --- <unnamed>
  +++ <unnamed>
  @@ -1,5 +1,8 @@
  -
   # some other code
   
  -if determined_value := some_function_call():
  +determined_value = some_function_call()
  +logger.info("Determined value was %s", determined_value)
  +validate(determined_value)
  +
  +if determined_value:
       do_action(determined_value)
whereas if I wrote it without walrus originally:

  --- <unnamed>
  +++ <unnamed>
  @@ -1,5 +1,8 @@
   # some other code
   
   determined_value = some_function_call()
  +logger.info("Determined value was %s", determined_value)
  +validate(determined_value)
  +
   if determined_value:
       do_action(determined_value)
  
then the diff is easier to read, and the intent is clearer because diff can simply infer that what's happening is the semantic addition of two lines.

Code is read more then it's written, and changed more then originally created, and making the change case clearer makes sense.

Re:Learning Go as a Python Developer: The Good and the Bad by tmh88j on Tuesday July 19, 2022 @10:56PM (#32160822)
Your issue is with the readability of the diff? That is so trivial. You're trying to find anything to complain about at this point. How about just look at the code? You should be doing that anyway.
Re:Learning Go as a Python Developer: The Good and the Bad by XorNot on Wednesday July 20, 2022 @12:19AM (#32161368)
Diffs are the predominant way people relate to code changes via PRs. It is standard practice to restructure patch sets to produce a set of easy to read to changes which explain what is happening - what "was" and what "will be".
Re:Learning Go as a Python Developer: The Good and the Bad by tmh88j on Wednesday July 20, 2022 @01:02AM (#32161608)
It's standard practice to look at the code alongside a diff. Better yet, use your IDE or a tool to show you far more detail than a diff possibly could. https://www.jetbrains.com/help/pycharm/comparing-files-and-f...

This is such a pedantic, non-issue I don't even know why I'm bothering acknowledging it.

Re:Learning Go as a Python Developer: The Good and the Bad by thayne on Monday July 18, 2022 @10:52PM (#32146990)
An example where I have wanted this many times before it existed is in something like:

while (n := s.read(buffer)) != 0:

    #do something with first n bytes of buffer
Without the walrus operator you either have to duplicate the read operation before the loop and at the end of the loop, or use while True with a break if n is zero. Both of which are ugly IMO.
Re:Learning Go as a Python Developer: The Good and the Bad by ok_dad on Monday July 18, 2022 @08:54PM (#32146101)
It’s really handy in comprehensions to do assignments outside the normal syntax.
Re:Learning Go as a Python Developer: The Good and the Bad by rewgs on Monday July 18, 2022 @10:15PM (#32146756)
I used to think this and rage at the current state of Python dependency management. Then I just buckled down and learned the various tools. It's honestly fine.
Re:Learning Go as a Python Developer: The Good and the Bad by tmh88j on Monday July 18, 2022 @07:16PM (#32145365)
> > - pip freeze

> all this does it just lists currently installed packages in a form that can be automatically read by pip

Are you referring to version specifiers[1] being optional or is there something more to versions that I don't understand? PEP 440 is a wall of text, maybe I should get around to reading it sometime.

[1] https://packaging.python.org/en/latest/glossary/#term-Versio...

Re:Learning Go as a Python Developer: The Good and the Bad by takeda on Monday July 18, 2022 @07:24PM (#32145452)
I mean pip freeze outputs packages and their versions in machine readable form that can be read and installed by pip install -r requirements.txt
Re:Learning Go as a Python Developer: The Good and the Bad by tmh88j on Monday July 18, 2022 @07:35PM (#32145552)
This whole chain started with someone pointing out the author doesn't seem to realize you can pin versions[1]. I'm just confused how people seemed to end up questioning what pip freeze does.

[1] https://news.ycombinator.com/item?id=32141573#32142190

Re:Learning Go as a Python Developer: The Good and the Bad by takeda on Monday July 18, 2022 @07:59PM (#32145741)
Perhaps I was just proving author's point.

I tried to say that from mentioned tools only poetry can be called as a dependency management.

The other tools are used for different purposes, but perhaps could be used as a piece of package management in some way. The mentioned docker and vendoring is irrelevant to Python and it even applies to Go.

Re:Learning Go as a Python Developer: The Good and the Bad by okasaki on Monday July 18, 2022 @05:00PM (#32143627)
Sometimes I feel people are using Python very differently than me. I just use pip freeze and virtualenv (these are Python basics, not some exotic tools) and I feel it works great.

Granted, you don't get a nice executable, but it's still miles ahead of C++ (people literally put their code into header files so you don't have to link to a library), and even modern languages like rust (stuff is always broken, or I have some incompatible version, even when it builds it doesn't work)

By the way if you're a Python user, Nim is worth checking out. It's compiled, fast and very low fuss kind of language that looks a lot like Python.

Re:Learning Go as a Python Developer: The Good and the Bad by pdimitar on Monday July 18, 2022 @05:08PM (#32143748)
> and even modern languages like rust (stuff is always broken, or I have some incompatible version, even when it builds it doesn't work)

Been working on and off with Rust for the last 3 years, never happened to me once -- with the exception of the Tokio async runtime that has breaking changes between versions. Everything else always worked on the first try (checked with tests, too).

Comparing Python with C++ doesn't make do argument any favours, and neither does stretching a single Rust accident to mean the ecosystem is bad.

Re:Learning Go as a Python Developer: The Good and the Bad by mcronce on Monday July 18, 2022 @06:04PM (#32144584)
This is consistent with my experience. Semantic versioning is very very widely used in the Rust ecosystem, so you're not looking at breaking changes unless you select a different major version (or different minor version, for 0.x crates) - which you have to do manually, cargo will only automatically update dependencies to versions which semver specifies should be compatible.

For crates that don't follow semver (which I'm fairly certain I've encountered zero times) you can pin a specific exact version.

Re:Learning Go as a Python Developer: The Good and the Bad by fiddlerwoaroof on Monday July 18, 2022 @06:18PM (#32144760)
When I was a Python dev, I never saw that happen in ten years or so of work. Pip freeze and virtualenv just worked for me.

I will say, though, that this only accounts for times where you’re not upgrading dependencies. Where I’ve always run into issues in Python was when I decided to upgrade a dependency and eventually trigger some impossible mess.

Re:Learning Go as a Python Developer: The Good and the Bad by sinsterizme on Tuesday July 19, 2022 @07:04AM (#32149900)
Piptools [1] resolves this by having a requirements.in file where you specify your top level dependencies, doing the lookup and merging of dependency versions and then generating a requirements.txt lock file. Honestly it’s the easiest and least complex of Python’s dependency tools that just gets out of your way instead of mandating a totally separate workflow a la Poetry.

[1] https://github.com/jazzband/pip-tools

Re:Learning Go as a Python Developer: The Good and the Bad by mypalmike on Monday July 18, 2022 @07:07PM (#32145277)
Upgrading dependencies is fraught in all languages.
Re:Learning Go as a Python Developer: The Good and the Bad by ramchip on Monday July 18, 2022 @08:11PM (#32145821)
It's really not. I've had a much, much worse experience with Python than Elixir / Go / Node for various reasons: lots of different tools rather than one blessed solution, people pinning specific versions in requirements (2.1.2) rather than ranges (~> 2.1), dependency resolution never finishing, pip-tools being broken 4 times by new pip releases throughout a project (I kept track)...

In Elixir I can do `mix hex.outdated` in any project, no matter who wrote it, and it'll tell me very quickly what's safe to update and link to a diff of all the code changes. It's night and day.

Thankfully, it's getting gradually better with poetry, but it's still quite clunky compared to what you get elsewhere. I noticed lately for instance that the silent flag is broken, and there's apparently no way to prevent it from spamming 10k lines of progress bars in the CI logs. There's an issue on Github, lost in a sea of 916 other open issues...

Re:Learning Go as a Python Developer: The Good and the Bad by oars on Monday July 18, 2022 @07:27PM (#32145481)
Haven't had any issues in Go so far.
Re:Learning Go as a Python Developer: The Good and the Bad by mypalmike on Monday July 18, 2022 @09:42PM (#32146500)
As soon as you take 2 dependencies in any language, there's a chance you will not be able to upgrade both of them to latest versions because somewhere in the two dependency subgraphs there is a conflict. There's no magic to avoid this, though tooling can help find potentially working versions (at least by declaration). It's often the case that you don't encounter conflicts in Python or other languages, but I don't imagine that Go is immune.
Re:Learning Go as a Python Developer: The Good and the Bad by jhardy54 on Monday July 18, 2022 @10:01PM (#32146660)
Are you familiar with npm? I haven’t used it deeply in years, but the last time I checked it still supported conflicting dependencies.

If A and B both depend on different versions of C, then both versions of C are installed and A/B see the version they want.

Re:Learning Go as a Python Developer: The Good and the Bad by mypalmike on Monday July 18, 2022 @10:20PM (#32146783)
I've used npm but an not familiar with these kinds of details of it. There would seem to be some potential putfalls, such as two libraries accessing a single system resource (a config file, a system socket, etc.). I will take a look into this though. Thanks.
Re:Learning Go as a Python Developer: The Good and the Bad by fiddlerwoaroof on Wednesday July 20, 2022 @07:38PM (#32173875)
npm works around some problems like this with a concept of "peer dependencies" which are dependencies that can only be depended on once. The typical dependency, though, is scoped to the package that requires it.
Re:Learning Go as a Python Developer: The Good and the Bad by pkolaczk on Monday July 18, 2022 @11:11PM (#32147106)
Rust can include different versions of the same library (crate) in a single project just fine. As long as those are private dependencies, no conflicts would happen. A conflict would happen only if two libraries pass around values of a shared type, but each wanted a different version of the crate that defines the type.

I can't see a reason why Go would be different.

Re:Learning Go as a Python Developer: The Good and the Bad by tempest_ on Monday July 18, 2022 @05:09PM (#32143764)
Comparing anything to C++ is a very low bar.

I have rarely encountered issues in rust. Most rust crates stick to semver so you know when there will be a breaking change. My rust experience with Cargo can only be described as problem free(though I only develop for x86 linux).

As for pip freeze and virtualenv things start to fall apart especially quickly when you require various C/C++ dependencies (which in various parts of the ecosystem is a lot) as well as different python versions (if you are supporting any kind of legacy software). This is also assuming other people working on the same project have the same python yadda yadda the list goes on, its not great.

Re:Learning Go as a Python Developer: The Good and the Bad by geysersam on Monday July 18, 2022 @07:41PM (#32145593)
> pip freeze and virtualenv things start to fall apart especially quickly when you require various C/C++ dependencies

Yes 100 times. That can be incredibly frustrating. During the last year I've used a large (and rapidly evolving) C++ project (with many foss C/C++ dependencies) with Python bindings. We've collectively wasted sooo many hours on dependency issues in the team.

Long compilation times contribute considerably to slow down the feedback loop when debugging the issues.

Re:Learning Go as a Python Developer: The Good and the Bad by fiddlerwoaroof on Monday July 18, 2022 @06:14PM (#32144716)
The issue I’ve encountered in rust is looking at a library and seeing that it requires a nightly build of the compiler.
Re:Learning Go as a Python Developer: The Good and the Bad by smeagull on Monday July 18, 2022 @07:13PM (#32145335)
If I did that I'd have no harddrive space left due to how large python dependencies are.
Re:Learning Go as a Python Developer: The Good and the Bad by Spivak on Monday July 18, 2022 @08:21PM (#32145883)
Wat? What dependencies? I have right now 100 separate Python projects which each have their own venv, their own isolated dependencies, and their own isolated copy of Python itself and my code dir doesn’t even crack top 10 hard drive space.
Re:Learning Go as a Python Developer: The Good and the Bad by Game_Ender on Monday July 18, 2022 @10:29PM (#32146846)
Machine learning packages like Tensorflow can result in GB size package directories.
Re:Learning Go as a Python Developer: The Good and the Bad by stavros on Tuesday July 19, 2022 @08:00AM (#32150314)
Yes, the biggest available library is big, that's hardly an argument though.
Re:Learning Go as a Python Developer: The Good and the Bad by smeagull on Thursday July 21, 2022 @08:46AM (#32178654)
A really common library that gets used in every project for common use-cases is big.

It's a very good argument.

Re:Learning Go as a Python Developer: The Good and the Bad by stavros on Thursday July 21, 2022 @08:48AM (#32178667)
No, saying "one common library is big" isn't a good way to show that "Python dependencies are big", which is what the initial claim was. Most libraries are tiny, and there's a very big one because it does many things. If Tensorflow were in JS it wouldn't be any smaller.
Re:Learning Go as a Python Developer: The Good and the Bad by rewgs on Monday July 18, 2022 @10:27PM (#32146830)
Yup. PyEnv for installing various versions. /path/to/pyenv/version/bin/python -m venv venv. Activate the venv (I made "av" and "dv" aliases in my .zshrc). Done, proceed as usual.
Re:Learning Go as a Python Developer: The Good and the Bad by chpmrc on Monday July 18, 2022 @05:36PM (#32144186)
- Poetry is a 3rd party package manager, I'm sure it's great but it's not widely used (yet)

- Pip freeze just pins all dependencies at once to requirements.txt

- I don't know what "vendoring in dependency code" means

- I've never used pipreqs in my life (and 80% of my work has been in Python)

- Virtualenvs are just a convenient way to keep project runtimes separated

And for 90% of Python projects in existence the following is sufficient (assuming Python3 is installed):

- python -m venv .venv

- source .venv/bin/activate

- pip install -r requirements.txt

That's it. And all of that requires a single dependency: Python. Could it be better? Sure. But to call that a "mess" is an exaggeration.

Re:Learning Go as a Python Developer: The Good and the Bad by linsomniac on Monday July 18, 2022 @05:53PM (#32144423)
"Vendoring" means including the library in your package. So instead of listing it in requirements.txt, you copy the code of the library.
Re:Learning Go as a Python Developer: The Good and the Bad by giantrobot on Monday July 18, 2022 @07:32PM (#32145521)
You can get that by bundling up your venv. When you install a package is a venv it installs it into that venv rather than the system. As far as the venv is concerned it is the system Python. Unfortunately passing around a venv can be problematic, say between Mac and Linux or between different architectures when binaries are involved.
Re:Learning Go as a Python Developer: The Good and the Bad by KptMarchewa on Monday July 18, 2022 @08:48PM (#32146062)
>- python -m venv .venv

>- source .venv/bin/activate

If those two steps were automatic like in node.js we'd literally have 99% less problems.

Re:Learning Go as a Python Developer: The Good and the Bad by jeremydw on Monday July 18, 2022 @07:43PM (#32145604)
Sounds like someone has never been asked to clone and run software targeting Python 3.x when their system-installed Python is 3.y and the two are incompatible.
Re:Learning Go as a Python Developer: The Good and the Bad by Spivak on Monday July 18, 2022 @08:27PM (#32145912)
Then you use pyenv and pyenv virtualenv.

    pyenv install 3.9.47
    pyenv virtualenv 3.9.47 my app
    git clone …/myapp
    cd …/myapp
    pyenv local myapp
    pip install -r requirements.txt
Is it annoying, maybe, but I normally don’t trust system deps for anything.
Re:Learning Go as a Python Developer: The Good and the Bad by chpmrc on Tuesday July 19, 2022 @01:04PM (#32154730)
Sounds like someone is making assumptions on the way I work :) As a matter of fact I have and the solution is pyenv. Node has a similar utility called n.
Re:Learning Go as a Python Developer: The Good and the Bad by throwawaymaths on Monday July 18, 2022 @04:16PM (#32143081)
Now try installing tensorflow. Treat yourself to ice cream if you get it to install without having to reinstall Linux and without borking the active project you're on.
Re:Learning Go as a Python Developer: The Good and the Bad by raffraffraff on Monday July 18, 2022 @04:48PM (#32143459)
And unless things have gotten a lot better in the 2 years since I last did `pip install numpy` on ARM, prepare for a very long wait because you'll be building it from source.
Re:Learning Go as a Python Developer: The Good and the Bad by 0x008 on Tuesday July 19, 2022 @05:03AM (#32149159)
Sometimes if you don’t install wheel, it will also automatically fall back to source build. Very sweet.
Re:Learning Go as a Python Developer: The Good and the Bad by Gilboboy on Tuesday July 19, 2022 @02:57AM (#32148346)
There's a precompiled package for ARM these days.
Re:Learning Go as a Python Developer: The Good and the Bad by codethief on Tuesday July 19, 2022 @08:49AM (#32150716)
Does it come with GPU support?
Re:Learning Go as a Python Developer: The Good and the Bad by derac on Monday July 18, 2022 @05:04PM (#32143679)
The major issues you'll see involve library version mismatches. It's a very good idea to use venv with these tools since there are often mismatches between projects.
Re:Learning Go as a Python Developer: The Good and the Bad by throwawaymaths on Tuesday July 19, 2022 @12:31AM (#32147562)
Tensorflow sometimes is pinned to Nvidia drivers, and protobuf. And I think it has to be system level unless you reaaaaally want to fiddle with the internals.
Re:Learning Go as a Python Developer: The Good and the Bad by smeagull on Monday July 18, 2022 @07:14PM (#32145345)
I'd be too fat from all that icecream.
Re:Learning Go as a Python Developer: The Good and the Bad by throwawaymaths on Tuesday July 19, 2022 @12:33AM (#32147576)
Ok. I know two companies that exist because of this problem.
Re:Learning Go as a Python Developer: The Good and the Bad by earthboundkid on Monday July 18, 2022 @11:53PM (#32147350)
The core problem with Python dependencies is denial. There are tons of people who make excuses for a system that is so bad Randall Munroe declared his dependencies a “superfund site” years ago. In a good ecosystem, that comic would have prompted change. Instead, it just prompted a lot of “works for me comments”. Monads are just monoids in the category of endofunctors, and C just requires you to free memory at the right time. Python dependencies just work as long as you do a thing I’ve never seen my colleagues do at any job I’ve worked at.
Re:Learning Go as a Python Developer: The Good and the Bad by MonkeyMalarky on Monday July 18, 2022 @04:34PM (#32143294)
Don't forget Anaconda because you're on windows and have no idea how to compile random packages that are really C++ code with Python bindings!
Re:Learning Go as a Python Developer: The Good and the Bad by jjoonathan on Monday July 18, 2022 @04:40PM (#32143357)

    Solving environment |
    Solving environment /
    Solving environment -
    Solving environment \
    Solving environment |
    ...
One day it takes 10 seconds, next month it takes 10 minutes, and the month after that it takes 30 minutes and then fails entirely.
Re:Learning Go as a Python Developer: The Good and the Bad by MonkeyMalarky on Monday July 18, 2022 @04:48PM (#32143471)
That's half of why from 2017 to 2021 I had a yearly "uninstall Anaconda and start fresh" routine. The other half is because I'd eventually corrupt my environments and have no choice but to start over.
Re:Learning Go as a Python Developer: The Good and the Bad by jjoonathan on Monday July 18, 2022 @05:05PM (#32143709)
Me too. In 2021 it got so egregious that I finally jumped ship. Pip or bust.

Since you specifically mentioned 2021 instead of 2022, I half suspect you had the same experience.

Re:Learning Go as a Python Developer: The Good and the Bad by darkarmani on Monday July 18, 2022 @06:09PM (#32144651)
Are you using conda-forge? Solving from over 6TBs of packages can take quite a while. Conda-forge builds everything. This isn't a criticism, but because of that the number of packages is massive.
Re:Learning Go as a Python Developer: The Good and the Bad by jjoonathan on Monday July 18, 2022 @07:44PM (#32145611)
"You can't use the big channel with all the packages because it has all the packages" isn't an exoneration, it's an indictment.

To answer your question: yes, we were using conda-forge, and then when it stopped building we moved to a mix of conda and a critical subchannel, and then a major GIS library broke on conda and stayed that way for months so we threw in the towel and just used pip + a few touch-up scripts. Now that everyone else has followed suit, pip is the place where things "just work" so now we just use pip, no touchups required.

Re:Learning Go as a Python Developer: The Good and the Bad by korijn on Monday July 18, 2022 @04:45PM (#32143428)
Is it a mess? Yes. But, is the problem to be solved perhaps much simpler "in other languages"? Do you interface with C++ libraries, system-managed dependencies, and perhaps your GPU in these other languages? Or are all your dependencies purely coded in these other languages, making everything simpler?

Of course the answer to these questions could be anything but to me it feels like attacks on Python's package management are usually cheap shots on a much much more complicated problem domain than the "complainers" are typically aware of.

Re:Learning Go as a Python Developer: The Good and the Bad by pdimitar on Monday July 18, 2022 @05:07PM (#32143721)
Or the "complainers" work with Rust and Elixir and giggle at Python's last-century dependency-management woes, while they run a command or two and can upgrade and/or pin their dependencies and put that in version control and have builds identical [to those on their dev machines] in their CI/CD environment.

¯\_(ツ)_/¯

Your comment hints that you are feeling personally attacked when Python is criticized. Friendly unsolicited advice: don't do that, it's not healthy for you.

Python is a relic. Its popularity and integration with super-strong C/C++ libraries has been carrying it for at least the last 5 years, if not 10. There's no mystery: it's a network effect. Quality is irrelevant when something is popular.

And yes I used Python. Hated it every time. I guess I have to thank Python for learning bash scripting well. I still ended up wasting less time.

Re:Learning Go as a Python Developer: The Good and the Bad by Zopieux on Monday July 18, 2022 @05:59PM (#32144523)
It's a bit ironic to pick someone's up on "taking things personally upon criticism", then proceeding to display a deep, manichean, unfounded hatered for a language that, despite its numerous flaws, remains a popular and useful tool.
Re:Learning Go as a Python Developer: The Good and the Bad by pdimitar on Monday July 18, 2022 @06:08PM (#32144634)
Hanging people upon dawn was popular as well; people even got their kids for the event and it was happening regularly. Popularity says nothing about quality or even viability.

Use Python if it's useful for you, obviously. To me though the writing is on the wall -- it's on its loooong and painful (due to people being in denial) way out.

EDIT: I don't "hate"; it was a figure of speech. Our work has no place for such emotions. I simply get "sick of" (read: become weary of) something being preached as good when it clearly is not, at least in purely technical terms. And the "hate" is not at all unfounded. Choosing to ignore what doesn't conform to your view is not an argument.

Re:Learning Go as a Python Developer: The Good and the Bad by korijn on Tuesday July 19, 2022 @01:43AM (#32147948)
> Your comment hints that you are feeling personally attacked when Python is criticized.

I am not feeling personally attacked (I am not married to Python), I am mostly just tired of reading the same unproductive type of complaints over and over again. This attitude is not unique to Python's situation, but actually is typical to our industry. It makes me want to find a different job, on some days.

The community is trying to improve the situation but there is no way to erase Python's history. So it's always going to continue to look messy if you keep looking back. The complaint is unproductive, or in other words, not constructive.

Re:Learning Go as a Python Developer: The Good and the Bad by pdimitar on Tuesday July 19, 2022 @07:36AM (#32150128)
I can agree with your comment. What's missing is the possibility to, you know, just jump ship.

You might not be married to Python but it sure looks that way for many others. I switched main languages no less than 4 times in my career and each time it was an objective improvement.

The thing that kind of makes me look down on other programmers is them refusing to move on.

Re:Learning Go as a Python Developer: The Good and the Bad by korijn on Tuesday July 19, 2022 @11:29AM (#32152944)
For nuance and to address your point, I have worked with PHP for about six years, .NET for five, C++ for two, and Python for seven.

I still dabble in all of them. Who knows when I will move on to the next. Rust looks nice. I tried go.

But they do not yet provide any of the tools/libraries I need for my work. That's how I've always selected my programming language.

So I would first need to invent the universe before I can create valuable things. Instead I will just wait until their ecosystems mature a little more.

I will end the discussion here though. Thanks for the response!

Re:Learning Go as a Python Developer: The Good and the Bad by pdimitar on Tuesday July 19, 2022 @11:34AM (#32153037)
That's a valid reason to hold on. Thanks for indulging the discussion!
Re:Learning Go as a Python Developer: The Good and the Bad by throwawaymaths on Tuesday July 19, 2022 @12:37AM (#32147595)
Yes. In elixir, you can install GPU stuff (Nx) with very few problems. Some people have built some really cool tools like burrito, which cross-compile and bundles up the VMs to other architectures. Even before that it's been pretty common to cross-compile from x86 to an arm raspberry pi image in the form of Nerves.

As a rule elixir devs don't do system level dependencies, probably because of lessons learned from the hell scape that is python (and Ruby)

Yesterday I onboarded a coworker onto the elixir project, he instinctively put it into a docker container. I laughed and just told him to run it bare (he runs macos, I run Linux). There were 0 problems out of the box except I forgot the npm incantations to load up the frontend libraries.

Re:Learning Go as a Python Developer: The Good and the Bad by korijn on Tuesday July 19, 2022 @01:52AM (#32147991)
Ok, now I am curious :) let's try some typical problems from the Python package ecosystem:

- Can you resolve precompiled GPU dependencies with system managed CUDA driver versions?

- Are there packages that can convert PDF to PNG without system dependencies?

- Can you run a Qt GUI app on CI without needing to do any additional system setup?

With regards to cross compilation tools like burrito, that's neat! But Python is not a compiled language.

Re:Learning Go as a Python Developer: The Good and the Bad by throwawaymaths on Tuesday July 19, 2022 @05:27PM (#32158209)
1) not sure how Nx does it.

2) not that I can find for that specific task but the typical strategy is to download (or compile) a binary, drop it into a {project-dependency}[0]-local "private assets" directory, and call out the binary. This is for example how I embed zig pl into elixir (see "zigler") without system-level dependencies. Setting this up is about 40 lines of code.

3) wx is preferred in the ecosystem over qt, but this (and openssl) are the two biggies in terms of "needs system deps", though it's possible to run without wx.

For native graphics, elixir is treading towards glfw, which doesn't have widgets, but from what I hear there are very few if any gotchas in terms of using it.

I bring up cross-compilation, because burrito allows you to cross-compile natively implemented code, e.g. bcrypt that's in a library. So libraries that need c-FFI typically don't ship binaries, they compile at build time. Burrito binds "the correct" architecture into the c-flags and enables you to cross compile c-FFI stuff, so you don't have a system level dependency.

[0] not that this has happened, but two dependencies in the same project could download different versions of the same binary and not collide with each other.

Re:Learning Go as a Python Developer: The Good and the Bad by korijn on Wednesday July 20, 2022 @07:03AM (#32163990)
So it seems python and elixir are not that different at all. You are handling these problems the same way as the Python community is.
Re:Learning Go as a Python Developer: The Good and the Bad by ActorNightly on Monday July 18, 2022 @06:18PM (#32144767)
Its not a mess, people just make it a mess because of the lack of understanding around it, and getting lazy with using a combination of pip install, apt install, and whatever else. Also, the problem is compounded by people using Mac to develop, which have a different way of handling system wide python installs from brew, and then trying to port that to Linux.
Re:Learning Go as a Python Developer: The Good and the Bad by 0x008 on Tuesday July 19, 2022 @05:04AM (#32149167)
One could argue that if you need a lot of understanding it is a mess.
Re:Learning Go as a Python Developer: The Good and the Bad by KptMarchewa on Monday July 18, 2022 @08:43PM (#32146033)
No one sane actually advocates using system python to develop anything, whether on mac or linux
Re:Learning Go as a Python Developer: The Good and the Bad by feet on Monday July 18, 2022 @04:32PM (#32143272)
Even using conda to manage reqs is an absolute nightmare. Did a subreq get updated? Did the author of the library pin that subreq? No? Have fun hunting down which library needs to be downgraded manually
Re:Learning Go as a Python Developer: The Good and the Bad by MonkeyMalarky on Monday July 18, 2022 @06:40PM (#32145005)
I used a couple of tricks to solve this. First, make cond env export a build step and environment.yml an artifact so you've got a nice summary of what got installed. Second, nightly builds so you aren't surprised by random package upgrade errors the next time you commit code to your project.
Re:Learning Go as a Python Developer: The Good and the Bad by feet on Monday July 18, 2022 @07:47PM (#32145627)
My bad experiences are usually around setting up projects created by others, sadly
Re:Learning Go as a Python Developer: The Good and the Bad by SloopJon on Monday July 18, 2022 @06:35PM (#32144945)
This has indeed been eye opening. We got bit by a dependency problem in which TensorFlow started pulling in an incompatible version of protobuf. After reading these comments, I don't think that pip freeze is quite what we want, but poetry sounds promising. We have a relatively small set of core dependencies, and a bunch of transitive dependencies that just need to work, and which we sometimes need to update for security fixes.
Re:Learning Go as a Python Developer: The Good and the Bad by codethief on Tuesday July 19, 2022 @08:53AM (#32150756)
Why do you think that `pip freeze` wouldn't be what you want? (I once had the exact same issue with TF and protobuf and specifying the exact protobuf version I wanted solved it.)
Re:Learning Go as a Python Developer: The Good and the Bad by brightball on Monday July 18, 2022 @10:35PM (#32146886)
When I tried learning Python, this mess is what turned me off so badly. Python is the first language I ever came across where I felt like Docker was necessary just to keep the mess in a sandbox.

Coming to that from hearing stories that there was supposed to be one way to do everything disenchanted me quickly.

Re:Learning Go as a Python Developer: The Good and the Bad by shaklee3 on Tuesday July 19, 2022 @01:10AM (#32147763)
python has had virtual environments forever now. what was it missing?
Re:Learning Go as a Python Developer: The Good and the Bad by brightball on Tuesday July 19, 2022 @09:12AM (#32150954)
Every article I found suggested different versions of Python, like Anaconda. They all suggested different virtual environments too. Rarely was an explanation given. At the time, the mix of Python 2 vs Python 3 was a mess.

The code itself was okay, but everything around it was a train wreck compared to every other language I’d been using (Go, Java, Ruby, Elixir, even Perl).

I attempted to get into it based on good things I’d heard online, but in the end it just wasn’t my cup of tea.

Re:Learning Go as a Python Developer: The Good and the Bad by Bullfight2Cond on Monday July 18, 2022 @04:42PM (#32143387)
one more that supports the latest standards: https://pdm.fming.dev/
Re:Learning Go as a Python Developer: The Good and the Bad by bsder on Monday July 18, 2022 @07:24PM (#32145449)
> This is simply a mess and it's handled much better in other languages.

I don't agree.

The problem is simply that Python encompasses a MUCH larger space with "package management" than most languages. It also has been around long enough to generate fairly deep dependency chains.

As a counterexample, try using rust-analyzer or rust-skia on a Beaglebone Black. Good luck. Whereas, my Python stuff runs flawlessly.

What many newer languages do is precompile the happy paths(x86, x86-64, and arm64) and then hang you out to dry if that's not what you are on.

Re:Learning Go as a Python Developer: The Good and the Bad by ksdnjweusdnkl21 on Monday July 18, 2022 @04:42PM (#32143378)
I agree that it is handled better in many other languages. However, Go has some weird thing with imports going on. When I tried to learn it I just could not import a function from another file. Some env variable making the program not find the path. Many stackoverflow/reddit threads condecendenly pointed to some setup guide in official docs which did not fix or explain the situation.

After an few hours or so of not making much progress in AOC day 1 I just gave up and never continued learning Go.

Re:Learning Go as a Python Developer: The Good and the Bad by Gordonjcp on Monday July 18, 2022 @06:54PM (#32145145)
It's not crazy at all. You use requirements.txt to keep a track of the dependencies needed for development, and you put the dependencies needed to build and install a package into setup.py where the packaging script can get it.

These are two different things, because they do two different jobs.

It's really very simple.

Re:Learning Go as a Python Developer: The Good and the Bad by punnerud on Monday July 18, 2022 @05:30PM (#32144091)
Not a mess, but options. That’s what’s encourage innovation and open up for new ideas.

This Fortran vs Python example is worth reading: https://cerfacs.fr/coop/fortran-vs-python

Re:Learning Go as a Python Developer: The Good and the Bad by smeagull on Monday July 18, 2022 @07:12PM (#32145331)
The real issue is that you can't have multiple versions installed at the same time.

if you could import numpy==3.2 it'd solve many problems.

Re:Learning Go as a Python Developer: The Good and the Bad by thinker5555 on Monday July 18, 2022 @07:34PM (#32145548)
Granted, I'm more of a hobbiest than a dev, but I think this is part of the problem that virtualenvs are supposed to help solve. One project (virtualenv) can have numpy==3.2, and another can have numpy==3.1. Maybe I'm naive, but it seems like having a one project with multiple versions of numpy being imported/called at various times would be asking for trouble.
Re:Learning Go as a Python Developer: The Good and the Bad by Spivak on Monday July 18, 2022 @08:36PM (#32145977)
The thing you can’t do is solve for this situation.

    A depends on B, C
    B depends on D==1.24
    C depends on D==2.02
There should in an super ideal world be no problem with this. You would just have a “linker” that inserts itself in the module loader that presents different module objects when deps ask for “D” but it hasn’t happened yet.
Re:Learning Go as a Python Developer: The Good and the Bad by goodpoint on Tuesday July 19, 2022 @07:36AM (#32150131)
That's a feature, not a bug.
Re:Learning Go as a Python Developer: The Good and the Bad by smeagull on Thursday July 21, 2022 @08:47AM (#32178662)
It's really not. I can't install tensorflow more than 10 times before I'm done. Virtualenvs are not fit for purpose.