News for hackers. Stuff that matters.
Re:An analysis of module names inside top PyPI packages by dheera on Wednesday July 10, 2024 @08:30PM (#40932789)
I hate this shit.

    yaml -> pip install pyyaml
    cv2 -> pip install opencv-contrib-python
    PIL -> pip install pillow (wtf, this should be a misdemeanor punishable by being forced to used windows for a year)
And can we please ban "py" and "python" from appearing inside the name of python packages?

Or else I'm going to start writing some python packages with ".js" in their name.

Re:An analysis of module names inside top PyPI packages by ziml77 on Wednesday July 10, 2024 @10:13PM (#40933194)
I have to look up PIL every time I use it to remember if I install PIL and import pillow or install pillow and import PIL.

Imports can be aliased, so why allow this mismatch at all? PyPI should have enforced that each package contains one top-level module whose name is identical to the name used to install it.

Re:An analysis of module names inside top PyPI packages by woodruffw on Wednesday July 10, 2024 @11:21PM (#40933392)
Imports can be aliased as bindings; they can't be aliased at the import machinery layer, which makes the PIL/pillow distinction necessary. The adjacent subthread has an example of this.
Re:An analysis of module names inside top PyPI packages by cqqxo4zV46cp on Thursday July 11, 2024 @05:07AM (#40934828)
Starting any sentence in 2024 with “PyPI should have…” is a pretty ridiculous premise. We learn things over time, and PyPI itself wasn’t exactly operating on a green field.
Re:An analysis of module names inside top PyPI packages by remram on Thursday July 11, 2024 @11:40AM (#40937866)
There used to be a PIL, someone made a new compatible distribution. They had to use the same import name to be compatible with existing code, they had to pick another name on PyPI that wasn't taken. It's kind of an extreme case.
Re:An analysis of module names inside top PyPI packages by woodruffw on Wednesday July 10, 2024 @09:24PM (#40933006)
Banning "py" would catch "mypy" and "pydantic", both of which you probably don't intend to catch.

pillow is imported as `PIL` because it's a fork of the original PIL[1]. There's a very strong argument that Python's ability to retain the same import name across package name changes like that is a valuable source of flexibility that has benefited the ecosystem as a whole.

[1]: https://pypi.org/project/PIL/

Re:An analysis of module names inside top PyPI packages by throw-the-towel on Wednesday July 10, 2024 @09:32PM (#40933038)
> Python's ability to retain the same import name across package name changes...

As in, `import pillow as PIL`?

Re:An analysis of module names inside top PyPI packages by woodruffw on Wednesday July 10, 2024 @09:39PM (#40933075)
> As in, `import pillow as PIL`?

As in, not changing your imports at all, and just changing your dependency from PIL to pillow. This has two substantial advantages:

1. You only have to change one line (the dependency), not an indefinite number of source files. This is less of an issue now that the Python community has high-quality refactoring tools, but it's still the past of least resistance.

2. More importantly: `import pillow as PIL` is not referentially transparent: the `PIL` binding that it introduces is a `module` object, but that object can't be used in subsequent imports. In other words, blindly performing an `import X as Y` refactor would break code like this:

    import PIL
    from PIL import whatever
You can observe this for yourself locally:

    >>> import ssl as lol
    >>> from lol import CERT_NONE
    ModuleNotFoundError: No module named 'lol'
    >>> from ssl import CERT_NONE
This is arguably a defect in Python's import and module machinery, but that's how it currently is. Renaming the dependency and keeping the module name is far less fraught.
Re:An analysis of module names inside top PyPI packages by dheera on Thursday July 11, 2024 @01:37AM (#40933950)
The related thing that bothers me deeply is that

    import PIL
does not make PIL.Image available. What the hell else do you expect me to do with PIL? Why isn't PIL.Image included in importing PIL? You have to explicitly do either of these

    import PIL.Image
    from PIL import Image
Re:An analysis of module names inside top PyPI packages by woodruffw on Thursday July 11, 2024 @01:56AM (#40934011)
That’s because it’s a module within the PIL module, not an attribute of PIL. But that doesn’t really have anything to do with the original comment; that’s a different quirk of Python’s import machinery.

(Understanding the difference between packages, module hierarchies, and module attributes is table stakes for architecting a large Python package correctly. PIL almost certainly does this to prevent hard-to-debug circular imports elsewhere in their codebase.)

Re:An analysis of module names inside top PyPI packages by Jasper_ on Thursday July 11, 2024 @02:08PM (#40939119)
It's a strange distinction, because the standard library sometimes eschews this. `os.path` is accessible through just `import os`, because they made os.py import it into the local namespace.

I wish it was clearer sometimes what was a module, and what was an attribute in the core import syntax. `import foo; foo.bar` only breaks if it's a module, and `import foo.bar` only breaks if it's an attribute. If you do `from foo import bar`, the syntax works with both.

Re:An analysis of module names inside top PyPI packages by ciupicri on Thursday July 11, 2024 @03:18PM (#40939683)
Just because `os.path` is accessible through just `import os`, doesn't mean that you shouldn't import it explicitly. As the Zen of Python says, explicit is better than implicit. After all it's documented separately at https://docs.python.org/3/library/os.path.html

If you see `os.path.basename` what could `os.path` be? It would be a module most of the time because it's written with lowercase. `itertools.chain.from_iterable` [1] would be a notable exception.

[1]: https://docs.python.org/3/library/itertools.html#itertools.c...

Re:An analysis of module names inside top PyPI packages by cozzyd on Wednesday July 10, 2024 @08:51PM (#40932872)
Unless something is a binding, baking a package after the programming language is super weird. Like what if you change the implementation language later?
Re:An analysis of module names inside top PyPI packages by rty32 on Wednesday July 10, 2024 @09:31PM (#40933034)
> what if you change the implementation language later?

I don't think that is a thing that happens in real life.

* Practically, one package is associated with exactly one github repository, sometimes a few. You would see implementation switching from JavaScript to TypeScript, but almost never from python to Go. Normally people start a brand new project for that kind of thing. * The reality is that each language has its own library ecosystem, and people reinvent the wheel at least once for each language. I wish we live in a world where you could save the effort, instead implement everything only once and it runs efficiently and has idiomatic APIs everywhere. But that's not how it works. If you create a package for a language, that's it. You could reimplement the same thing like by line in another language, but that would be a different package for that language.

Re:An analysis of module names inside top PyPI packages by cozzyd on Wednesday July 10, 2024 @09:49PM (#40933116)
It's pretty common for e.g. old scientific software to get rewritten from Fortran to C++ with a version bump.
Re:An analysis of module names inside top PyPI packages by dheera on Thursday July 11, 2024 @01:15AM (#40933869)
Yeah but what is common in real life is writing multiple parallel libraries for {Python, NodeJS, ...} with a nearly identical API. In this case I would think that if the Python command is `pip install foo`, the NodeJS command should be `npm install foo`. It's redundant to do `pip install foo-python` when pip is only for Python, and opens the door for stealthy attacks where someone else creates `pip install foo` on PyPI that is forked from your repo and mirrors your API exactly but steals data and credentials and sends it to malicious servers.
Re:An analysis of module names inside top PyPI packages by kortex on Thursday July 11, 2024 @02:16AM (#40934072)
> when pip is only for Python

That's the neat part, it's not! You can distribute basically any kind of data with pip, within reason. Iirc Cmake can be pip-installed.

Re:An analysis of module names inside top PyPI packages by rented_mule on Thursday July 11, 2024 @03:10AM (#40934335)
`pip install nodejs-bin` gets you node, including npm, in your venv along with bindings for calling it all from Python.
Re:An analysis of module names inside top PyPI packages by nilamo on Wednesday July 10, 2024 @09:00PM (#40932913)
Pillow is a special case, in that it was always meant as a drop in replacement for the PIL, and you only changed the requirements.txt
Re:An analysis of module names inside top PyPI packages by ziml77 on Wednesday July 10, 2024 @10:27PM (#40933233)
Feels to me like that was a deficiency in the package management tools. Like if your requirements file could define a global alias, it would allow people who want that easy one-line change to install pillow as PIL. But everyone else who was starting fresh or who was okay with doing a few edits to their Python files could install pillow and use it as pillow.

I guess though that there could be an issue with some dependencies being written against PIL and others being written against pillow?

Re:An analysis of module names inside top PyPI packages by RockRobotRock on Thursday July 11, 2024 @04:55AM (#40934782)
It's funny and sad how you remember the stupid aliases after a while.