News for hackers. Stuff that matters.
Re:The Trouble with __all__ by kstrauser on Thursday August 01, 2024 @02:44PM (#41132196)
Eh. Python does indicate public and private APIs. "Name" is public. "_Name" is internal, but you can mess with it if you need to. "__Name" is private, and if you go there anyway and the thing explodes and gives you a bad haircut, you were warned.

Python's position is that we're all adults here. Don't touch "_Name". Really don't touch "__Name". You can if you're an expert and willing to take responsibility for your actions.

Addendum: And in this ModuleWrapper thing, I can still access `core._module.PrivateApi` so we're back to square one.

Re:The Trouble with __all__ by 0x63_Problems on Thursday August 01, 2024 @03:13PM (#41132521)
In my experience, Python devs rarely use the underscore.

Package authors are pretty good about it. But when the only people using your code are your coworkers, people don't seem to put as much thought into having a clean interface.

Re:The Trouble with __all__ by polotics on Thursday August 01, 2024 @03:30PM (#41132682)
The notion that developers are not to be treated as adults and must have access forbidden to them, or else tight coupling will develop and rot the code, is super strange to me.

If you cannot trust your coworkers to respect the _meaning of __names, then how can you trust them with much harder concerns like algorithms and data-structures?

The article contains this amazing quote: """I’ve seen a unicorn startup dump their existing codebase and start over because the modules within it became so tightly coupled together, it was impossible to effectively develop within it or break them apart."""

This quote definitely needs flesh on its bones. I don't think this was Instagram, and I am not sure who else? Could the author chime in maybe?

Re:The Trouble with __all__ by MrJohz on Thursday August 01, 2024 @04:06PM (#41133021)
For me it's less about trusting coworkers, and more about trusting myself. When I see these sorts of discussions through those eyes, I can understand much more why people want guard rails when they're programming. I fancy myself a fairly competent dev, but I've seen code that I've written on a bad day where I can barely even understand what I was trying to achieve, let alone what I actually did achieve. Ideally, yes, there's code review and a second pair of eyes to make sure everything does make sense, and that my publics are public and privates are private. But sometimes a reviewer misses things as well, or the reviewer isn't as familiar with that bit of the codebase, or whatever other problem - and mistakes happen.

Obviously mistakes can also happen with guard rails enabled. In a language with explicit exports or private-by-default attributes, you can still make mistakes and expose more implementation details than you wanted. But in my experience, that happens a lot less often than it does in Python, where privacy is just a matter of convention that's easy to forget.

So in that regard, yes I trust myself and my coworkers, even with the difficult parts, but I also assume that we'll all occasionally make mistakes, and if I can find ways to avoid those mistakes, I'm all for it.

Re:The Trouble with __all__ by eddd-ddde on Thursday August 01, 2024 @04:00PM (#41132964)
Well, I don't trust my coworkers, neither do I trust my self. That's why code review is a thing.

It's easy to miss an internal api access, so why not make it just impossible?

Re:The Trouble with __all__ by the1024 on Thursday August 01, 2024 @04:29PM (#41133271)
My experience primarily comes from scale-ups, where the companies grew fast and hired fast.

Without putting specific companies on blast, I hear about this problem consistently from growth stage startups that scaled on Python. It's a common reason people reach for microservices - trying to use network boundaries to cover for the lack of module boundaries.

Ideally you can trust coworkers; unfortunately that's not always the case. Here's a quote from a developer in response to the article in another forum: """ It can even be very useful sometimes to be able to import private stuff. In a large ecosystem where you use many modules from other teams, you might need to use some private functionality. """ If something is available for an individual developer that solves their problem, often they're not thinking about the fact that it's private or creates a brittle dependency with no contract.

Re:The Trouble with __all__ by kstrauser on Thursday August 01, 2024 @03:16PM (#41132547)
That’s a reasonable decision if no one else will be using the code.
Re:The Trouble with __all__ by VeejayRampay on Thursday August 01, 2024 @05:01PM (#41133580)
"we're all adults here" is self-justification for the fact that the language is unable to enforce anything
Re:The Trouble with __all__ by kstrauser on Thursday August 01, 2024 @05:14PM (#41133699)
The false assumption that a language can enforce something is the root of 10,000 CVEs.

For instance, in C, you have read/write access to the whole program's memory space. Want to work around some security mechanism? A well-aimed pointer will do the right trick.

Python doesn't prevent you from ever touching _internal or __private names. If you're reviewing a Python PR that does that, you should be as suspicious as if you were reviewing C code that stomps all over memory. In either case you almost certainly should not be doing that.

But sometimes, only sometimes, you might be the right person in the right use case to do it. Then you can ignore the linters screaming at you and concentrate on convincing your coworker this is a good idea.