News for hackers. Stuff that matters.
Re:Why do we need modules at all? (2011) by nickjj on Tuesday August 27, 2019 @08:06AM (#20808672)
I really like the idea of a no module system.

If you look at the /usr/bin directory of a Linux system there aren't any namespaces. There's 1,000+ binaries thrown into 1 folder, each with a distinct name and things work nicely.

Imagine if you tried to group those binaries into 1 or more folder driven namespaces. Where would things go? The amount of bike shedding would be endless and I don't think it would be a better solution in the end even if you managed to pull it off.

All of Joe's points in that post are spot on.

Re:Why do we need modules at all? (2011) by coldtea on Tuesday August 27, 2019 @09:11AM (#20809160)
>If you look at the /usr/bin directory of a Linux system there aren't any namespaces. There's 1,000+ binaries thrown into 1 folder, each with a distinct name and things work nicely.

That's the last example of a nice design I would bring, and the worst possibly argument in favor of no modules/namespacing.

A flat /usr/bin/... with 1000+ binaries in UNIX is a bad solution we ended up with because back in the day where UNIX was created there were just a few dozens of binaries, so it was "good enough".

Re:Why do we need modules at all? (2011) by onion2k on Tuesday August 27, 2019 @10:19AM (#20809819)
...back in the day where UNIX was created there were just a few dozens of binaries, so it was "good enough".

What's actually wrong with it though? Why isn't it still "good enough"? It's hard to read the list, but Linux has plenty of tools to fix that particular problem. I don't really see the issue.

Re:Why do we need modules at all? (2011) by coldtea on Tuesday August 27, 2019 @10:47AM (#20810107)
It's all a big dump, discoverability is bad.

Name clauses.

Version clauses. "All binaries inside a folder" needs renames to allow for a different version (even if it's a different minor version).

It's just the binaries, which was OK back in the "program = single binary", before "man" even, days, but now it splits the program from different assets it uses (man pages, default configuration, image assets, etc). This in turn makes deleting/moving more difficult, and adds all kinds of baggage to package managers.

>It's hard to read the list, but Linux has plenty of tools to fix that particular problem.

That's a description of the problem to me. Needing "plenty of tools" to fix an initial bad commitment.

Re:Why do we need modules at all? (2011) by nickjj on Tuesday August 27, 2019 @11:31AM (#20810659)
> It's all a big dump, discoverability is bad.

I see it as easier to discover.

Imagine if you had to goto /usr/bin/output/user to find the whoami binary.

If you did, who is responsible for making those directories and what happens when you have a binary that could really belong in 5 different places?

Re:Why do we need modules at all? (2011) by raehik on Tuesday August 27, 2019 @10:57AM (#20810225)
This is one of the many problems Guix solves on a slightly lower level than most package managers, because binaries live inside Guix packages (/usr/bin is simply populated by absolute symlinks). Of course package managers also do this, but Guix kind of lets you symlink to a (unique) package, rather than `install`.
Re:Why do we need modules at all? (2011) by johnisgood on Tuesday August 27, 2019 @11:51AM (#20810960)
For stuff I compile from scratch I just use a symlink-based package manager I found online:

https://zolk3ri.name/cgit/zpkg/

It seems very minimal but supports my requirements. It makes sense to me, because everything gets installed (usually via make install) to ~/.local/pkg/foo-1.0/ and the like, and then symlinks are created to, say, ~/.local/{bin,etc,include,lib,man,share,var}. According to the source code, it is configurable via environment variables. It seems to share similarities with GNU's stow.

Re:Why do we need modules at all? (2011) by grumbel on Tuesday August 27, 2019 @01:11PM (#20812078)
For one I think the example doesn't really fit, as Linux binaries are much closer to modules than they are to plain functions, as each of those binaries has numerous command line flags to change it's behavior quite radically. Functions as you encounter in a programming language are much simpler and more focused.

The other issue is that the approach is extremely inflexible, if you want to replace /usr/bin/program with a new version, everybody on the system is forced to use the new version, even if the new version might break stuff. Traditional `/usr/bin` doesn't provide a solution for this. You can of course work around that by using `/opt` or installing things in your home directory, but at that point you are just using the file system as namespace.

Re:Why do we need modules at all? (2011) by ggg3 on Tuesday August 27, 2019 @10:26PM (#20816339)
very good point.

i think the solution to this problem (module or not) will be the same solution to the shared lib problem.

npm is a dumpster fire, but the larger projects are usually fine (the /bin equivalent). the worst problem is that every large project pull in different versions of whatever libs they wanted to use (which the singleton util lib proposed in the article would solve)

Re:Why do we need modules at all? (2011) by abernard1 on Tuesday August 27, 2019 @01:37PM (#20812381)
> What's actually wrong with it though? Why isn't it still "good enough"?

Anyone who has had to try to manage security permissions on a big flat list of files knows it's like holding jello with rubber bands.

As an another example people might be familiar with: having to manage a big PHP or Ruby codebase where everything is separated by the models, views, and controllers a hierarchical level _above_ the business function. It's a nightmare. Trying to pull out the useful business functionality that's interleaved everywhere makes it very hard to pull codebases apart to where different teams can work on them.

With respect to Joe's comment here:

> Bad: It's very difficult to decide which module to put an individual function in. Break encapsulation (see later)

As much as I like Joe's thought process, that "difficult" part is otherwise known as "the real work". All systems which scale in complexity--as opposed to the simple problem scaled in performance--need strong boundaries. They also all have a hierarchical component to them.

Re:Why do we need modules at all? (2011) by coding123 on Tuesday August 27, 2019 @09:51AM (#20809543)
This was the hardest thing to get used to in linux coming from Windows. In Windows an installed program lives in Program Files and has a shortcut that starts it from it's home folder. That's very different from a namespace collision possible global install that can be run from any folder.

I still don't fully agree with it but it makes some things easier. The Windows way also made multiple version installs possible.

Re:Why do we need modules at all? (2011) by dragonwriter on Tuesday August 27, 2019 @10:00AM (#20809629)
> In Windows an installed program lives in Program Files and has a shortcut that starts it from it's home folder.

And command line tools have that folder added to the PATH, and Windows has a real and ongoing problem with PATH getting too long for the OS to handle when you have enough of them.

Windows is GUI optimized with shell as an afterthought, Linux is vice versa.

Re:Why do we need modules at all? (2011) by Crinus on Tuesday August 27, 2019 @10:44AM (#20810070)
FWIW you get the same on Linux by installing stuff in /usr/whatever/wherever, /usr/local/whatever/wherever or /opt (which is the closest to Program Files-like place) and making symlinks to the "global" binaries (those meant to be accessible via the shell) while keeping everything else in the application-specific folder.

It isn't very uncommon to see binaries inside /usr/bin being symlinks to binaries in other directories, even if you stick with distribution-only packages.

Re:Why do we need modules at all? (2011) by cmcaine on Tuesday August 27, 2019 @11:13AM (#20810428)
You can have multiple versions of a program installed in linux. Most package managers won't help you very much with that, but it's about as easy as windows if the programs are interpreted or statically linked or compile easily with your system libraries.

NixOS and distri are notable package managers that do make installing different versions of packages pretty easy.

Re:Why do we need modules at all? (2011) by mrighele on Tuesday August 27, 2019 @10:42AM (#20810051)
But a unix system has a module system: the filesystem and its folders. The fact that there are a number of "functions" that are in the same "default namespace" is just a convenience: if needed two binaries with the same name can coexist.

Also i will add that in fact the systen binaries are already split: at minimum /usr/bin and /usr/sbin (but also historically, /bin and /sbin although i think in recent linux system the last two are just symlinks to the formers, or the other way around)

Re:Why do we need modules at all? (2011) by NikkiA on Tuesday August 27, 2019 @08:31AM (#20808845)
quick, I have a linux box here that I use for some scheme and some CLR development, what does `csi` run ? chicken-scheme's interpreter or microsoft's C# repl ?
Re:Why do we need modules at all? (2011) by tremon on Tuesday August 27, 2019 @01:40PM (#20812428)
Run `csi --help` and find out?
Re:Why do we need modules at all? (2011) by TeMPOraL on Tuesday August 27, 2019 @05:22PM (#20814503)
And hope it won't delete your working directory.

Seriously though, considering the CLI software I use occasionally on my Linux machine, for any given program one - and only one - of the following will tell you its version: -v, -version, --version, version. I never can guess in advance which one is the right to use.

Re:Why do we need modules at all? (2011) by dmitriid on Tuesday August 27, 2019 @08:19AM (#20808764)
There are namespaces, you just don't see them. That's why there's python and python3, cmake and cmake2 (IIRC) etc.

With functions it's worse. In a flat namespace you will inevitably run into a problem of two people coming up with the same name.

Re:Why do we need modules at all? (2011) by nickjj on Tuesday August 27, 2019 @08:31AM (#20808846)
> There are namespaces, you just don't see them. That's why there's python and python3, cmake and cmake2 (IIRC) etc.

IMO python and python3 is a very welcome feature not a negative side effect of no namespaces. At call time it's super explicit so you know exactly what version you're using.

But in most module driven programming languages you might import the foo function at the top of your file and then use foo in 5 different places within that file. However, you spend 99% of your time working with the code in the file not glancing at imports, so now you're left wondering not only where foo is coming from, but who provided foo. Is it from the standard library, your own code base or a third party author? Suddenly you need to keep all of this in your head and it sucks.

Phoenix (a web framework for Elixir) has been taking steps to remove a lot of loose function imports so you know what module they are coming from at call time, but that's because Elixir as a language has modules. But even still, explicitness where you're using it is so much better in the long run for maintenance, even if it involves typing a few more characters.

Re:Why do we need modules at all? (2011) by Anonymous4C54D6 on Tuesday August 27, 2019 @10:59AM (#20810259)
As way-too-often, I think the solution should be a good IDE. There is nothing stopping your IDE from completing "foo" as "module1.foo" and offering "module2.foo" as well if it exists. Alternatively the IDE could just display "module1.foo" instead of foo because that is what the import dictates.

As always with dynamic things and editing code for them, this will be more difficult to get right when imports are dynamic.

Re:Why do we need modules at all? (2011) by dkersten on Tuesday August 27, 2019 @08:32AM (#20808864)
> There are namespaces, you just don't see them. That's why there's python and python3, cmake and cmake2 (IIRC) etc.

That's a versioning issue, that would happen with namespaces too. You'll have to disambiguate somehow and if a version number is the natural way, then that's the natural way regardless of if a single name or namespaced.

Keeping the next incompatible version as a separate thing (eg python vs python3, instead of continuing python but with semvers saying that 3.0 breaks 2.x compatibility) is actually a good idea. It leads to fragmentation, sure, but it makes dealing with the different versions easier IMHO and incompatible versions may as well be totally different things anyway.

> With functions it's worse. In a flat namespace you will inevitably run into a problem of two people coming up with the same name.

Yes, completely agree!

Re:Why do we need modules at all? (2011) by lootsauce on Tuesday August 27, 2019 @02:09PM (#20812713)
Just spit-balling here... I think a proposal like this can benefit from making some assumptions about tooling, the database of functions and their metadata would logically be tightly integrated into your editor. The official name of the function could simply be a hash of its contents, the serialized ast then hashed could be another option for matching exactly-syntactically equivalent code. Then human friendly names could be automatically searchable and tacked on in the code via tooling which presents the metadata in-line but does not become part of the source.
Re:Why do we need modules at all? (2011) by tosh on Tuesday August 27, 2019 @08:23AM (#20808790)
You could use something like uuids for naming functions and attach meta data (including e.g. human readable names and/or namespaces).
Re:Why do we need modules at all? (2011) by pjc50 on Tuesday August 27, 2019 @08:32AM (#20808856)
> uuids for naming functions

Microsoft COM would like a word. {20D04FE0-3AEA-1069-A2D8-08002B30309D}

Re:Why do we need modules at all? (2011) by ChrisSD on Tuesday August 27, 2019 @08:35AM (#20808883)
People on HN keep reinventing COM. I swear every other week we have discussions like the above in at least one thread.
Re:Why do we need modules at all? (2011) by pjc50 on Tuesday August 27, 2019 @08:47AM (#20808959)
Yup. There's a lot of factors at play. We're a very ahistorical field; lots of folks are self-taught and therefore reinventing everything is normal to them; the dev universe is heavily siloed into "Web+UNIX" versus "Microsoft" versus some now moribund siloes (Websphere! Solaris! Alpha!); Microsoft have almost no community compared to the FLOSS world (although .net is changing this a bit); and finally the simple fact that COM and you and I are very, very old.

I'm not even sure that reinventing COM would be a bad idea - you can definitely do some great things with it, and the specific implementation details of COM are crufty. But doing so deliberately rather than accidentally would be better, with a review of benefits and problems of COM approaches.

Re:Why do we need modules at all? (2011) by pjmlp on Tuesday August 27, 2019 @09:43AM (#20809475)
COM has been reinvented, as UWP. :)

As for the community, maybe not on SV, but there are plenty of Microsoft communities and meeting groups.

Re:Why do we need modules at all? (2011) by NikkiA on Tuesday August 27, 2019 @11:04AM (#20810327)
Tell them it's called 'Core Foundation Plugin' and point at the apple docs, they'll start loving it.

https://developer.apple.com/documentation/corefoundation/cfp...

Re:Why do we need modules at all? (2011) by abstract7 on Tuesday August 27, 2019 @01:50PM (#20812532)
It's "Component Object Model" for anyone out there that almost gave up after googling "COM".
Re:Why do we need modules at all? (2011) by jcelerier on Tuesday August 27, 2019 @08:27AM (#20808827)
> and things work nicely.

no they don't

Re:Why do we need modules at all? (2011) by brightball on Tuesday August 27, 2019 @08:38AM (#20808904)
I’m a big fan of Joe...but I also remember PHP before it had namespaces...and it was a train wreck. IMO all of the improvements in the PHP world have happened because they added them.

I can see it for core libraries, but outside of that I think namespaces are necessary for the greater language ecosystem.

Re:Why do we need modules at all? (2011) by nickjj on Tuesday August 27, 2019 @08:43AM (#20808933)
I was there in the PHP 4.x days too during the early 2000s, and you know, I really liked it. It just made things so simple to understand and work with. Look up a function, see a bunch of examples in the docs and use it right away without having to worry about imports and lugging around a ton of baggage.

I think the main reason PHP got such a bad rap for flat functions is because they implemented it poorly. So many things were extremely inconsistent when it came to naming conventions and argument orders that it became near impossible to remember them in a systematic way. But couldn't that problem be addressed today since we're aware that naming consistency is an important programming language feature?

> I can see it for core libraries, but outside of that I think namespaces are necessary for the greater language ecosystem.

Yeah I could get on board with that. Docker sort of does this now with Docker images. At the UI level (the tooling we use in our day to day like the CLI) we don't need to access "elixir" or "python" images with a namespace (Docker adds it for us automatically), but we are free to create our own brightball/elixir or nickjj/elixir images to avoid conflicts.

But in a programming world, I would like to type brightball_elixir at call time in my code instead of importing elixir from brightball, because with the latter I have no idea if elixir is coming from you, me, or the standard library unless I move away from the code and look at the import.

Re:Why do we need modules at all? (2011) by brightball on Tuesday August 27, 2019 @09:01AM (#20809066)
I'm mainly talking about the framework world at that time. They all reinvented EVERYTHING because they couldn't just easily include a shared database library, templating library, etc. They all reproduced their own with their own names to get around it.

That type of stuff. The core language was about the same regardless, but the ecosystem improved dramatically.

Re:Why do we need modules at all? (2011) by nickjj on Tuesday August 27, 2019 @09:18AM (#20809225)
Ah ok. When it came to using PHP back then I didn't even use third party frameworks. I just cherry picked things like database libraries and pulled them into my code base.
Re:Why do we need modules at all? (2011) by ggg3 on Tuesday August 27, 2019 @10:31PM (#20816370)
you should re-read the article. the main point is that it will solve frameworks (not exactly what the author aimed at, this I am extrapolating) reinventing everything. all utils methods will live in the proposed repository in the sky with all methods. frameworks will just contain their higher level logic, until that becomes common enough to use standard types in the util repository in the sky.
Re:Why do we need modules at all? (2011) by adev_ on Tuesday August 27, 2019 @09:06AM (#20809113)
> I’m a big fan of Joe...but I also remember PHP before it had namespaces.

To be picky: module and namespace are different concepts.

C++ has namespace and (soon) will have namespaces AND modules.

Namespaces can be seen in C++ as a pure hardcoded prefix for your functions and are a perfectly valid concept with Joe's idea of a "giant KV store for functions".

Re:Why do we need modules at all? (2011) by cjfd on Tuesday August 27, 2019 @08:27AM (#20808831)
Well, with 1000 names you still don't get (many) name collisions so no modules/namespaces is fine. At some point you will find yourself introducing smurf namespacing and then I would consider modules/namespaces the better solution.
Re:Why do we need modules at all? (2011) by enriquto on Tuesday August 27, 2019 @08:33AM (#20808870)
If you think about it, smurf namespacing is much simpler thus more elegant than a "real" namespace system.
Re:Why do we need modules at all? (2011) by cjfd on Tuesday August 27, 2019 @08:50AM (#20808984)
In one sense, yes, but in another no. The main problem is that one can get real long names.
Re:Why do we need modules at all? (2011) by enriquto on Tuesday August 27, 2019 @09:04AM (#20809097)
Depending on your philosophy, that may be a feature to you. I tend to like this effect, because it forces the programmer to avoid building deeply nested structures, which is too easy to do when you abuse modules/namespaces.
Re:Why do we need modules at all? (2011) by cjfd on Tuesday August 27, 2019 @09:30AM (#20809348)
That is a valid point. There is a tendency for things to go the java way.
Re:Why do we need modules at all? (2011) by goatlover on Tuesday August 27, 2019 @11:10AM (#20810390)
Because long names in Java meant avoiding deep nesting?