News for hackers. Stuff that matters.
Re:Flat Tree Dependency Resolution in Npm v3 by kylecordes on Thursday December 31, 2015 @11:52AM (#10818381)
I understand that this is a hairy, messy problem they are solving, and that they traded off one aspect of the problem for another (Install order dependency! as a new "feature" in 2015!).

But I wish they had aimed higher. The better goal would be that the entire state of the node modules directory is a pure function of the contents of the package.json file plus the platform details (compiler used for native modules).

While they're at it, the ecosystem could be improved considerably if there was some sort of obvious "penalty" applied to any package that compiles native code, because such things cause considerable extra trouble for Windows users. A visible penalty, transitively carried up the dependency tree, would discourage use of such modules with native code; projects would use them (depend on them) only if absolutely necessary instead of accidentally, all over, all the time.

Re:Flat Tree Dependency Resolution in Npm v3 by davexunit on Thursday December 31, 2015 @12:04PM (#10818446)
>The better goal would be that the entire state of the node modules directory is a pure function of the contents of the package.json file plus the platform details

Yes, this would have been much better than this new, also-broken approach. Functional package managers like the one I work on, GNU Guix, have this problem solved and solved well. Language-specific package manager developers would do well to either implement similar systems in their projects, or realize the inherent problems with limiting the scope of dependency management to a single language rather than the full dependency graph and switch to using functional package managers.

Re:Flat Tree Dependency Resolution in Npm v3 by munificent on Thursday December 31, 2015 @12:22PM (#10818530)
> Language-specific package manager developers would do well to either implement similar systems in their projects

This is what we did with Dart's package manager[1]. It works like Bundler where it finds a single set of package versions that satisfy all of the constraints and there are no duplicate packages. It's a challenge because constraint solving is NP-complete, but it works very well in practice.

[1]: https://pub.dartlang.org/

> realize the inherent problems with limiting the scope of dependency management to a single language rather than the full dependency graph and switch to using functional package managers.

The problem then is now your language has a dependency on an outside package manager, one which is often OS specific. Most modern languages need to support a variety of OSes.

Re:Flat Tree Dependency Resolution in Npm v3 by davexunit on Thursday December 31, 2015 @12:49PM (#10818684)
>This is what we did with Dart's package manager[1]. It works like Bundler where it finds a single set of package versions that satisfy all of the constraints and there are no duplicate packages.

Constraint solving is the status quo, and I am strongly opposed to it. Rather than saying "I want foo >= 1.0" in a package manager, you should have the build system test the environment for the features you need. This is what the Autotools and other build systems that have been around for decades do. This way, the project isn't tied to a single package manager and, in the case of free software, greatly eases the burden on package maintainers that want to add your software to their distribution.

Functional package management doesn't need constraint solving because every package in the system precisely describes itself: exactly which dependencies are needed for build-time and runtime (which precisely describe themselves, recursively, all the way down to libc), precisely which source code (tarball/directory/whatever + checksum), and the exact build script that turns that source code into a (hopefully bit-reproducible) binary. There's no constraint solving to determine what the dependency tree is, it's already been encoded in the package objects themselves. This is what any robust package manager should enable, but the only two that I know of that do this are Nix and GNU Guix.

>The problem then is now your language has a dependency on an outside package manager, one which is often OS specific. Most modern languages need to support a variety of OSes.

A programming language should not depend on any specific package manager. This is what build systems are for! More and more language communities conflate the two, but this is a huge mistake that we are paying for by making our software nearly impossible to reproduce from source code. [0] Everyone just uses pre-built binaries full of circular dependencies and reproducibility issues because things are so tangled that no one actually knows how to build anything from source anymore.

I think the value of language-specific package managers lies in providing an easy way to fetch pure (no native code) modules to faciliate simple code sharing and helping newcomers get bootstrapped quickly. However, for serious software development and deployment they are terrible and we desperately need better tools. I think functional package managers are the tools we need, as they have greatly simplified software building and deployment for me whilst also greatly increasing the reliability of the systems that use them thanks to transactional upgrades and rollbacks.

[0] http://www.vitavonni.de/blog/201503/2015031201-the-sad-state...

Re:Flat Tree Dependency Resolution in Npm v3 by nawitus on Thursday December 31, 2015 @03:03PM (#10819369)
Language specific package managers seem to actually work in real life and are cross platform, e.g. npm. But I'm not aware of anything for e.g. C++ which would offer the features of npm (mainly installing dependencies with a single command and supporting GNU/Linux, OS X and Windows).

Is there a single non-language specific package manager which is cross platform, and supports a large percentage of packages for a given programming language?

Note that I'm not saying npm is excellent, but I'm not aware of a single successful language-independent package manager. If there are none, why?

Re:Flat Tree Dependency Resolution in Npm v3 by Arcsech on Thursday December 31, 2015 @04:08PM (#10819687)
The aforementioned GNU Guix and the similar Nix project are exactly that: Cross-platform (mostly, at least Nix requires a unixy layer on Windows) language-independent package managers.

One could argue that they aren't "successful" (in that they don't have all the packages you might want), but they are still quite new and are pretty different from traditional package managers. I believe that Nix/Guix are the future, but a future that will take some time to become widespread.

Re:Flat Tree Dependency Resolution in Npm v3 by Touche on Thursday December 31, 2015 @04:32PM (#10819805)
> Cross-platform (mostly, at least Nix requires a unixy layer on Windows)

Not good enough, language package managers must work on Windows, with a regular command prompt.

Re:Flat Tree Dependency Resolution in Npm v3 by davexunit on Thursday December 31, 2015 @06:53PM (#10820408)
Windows is hopeless. If Windows is a must, you really cannot do much better than a package manager per language that you use. No binary reproducibility, no transactional management, no provenance, no thanks.
Re:Flat Tree Dependency Resolution in Npm v3 by gcr on Thursday December 31, 2015 @03:36PM (#10819550)
Conda aspires to be exactly that. It attempts to package the entire Python scientific stack, but its general purpose nature lets you package other kinds of software as well (for instance, conda can replace compiling arbitrary C programs from source and installing them into your home folder)
Re:Flat Tree Dependency Resolution in Npm v3 by tgamblin on Thursday December 31, 2015 @11:31PM (#10821106)
Spack [1,2] is our attempt to do this for HPC, where we have two things going on: 1) Our applications aren't single-language. They include Fortran, Python, C, C++, Yorick (!), Lua, etc. etc. 2) We have a major combinatorial versioning problem, because HPC clusters (and scientific software in general) rely on ABI-incompatible libraries like MPI, and MANY different compilers.

Spack is build-from-source and takes a lot of inspiration from Nix and its descendants, but it adds support for things like swapping out compilers/MPI versions/etc., as well as a syntax for composing new builds of things on the fly That is, you can build something with a slightly different version of a dependency without ever editing a package file. Our users to be able to build a package with 6 different compilers and 3 different MPI library versions, and have their 18 versions coexist on the same system. This has been super helpful for testing production code at LLNL, because we have to run our stuff on other peoples' often very different clusters. For instance, at LANL, we prefer PGI compilers. At LLNL we tend to like Intel or Clang.

One reason languages tend to grow their own package managers is that there are all kinds of peculiarities about how different languages manage modules/extensions. Have you seen how many different ways there are to deploy Python modules? It's very hard to have multiple versions of things coexisting in the same Python env, and things like pkg_resources require editing client code [3]. Ew. Spack tries to address this by allowing different language packages to implement their own module activation/deactivation logic, so you can kind of have a language-specfic package manager for each install of the language. I'd rather have something more like Nix's profiles, though.

I wouldn't say that the project is huge yet, but we've had some interest from mostly other HPC sites. It would be cool if the project caught on in the "real" world, too.

[1] https://github.com/llnl/spack

[2] http://www.computer.org/csdl/proceedings/sc/2015/3723/00/280...

[3] http://stackoverflow.com/questions/5265731

Re:Flat Tree Dependency Resolution in Npm v3 by civodul on Thursday January 07, 2016 @09:20AM (#10858021)
I find that Spack in fact takes very little inspiration from functional package managers, Nix and GNU Guix. Dependencies are specified in a loose fashion which, on one hand, gives a lot of flexibility, but on the other hand is detrimental to reproducibility.

More importantly, part of the dependency graph, including compilers, is treated specially and essentialy considered outside of the scope of Spack. This seriously hinders reproducibility, as we tried to explain in https://hal.inria.fr/hal-01161771/en .

Re:Flat Tree Dependency Resolution in Npm v3 by davexunit on Thursday December 31, 2015 @06:55PM (#10820417)
>'m not aware of a single successful language-independent package manager

Ummmm... dpkg and yum? You know, the package managers that manage the software on everyone's production servers. Or do these not count because they don't work on Windows or something?

Re:Flat Tree Dependency Resolution in Npm v3 by spinningarrow on Friday January 01, 2016 @04:51AM (#10821515)
Yes, his previous sentence specifically mentioned 'cross-platform'.
Re:Flat Tree Dependency Resolution in Npm v3 by alextgordon on Thursday December 31, 2015 @01:09PM (#10818784)
When we ship node modules to be embedded into Chocolat.app, we have to strip out all the tests and documentation because otherwise it bloats the download size. And the computer it eventually runs on may not be connected to the internet.

The nice thing about node is that it's quite small (3.4MB zipped). We don't have to ship npm with the app, and certainly an OS level package manager would be way too large!

Re:Flat Tree Dependency Resolution in Npm v3 by nawitus on Thursday December 31, 2015 @03:03PM (#10819373)
Each package should strip tests and documentation from the package. Of course, many don't..
Re:Flat Tree Dependency Resolution in Npm v3 by tracker1 on Thursday December 31, 2015 @06:30PM (#10820329)
True enough... it's not always easy to consider as creating an npm module and building/pushing to NPM is really easy to do.
Re:Flat Tree Dependency Resolution in Npm v3 by paulyg on Thursday December 31, 2015 @04:55PM (#10819913)
This is also what Composer (package manager for PHP) did. They ported openSUSE's libzypp to PHP to solve the dependency management problem. After resolving versions the dependencies are installed in a flat list.
Re:Flat Tree Dependency Resolution in Npm v3 by ploxiln on Thursday December 31, 2015 @12:36PM (#10818613)
This "penalty" you propose would be a cosmetic thing, it wouldn't really make a difference. Many devs would choose to use the module as a dependency of their modules.

The real solution to the sub-dependency-insanity of npm is ... use fewer dependencies. Don't use dependencies that have sub-dependencies. Write more code to directly solve your problem.

This is a viewpoint you won't often hear, because it comes from someone who doesn't write js or use node ... I've just had to fix the deployment of some node applications written by frontend devs a few times. I've seen npm trees with over 1000 modules to do some very trivial stuff. Node actually seems nice, and it has all the basics you need built in.

I typically use C or Python, which pretty much never use nested dependencies, and have never felt the desire to install/update/manage/debug exponentially more dependencies.

Re:Flat Tree Dependency Resolution in Npm v3 by feross on Thursday December 31, 2015 @12:45PM (#10818658)
I wouldn't recommend using fewer dependencies, but smaller ones. One of the best features of npm is that it solves the "dependency hell" problem. You can depend on modules freely and not worry about making it harder for users of your module to install it.

The issue comes when people create these huge "do-everything" modules. This is worth a read: https://github.com/sindresorhus/ama/issues/10#issuecomment-1...

Re:Flat Tree Dependency Resolution in Npm v3 by Touche on Thursday December 31, 2015 @04:17PM (#10819730)
That just creates a situation where you depend on an insane number of dependencies and your npm install takes 10 minutes.
Re:Flat Tree Dependency Resolution in Npm v3 by Silhouette on Thursday December 31, 2015 @10:53PM (#10821049)
One of the best features of npm is that it solves the "dependency hell" problem. You can depend on modules freely and not worry about making it harder for users of your module to install it.

That statement is true in theory, as long as everyone follows semantic versioning and dependencies never have unexpected interactions because of bugs.

Unfortunately, in the real world, neither of these conditions holds perfectly. The moment you depend on any module in your package.json that directly or indirectly depends on any other module where any version is not precisely fixed, you are immediately running uncontrolled and probably untested (by you) code. Moreover, if anything does go wrong, you may also have considerable difficulty identifying and reproducing the exact combination of versions of everything that caused the failure.

I have spent far too many days over the past few months tracking down exactly these kinds of issues, sometimes in some of the best known and most popular modules available via npm. For example, as the article itself mentions, what you wind up with in your node_modules directory can depend on install order under the new scheme. I don't think this is a good thing.

I therefore agree with those who advocate minimising dependencies, within reason of course. I also advocate strict version numbering and 100% reproducible builds, which is unfortunately quite hard to achieve once indirect dependencies start entering the picture with Node and npm, under either the old or the new scheme.

Re:Flat Tree Dependency Resolution in Npm v3 by tracker1 on Thursday December 31, 2015 @06:33PM (#10820334)
where this really starts to bite in my mind, is when you wind up seeing multiple modules in different dependencies that do the same thing.

Lodash, Underscore, Ramda and other come to mind in particular here... It's really easy to wind up with at least two of those three. Part of why I wouldn't mind seeing a few "winners" even if slightly larger win out. We're at a point where we get to tear jQuery out of projects, but it's easy enough to wind up with bundles that are larger than if we'd just used jQuery or Zepto, or whatever.

Re:Flat Tree Dependency Resolution in Npm v3 by city41 on Thursday December 31, 2015 @11:55AM (#10818402)
Why do dev communities have to continually pay the Windows tax? If Microsoft keeps insisting on being an outlier in the world of OSes, shouldn't they be the ones burdened?

I gotta say, as someone who never ever uses Windows but maintains a couple of open source packages, I'm really sick of the Windows only problems that crop up.

Re:Flat Tree Dependency Resolution in Npm v3 by tvanantwerp on Thursday December 31, 2015 @12:01PM (#10818437)
As someone who has no choice but to use Windows at work, I'm really sick of all the problems that crop up because so many tools use were written by people who won't use Windows and haven't tested anything on it.
Re:Flat Tree Dependency Resolution in Npm v3 by tsuraan on Thursday December 31, 2015 @12:32PM (#10818593)
That's a funny point of view. Windows is made by a multi-billion dollar company, rented by countless wealthy software development shops, but it's volunteer open-source programmers who are supposed to take on the burden of supporting it? Seriously, I can't even comprehend that thought process.

I write software for fun, I release it on github, and some people find it useful. If there are other people who don't, they can improve it for their own purposes, or they can find or write something better. I don't think anybody has a right to expect me to support Microsoft's product. That's insane.

Re:Flat Tree Dependency Resolution in Npm v3 by Silhouette on Thursday December 31, 2015 @11:19PM (#10821086)
I don't think anybody has a right to expect me to support Microsoft's product. That's insane.

True enough, though it's also fair to consider a language/platform/ecosystem inferior for some work if it relies on that kind of individual and often less portable contribution for its effectiveness. A lot of FOSS advocates will promote community support and a broad contributor base as advantages of that style of development, but there is another side to that coin, which is that sometimes you get just what you (didn't) pay for and you're on your own in terms of support or working around any problems.

Re:Flat Tree Dependency Resolution in Npm v3 by saosebastiao on Thursday December 31, 2015 @12:13PM (#10818491)
If I had no choice but to use Windows at work, and using Windows meant I couldn't use a large amount of development tools that I rely on, then the last ones I would blame would be the open source developers that won't use or develop for Windows. Your employer is the problem here.
Re:Flat Tree Dependency Resolution in Npm v3 by krisdol on Thursday December 31, 2015 @12:14PM (#10818497)
Why do you expect developers to pay for software, OSes, and licenses they don't use in order to develop open source software for free? Help them out by contributing, or blame Microsoft who has the resources to make their own OS more compatible

Doesnt Microsoft have their own V8 or node.js fork now?

Re:Flat Tree Dependency Resolution in Npm v3 by ploxiln on Thursday December 31, 2015 @12:22PM (#10818535)
Kinda funny isn't it, after a decade where one might be required to use a proprietary VPN or email client that only runs on windows. And those things didn't use standard, published, or even consistent protocols, nor were they open-source. In fact they were hostile to compatible implementations. They had to be reverse-engineered.

So now you get just a little, watered-down taste of using the non-preferred platform for something.

Re:Flat Tree Dependency Resolution in Npm v3 by city41 on Thursday December 31, 2015 @12:13PM (#10818487)
I feel your pain, but am not convinced open source maintainers should have to be the ones to resolve the issue. Especially since so many projects are done using spare time and with no compensation.
Re:Flat Tree Dependency Resolution in Npm v3 by ag_dubs on Thursday December 31, 2015 @12:47PM (#10818668)
hi! i totally know this is a common feel for many npmjs users. this is why i've been running an empathy campaign for Windows, Windows Wednesday, where i do all my work (with/for npm) on windows. it's been good for finding bugs, either technical or UX. i'm working on improving the docs to make them friendlier to all platforms (not just OSX, which is the most common platform at npmjs).

npmjs cares about the enterprise + windows.

Re:Flat Tree Dependency Resolution in Npm v3 by city41 on Thursday December 31, 2015 @12:59PM (#10818737)
I think this is the wrong approach. We shouldn't pick up MS's slack, MS should. This just enforces the current situation and keeps enabling MS to ignore the problem.
Re:Flat Tree Dependency Resolution in Npm v3 by ag_dubs on Thursday December 31, 2015 @06:41PM (#10820361)
you might also consider the fact that OSX machines are expensive. and the education for learning linux is generally expensive or not accessible. so supporting windows also supports ... poor people, people from different countries that have different economies, etc.

should we not pick up capitalism's slack? capitalism sure isn't going to pick up its own slack, err... systemic method of restricting underprivileged/poor people's access to power.

Re:Flat Tree Dependency Resolution in Npm v3 by fapjacks on Friday January 01, 2016 @11:28PM (#10825032)
> the education for learning linux is generally expensive or not accessible

Wow, this is incredibly wrong. The cost of education to learn Linux is living in a country where you are likely to be literate and have access to a computer. That is literally the cost to learn Linux, guy. There is nothing like an MSDN subscription for Linux documentation.

Re:Flat Tree Dependency Resolution in Npm v3 by steveklabnik on Thursday December 31, 2015 @01:37PM (#10818918)
What problem is that? That they have a different operating system?

npm is also a bit different, in your objection: they're a company. Companies need users. Supporting more platforms to get more users seems like pretty much, well, business as usual.

Re:Flat Tree Dependency Resolution in Npm v3 by city41 on Thursday December 31, 2015 @01:47PM (#10818974)
That is a fair point. npm being a company does change the situation.
Re:Flat Tree Dependency Resolution in Npm v3 by gelasio on Thursday December 31, 2015 @01:22PM (#10818848)
So, are you of the opinion that OSS projects should never be built for any commercial, proprietary OS or just Windows (i.e. because it's not Unix)?

For instance - OS X window managers suck. Do you think that a project like Spectacle should have never been built because they're just picking up Apple's slack?

Re:Flat Tree Dependency Resolution in Npm v3 by city41 on Thursday December 31, 2015 @01:36PM (#10818914)
I don't see how a project like Spectacle, which is OSX specific, relates. If someone wants to make an OS specific project, more power to them. The issue arises on platforms that span OSes, and they just never quite span onto Windows correctly.

Microsoft is very happy that Node, Java, Clojure, etc work on Windows. It helps them a lot. But they make little to no effort to help bridge the gap, they usually leave that to everyone else. That's my gripe, right there.

And to be fair, this is not a black and white situation. It's complex and Microsoft's role in all of this is complex too. And sometimes this issue does crop up on other OSes, for example Docker on OSX is sometimes a little painful.

Re:Flat Tree Dependency Resolution in Npm v3 by gelasio on Thursday December 31, 2015 @02:07PM (#10819086)
"But they make little to no effort to help bridge the gap, they usually leave that to everyone else."

OK, but that's also where you're wrong, right there. In general, but also particularly for this exact issue - Microsoft considered attempting to fix the file path limit for the first release of Windows 10. Maybe you don't understand, but it's going to take a herculean effort to fix this. They want to do it and if you need proof of that I can find posts of them talking about what I just said.

Re:Flat Tree Dependency Resolution in Npm v3 by martindevans on Thursday December 31, 2015 @03:06PM (#10819391)
Don't UNC paths solve this problem already (30000+ char limit)?
Re:Flat Tree Dependency Resolution in Npm v3 by tracker1 on Thursday December 31, 2015 @06:46PM (#10820377)
The problem is the default/legacy APIs don't use the new paths... in .Net I had this problem a lot, and serious issues every method I tried to work around it would hard crash VS, let alone the application I was working on... in the end, after days trying the solution was to say f*ck it and just put the directory I was working with closer to the root of the drive in question.

Why they can't fix .Net under the covers to use UNC in windows to avoid the class of problem I don't know, understand or comprehend... and that's just one very common platform for windows. I would say the same for npm/node and libuv for that matter.

I use bash a lot in windows (installed with msysgit), and that's got it's own set of problems. More problematic is the number of node modules that rely on bash scripts for builds, that don't work at all in windows.... which makes it very hard to contribute.

Re:Flat Tree Dependency Resolution in Npm v3 by nawitus on Thursday December 31, 2015 @03:06PM (#10819390)
I believe Microsoft worked to get Node.js supported on Windows.
Re:Flat Tree Dependency Resolution in Npm v3 by steveklabnik on Thursday December 31, 2015 @04:26PM (#10819776)
Not just that, but they're working on making Node work with Chakra.
Re:Flat Tree Dependency Resolution in Npm v3 by Touche on Thursday December 31, 2015 @04:36PM (#10819824)
Can they make it so I can spawn a process and not worry about whether I'm spawning on unix or Windows?
Re:Flat Tree Dependency Resolution in Npm v3 by steveklabnik on Saturday January 02, 2016 @01:59PM (#10827351)
I don't know it at that level of detail.
Re:Flat Tree Dependency Resolution in Npm v3 by Maarten88 on Thursday December 31, 2015 @04:13PM (#10819710)
Yes, npm is very problematic/buggy on Windows. I wasted many hours because of https://github.com/npm/npm/issues/9696 (meaning any build on visual studio online breaks ~50% of the time simply running 'npm install').

Microsoft should finally fix the MAX_PATH issue in Windows, but npm should just fix their software: it's too important to be that buggy.

Re:Flat Tree Dependency Resolution in Npm v3 by gelasio on Thursday December 31, 2015 @01:05PM (#10818766)
It sounds like you're getting annoyed that the devs of certain projects like Node.js have chosen to support an OS that you don't use without asking you about it first... If you're in control of the project - you have every right to ignore Windows users. If you're not in control, then you really have zero right to complain. Zero.

Devs certainly aren't forced to "pay the Windows tax" in any way. Plenty of OSS projects completely ignore Windows. I'm sure you're aware though that Windows has the majority market share of desktop operating systems. If your software is interesting to the vast, vast population of Windows users, you might get some requests to support Windows. And since it's not an outlier in the market, other desktop OSes should obviously strive to be more like Windows ;)

Re:Flat Tree Dependency Resolution in Npm v3 by city41 on Thursday December 31, 2015 @01:22PM (#10818840)
Windows is very much an outlier in the world of software development. Unless you're specifically using Microsoft tech (ie .NET and all that), then you're in a world dominated by Linux and OSX.

I'm annoyed that Microsoft insists on going forward with a proprietary OS that does not conform to standards that pretty much every single other OS does. Sure, platform specific issues do happen for other OSes, but they are rare. They are extremely common for Windows.

For sure, I have no obligation to fix Windows specific issues in my projects. All of my projects are done in my spare time and given away for free. But Windows specific issues are still annoying and disappointing.

Another aspect of this that is annoying is that major, mission critical open source projects have to dedicate so many resources to Windows. For example this bug in ClojureScript: https://github.com/clojure/clojurescript/commit/80d46bdb7969...

Imagine if the Clojure team didn't have to worry about file separator differences, and the multitude of other Windows annoyances. How much extra energy, time and resources would they gain? I suspect a lot.

EDIT: And I would also say I have every right to complain about Microsoft. Ignoring standards and doing their own thing for their own financial gain at everyone else's expense is definitely worthy of complaint. Also see Internet Explorer.

Re:Flat Tree Dependency Resolution in Npm v3 by moron4hire on Thursday December 31, 2015 @02:35PM (#10819222)
>> Windows is very much an outlier in the world of software development. Unless you're specifically using Microsoft tech (ie .NET and all that)

I think you need to get out more. Or at least go to meetups that don't only meet at coffee shops. You're cutting out a huge chunk there, and there are still huge chunks that use Windows to do Java, C++, etc.

I mean, game development is a multi billion dollar industry that is almost exclusively Windows-but-not-.NET, outside of consoles, and even on consoles it's still Windows in a large number of cases.

>> I'm annoyed that Microsoft insists on going forward with a proprietary OS that does not conform to standards that pretty much every single other OS does.

This is different from Apple with OSX and iOS how? This is different from Google with Android how? This is different from Canonical with Unity how? This is different than RedHat and Debian and pretty much every other major Linux distro with systemd how?

And for crying out loud, file separator issues? That's not even a hard one.

Re:Flat Tree Dependency Resolution in Npm v3 by city41 on Thursday December 31, 2015 @05:35PM (#10820100)
>> This is different from Apple with OSX and iOS how?

I'll never run "npm install" or "gem install" or what have you on iOS, Android, etc. Proprietary OSes come in all shapes and sizes, all bringing their own pros and cons to the mix. I just happen to be focusing on a very large con of Windows at the moment.

As for OSX, Redhat, Debian, etc. Sure, they're all different, of course. But generally speaking, at least when it comes to Node and npm (and as far as I know, Ruby, Clojure, Python, etc, please correct me if I'm wrong), these OSes tend to get along pretty OK. Windows, does not.

>> And for crying out loud, file separator issues? That's not even a hard one.

And yet how many times has that problem been hit over the decades? I mean heck, look at the commit message, "Another Windows path issue". And yeah, it's not even a hard one, it just gets worse from there.

I came to that issue from here (which is my project): https://github.com/city41/reagent-breakout/issues/11

So for "not even a hard one", still took me 20 minutes of digging to find out what was going on. X minutes across all developers across how long Windows has been around really adds up. For just one of Windows's differences.

Re:Flat Tree Dependency Resolution in Npm v3 by moron4hire on Thursday December 31, 2015 @06:14PM (#10820258)
I don't think you realize just how different even various Linux distros running even the same kernel can be, and I think you're giving waaay too much credit to OSX/Linux for being "alike". MAKE build scripts have notoriously been a mess for decades because of the differences between distros.

Any reasonable language has the ability to query all of the platform-specific stuff directly from the standard library. You can't even count on two Windows users having their home directories in the same place, or two Arch users having all of their config files in the same place. It's a necessity to be able to abstract all of these away even for a single operating system, which means you get it for free when moving to other operating systems.

Re:Flat Tree Dependency Resolution in Npm v3 by city41 on Thursday December 31, 2015 @07:49PM (#10820611)
I do get that. I totally understand all OSes are very different. How often those differences surface really depends on a lot of things, mostly at what level of the OS you are working in. Node modules and apps tend to stay pretty high level. But again, I still stand by my original statement that Windows specific issues crop up more than any other OS. Just take a look at the README for node-gyp: https://github.com/nodejs/node-gyp -- yup in OSX you have to get Xcode and a few other OSX specific steps to arrive at gcc and make, but compare that to the Windows section ...
Re:Flat Tree Dependency Resolution in Npm v3 by gelasio on Friday January 01, 2016 @01:01PM (#10822592)
Other projects have to do a lot of work to support OS X too. So I guess I still don't see your point. Electron, for instance, had to program an entirely different object model to represent OS X's kludgy global menu bar when every other OS has menus attached to application windows.

I can't believe your defending OS X though when Apple makes you buy a whole computer from them to use it...meanwhile I can run a 6 month trial of Windows (over and over again) on any virtual machine. Honestly, it's way more work to support OS X than Windows if you really bother to look at it.

Re:Flat Tree Dependency Resolution in Npm v3 by city41 on Friday January 01, 2016 @02:25PM (#10822971)
Apple is definitely not innocent here. They force a lot of terrible decisions too. I think ultimately though, this conversation boils down to my bias and experience versus your bias and experience.
Re:Flat Tree Dependency Resolution in Npm v3 by gelasio on Saturday January 02, 2016 @01:40PM (#10827242)
Sure, everybodys bias is painfully obvious here on HN.

But I don't think that negates the fact that to develop anything at all for OS X I have to buy a whole computer from Apple, since it's damn near impossible to get a stable OS X experience by running it in a VM or directly on non-Apple hardware. Furthermore, remotely accessing Xcode on a Mac from a non-Mac computer is really painful since the only option is VNC - the bottom of the barrel, lowest common denominator of remote screen sharing protocols.

Re:Flat Tree Dependency Resolution in Npm v3 by moron4hire on Friday January 01, 2016 @10:02AM (#10822029)
There is no reason building Windows software has to be this hard, other than the maintainer just not putting even a reasonable amount of effort into it.

When developers are hostile torwards Windows, this is the result. This doesn't do anything to Microsoft, this just hurts other developers and users.

Re:Flat Tree Dependency Resolution in Npm v3 by city41 on Friday January 01, 2016 @02:18PM (#10822941)
gyp was created by Google for building Chrome. I would guess they've put a decent amount of effort into it.
Re:Flat Tree Dependency Resolution in Npm v3 by tracker1 on Thursday December 31, 2015 @06:39PM (#10820356)
Given windows is the vast majority of desktops, and that there is more variety between different Linux distros and OSX versions than there are in the decades of Windows APIs, I think you have things a little backwards.

I've done far more dev targeting linux deployments the past few years, and much of it is far nicer on *nix than windows... but your viewpoint seems to be kind of arrogant. I haven't worked for a company that has more than 100 employees that doesn't do most development on windows, even if that isn't the target for deployment. That includes two major financial institutions, some large internet services and many other smaller companies over the years.

Re:Flat Tree Dependency Resolution in Npm v3 by city41 on Thursday December 31, 2015 @07:46PM (#10820595)
I don't mean to be arrogant, and if I am honestly it comes from ignorance. I've worked in MS shops before (I did .NET, COM, etc for nearly a decade). This whole thing blew up out of frustration at Windows specific issues cropping up more often than any other OS in environments like Node. I do think that's hard to deny. If that's arrogant, then I don't know what to say :-/
Re:Flat Tree Dependency Resolution in Npm v3 by Silhouette on Thursday December 31, 2015 @11:12PM (#10821073)
There is a real problem here, but I think you're laying the blame a little unfairly at Microsoft's doorstep. It's probably fair to say that at least until relatively recently Microsoft has gone to far greater lengths to ensure compatibility and longevity of code build on its platforms than any other organisation in the history of software development.

We used to value writing portable software as a skill, we used to value languages and libraries with robust specifications that could be used to write portable code, and portable code is also relatively future-proof code. I've worked on large projects that shipped on literally a dozen or more different platforms at any given time and were maintained for well over a decade with the significant variations in platforms that happen over that kind of time frame. Those projects built probably 95+% of the same source code for each platform, with platform-specific APIs and conventions carefully isolated.

That attitude and the related skills seem to have been much less valued in recent years, not least by the Linux community. (What, you want to build this C or C++ code with a tool chain other than GCC and friends?) If people carelessly scatter platform-specific code all over their projects, then of course they won't be easily portable, but more often than not such limitations are entirely artificial and could easily be avoided at negligible cost. In my experience, this is also true of a lot of libraries with the likes of Node and Python where native code is used in managed packages, and I think it's fair to consider that the resulting portability limitations are a potential disadvantage when choosing the language for a project.

Re:Flat Tree Dependency Resolution in Npm v3 by jameshart on Thursday December 31, 2015 @04:27PM (#10819784)
Seems to me that reasonable people could disagree over whether POSIX should be considered the be-all and end-all of OS standardization.
Re:Flat Tree Dependency Resolution in Npm v3 by tonyarkles on Friday January 01, 2016 @12:05PM (#10822379)
I wouldn't call it the be-all and end-all, but rather the bare minimum. "If you implement this API spec, there's a good chance a lot of software can successfully build"
Re:Flat Tree Dependency Resolution in Npm v3 by frik on Thursday December 31, 2015 @12:21PM (#10818527)
Microsoft initially aimed POSIX compatibility with Windows NT 3.1. There were POSIX subsystems for WinNT 3-5.1. Though, things changed and got worse.
Re:Flat Tree Dependency Resolution in Npm v3 by um_ya on Thursday December 31, 2015 @12:22PM (#10818533)
Better yet, why don't they just make a tool that fixes the windows problem, as a standalone patch. No need to change NPM, just need to fix the problem on Windows... The whole flat folder thing looks really bad.
Re:Flat Tree Dependency Resolution in Npm v3 by tehnorm on Thursday December 31, 2015 @12:46PM (#10818663)
Aimed higher is correct. I've always wondered why no one looked at the way git does this under the covers and applied it to npm. A directed graph that points to (with a sym link or similar mechanism) the correct package version on the file system.

That way you have the best of both worlds - dependencies don't collide and a single copy of a package version on the file system in node_modules.

Re:Flat Tree Dependency Resolution in Npm v3 by tracker1 on Thursday December 31, 2015 @06:47PM (#10820380)
The biggest bug is that Windows Explorer gui has troubles with long file paths.
Re:Flat Tree Dependency Resolution in Npm v3 by Klathmon on Thursday December 31, 2015 @12:00PM (#10818430)
I love the path that some take which is to maintain both a "pure" JS-only solution and a "native" solution, and have the native one be optionally installed. That way it works as best it can automatically.

Now I know that can't always be the case as it adds a lot of work to the development and maintenance, but it is a really nice thing when i see it.

Re:Flat Tree Dependency Resolution in Npm v3 by DrJokepu on Thursday December 31, 2015 @12:59PM (#10818734)
Meh. Anyone who ever worked with the v8 C++ interop API knows that it's a rather unpleasant experience (unlike e.g. Python's C interop API, which is very nice). Nobody uses it out of pure laziness, but because They Have A Very Good Reason To Do So™.