News for hackers. Stuff that matters.
Re:Dev corrupts NPM libs 'colors' and 'faker', breaking thou... by heluser on Sunday January 09, 2022 @05:20PM (#29867904)
To resolve such issues the central maven repo, for example, makes artifacts immutable when you publish them
Re:Dev corrupts NPM libs 'colors' and 'faker', breaking thou... by mjlawson on Sunday January 09, 2022 @05:31PM (#29867991)
This is true for npm. After the incident with leftpad, you can't unpublish anymore. You can, however, publish a new patch update that completely breaks everything.
Re:Dev corrupts NPM libs 'colors' and 'faker', breaking thou... by throw_m239339 on Sunday January 09, 2022 @05:39PM (#29868080)
> This is true for npm. After the incident with leftpad, you can't unpublish anymore. You can, however, publish a new patch update that completely breaks everything.

You absolutely can unpublish, it just requires more steps. If NPM gets a DMCA takedown request they will absolutely have to fulfill it.

Re:Dev corrupts NPM libs 'colors' and 'faker', breaking thou... by dane-pgp on Sunday January 09, 2022 @05:55PM (#29868248)
> If NPM gets a DMCA takedown request they will absolutely have to fulfill it.

Assuming the package is released under a Free Software licence, what grounds would there be for a DMCA takedown?

I suppose a developer could include the lyrics to a pop song in their code (possibly encrypted), and then tell the copyright holder about it (since I don't think you can make a DMCA request on behalf of a copyright holder without their permission), but I would hope that such a poison-pill would be caught long before the package became widely depended on.

Perhaps you're thinking someone would risk perjury(?) charges for making a false DMCA request against their package, and NPM would act on the request without questioning it; but remember that NPM is owned by Microsoft and they have previously stood up to frivolous DMCA requests (after a fashion)[0]. That article has the lede: "Software warehouse also pledges to review claims better, $1m defense fund for open-source coders".

[0] https://www.theregister.com/2020/11/16/github_restores_youtu...

Re:Dev corrupts NPM libs 'colors' and 'faker', breaking thou... by josephcsible on Sunday January 09, 2022 @06:15PM (#29868455)
> I don't think you can make a DMCA request on behalf of a copyright holder without their permission

In theory, you're right. In practice, there's never any actual consequences for filing a false DMCA claim. Worst case is that the thing doesn't get taken down, but that's no worse than if they didn't file it at all.

Re:Dev corrupts NPM libs 'colors' and 'faker', breaking thou... by labster on Sunday January 09, 2022 @08:07PM (#29869332)
Corps don’t care about DMCA takedowns from natural persons. I sent a takedown once, the CEO replied that he was sorry it had come to that, but they still distributed it for years under a license I did not grant. This CEO is licensed to practice law in California, btw.
Re:Dev corrupts NPM libs 'colors' and 'faker', breaking thou... by mlyle on Sunday January 09, 2022 @10:49PM (#29870375)
Anyone is free to ignore DMCA notifications.

Some parties that are distributing other peoples' stuff lose a safe-harbor protection from liability themselves if they ignore it.

This means intermediaries who don't benefit much directly from distributing a given bit of content will immediately comply with the DMCA takedown process. But this does nothing if you send the notice to someone who is actually using it.

The correct move is to send DMCA to the infringer's ISP/host. Then the ISP has to take it down unless counter-notified that they say they're not infringing. In turn, that counter-notification improves your position for any litigation that may ensue.

Re:Dev corrupts NPM libs 'colors' and 'faker', breaking thou... by jonas21 on Sunday January 09, 2022 @05:59PM (#29868279)
Someone could have added non-free third-party code into the package (intentionally or inadvertently, it doesn't really matter).
Re:Dev corrupts NPM libs 'colors' and 'faker', breaking thou... by pc86 on Sunday January 09, 2022 @06:21PM (#29868502)
> but I would hope that such a poison-pill would be caught long before the package became widely depended on.

I'm not sure what about the current open source ecosystem makes you think anyone would catch something like this.

Re:Dev corrupts NPM libs 'colors' and 'faker', breaking thou... by zdragnar on Sunday January 09, 2022 @06:43PM (#29868677)
Funny, my company couldn't use Webpack 1 because a dependency of a dependency... depended on an ancient package from the days when it was common to not bother with attaching a license.

Legally, that meant that noone could use it. In practice, nobody but our legal department cared, so we had to wait for version 2 when the dependency chain was updated to remove it.

Re:Dev corrupts NPM libs 'colors' and 'faker', breaking thou... by Dylan16807 on Sunday January 09, 2022 @07:37PM (#29869117)
You couldn't override the package locally? Or was too much of that code actually needed?
Re:Dev corrupts NPM libs 'colors' and 'faker', breaking thou... by inetknght on Sunday January 09, 2022 @06:11PM (#29868413)
> I don't think you can make a DMCA request on behalf of a copyright holder without their permission

Tell that you Youtube's copyright trolls

Re:Dev corrupts NPM libs 'colors' and 'faker', breaking thou... by dane-pgp on Sunday January 09, 2022 @06:37PM (#29868632)
The people trolling YouTube over copyright are either making false Content ID claims[0] (not DMCA takedown requests), or claiming infringement based on an incorrect match of something they genuinely hold the copyright on.[1]

You're probably right, though, that there is enough imprecision in the system for someone to claim that someone else's code snippet infringes on the copyright of a code snippet the claimant had previously published.

[0] https://torrentfreak.com/u-s-indicts-two-men-for-running-a-2...

[1] https://freebeacon.com/culture/google-youtube-algorithm-copy...

Re:Dev corrupts NPM libs 'colors' and 'faker', breaking thou... by lmm on Sunday January 09, 2022 @07:25PM (#29869037)
> Assuming the package is released under a Free Software licence, what grounds would there be for a DMCA takedown?

Noncompliance with the license, e.g. by removing required copyright notices/attribution in the code (this has happened in the past). Or straight-up uploading someone else's non-free code.

Re:Dev corrupts NPM libs 'colors' and 'faker', breaking thou... by bombcar on Sunday January 09, 2022 @09:00PM (#29869687)
The developer can DCMA claiming the code doesn’t follow his license as famously happened with Bukkit (the Minecraft server tool).
Re:Dev corrupts NPM libs 'colors' and 'faker', breaking thou... by dane-pgp on Monday January 10, 2022 @04:51PM (#29883180)
I didn't remember that particular legal complication, so thanks for prompting me to look it up. It seems that his argument was that Bukkit couldn't be distributed because it contained Mojang's proprietary code, but the fact that it also contained some of his code meant that he was a copyright holder for the purposes of the DMCA.[0]

This seems like an edge case that wasn't anticipated by the DMCA, but I can see the argument that mixing GPL code with proprietary code is creating and distributing a derivative work, in violation of the GPL. Without proprietary code being present, though, I don't think a developer can DMCA takedown their own GPL software.

[0] "As the Minecraft Server software is included in CraftBukkit, and the original code has not been provided or its use authorized, this is a violation of my copyright." https://github.com/github/dmca/blob/master/2014/2014-09-05-C...

Re:Dev corrupts NPM libs 'colors' and 'faker', breaking thou... by mjlawson on Sunday January 09, 2022 @05:46PM (#29868175)
Not only does it require more steps, it also has to meet the following criteria[1]:

* no other packages in the npm Public Registry depend on

* had less than 300 downloads over the last week

* has a single owner/maintainer

So while your point is taken that unpublishing is possible under some circumstances, it is not for popular packages that are in use today.

[1] https://docs.npmjs.com/policies/unpublish

Re:Dev corrupts NPM libs 'colors' and 'faker', breaking thou... by throw_m239339 on Sunday January 09, 2022 @05:51PM (#29868208)
None of these points have any legal standing, from a copyright perspective.

https://news.ycombinator.com/item?id=29868199

Re:Dev corrupts NPM libs 'colors' and 'faker', breaking thou... by mjlawson on Sunday January 09, 2022 @06:11PM (#29868412)
You are technically correct. The best kind of correct! In practical terms, it depends on the license used. Since most licenses used in open source will prevent you from making these kind of requests, this consequence isn't likely to have any practical implications.
Re:Dev corrupts NPM libs 'colors' and 'faker', breaking thou... by kd5bjo on Sunday January 09, 2022 @07:02PM (#29868834)
You are assuming that the true rights holders of all the code in the package actually agreed to the given license. Someone unrelated to the package development can still claim it includes an illegally-copied, unlicensed version of their code.
Re:Dev corrupts NPM libs 'colors' and 'faker', breaking thou... by tomrod on Sunday January 09, 2022 @07:41PM (#29869143)
Despite the need to keep it clear, copyright does not reign supreme.
Re:Dev corrupts NPM libs 'colors' and 'faker', breaking thou... by throw_m239339 on Sunday January 09, 2022 @08:15PM (#29869379)
> Despite the need to keep it clear, copyright does not reign supreme.

neither do NPM TOS, or whatever Microsoft thinks they are entitled to, since NPM is owned by Microsoft.

Re:Dev corrupts NPM libs 'colors' and 'faker', breaking thou... by tomrod on Sunday January 09, 2022 @08:21PM (#29869421)
Which is not what I argued :^)
Re:Dev corrupts NPM libs 'colors' and 'faker', breaking thou... by dragonwriter on Sunday January 09, 2022 @07:33PM (#29869092)
> If NPM gets a DMCA takedown request they will absolutely have to fulfill it.

No, they don't. Honoring DMCA takedowns allow benefit from an additional safe harbor from any existing infringement liability for the alleged infringing content, but are not mandatory in their own.

Re:Dev corrupts NPM libs 'colors' and 'faker', breaking thou... by dathinab on Sunday January 09, 2022 @05:44PM (#29868144)
Except if they have reason to believe the code was uploaded with the permission of the copyright holder.

The they have gotten the right for npm to distribute the source code in context of npm.

Re:Dev corrupts NPM libs 'colors' and 'faker', breaking thou... by throw_m239339 on Sunday January 09, 2022 @05:49PM (#29868199)
> The they have gotten the right for npm to distribute the source code in context of npm.

There is absolutely no copyright or publishing right transfer that takes place when one "publishes" a package on NPM (or on Github). None.

The original author is absolutely entitled to a DMCA takedown notice and NPM would have to oblige him.

Re:Dev corrupts NPM libs 'colors' and 'faker', breaking thou... by Dylan16807 on Sunday January 09, 2022 @07:42PM (#29869151)
Your Content belongs to you. You decide whether and how to license it. But at a minimum, you license npm to provide Your Content to users of npm Services when you share Your Content. That special license allows npm to copy, publish, and analyze Your Content, and to share its analyses with others. npm may run computer code in Your Content to analyze it, but npm's special license alone does not give npm the right to run code for its functionality in npm products or services.
Re:Dev corrupts NPM libs 'colors' and 'faker', breaking thou... by dathinab on Sunday January 09, 2022 @06:14PM (#29868444)
First you agreed to ToS when you uploaded things to npm. I haven't read the terms but it should be enough for npm to publish on npm no matter the license.

Secondly and as important if you publish something under an Open Source license(1) then you _cannot unpublish it_. You granted copyright to _everyone_ for and existing both now and in the future to distribute and use it(2) (legally it's a bit more complex but that's what it boils down to).

(1): Assuming you had the legal right to do so, but if not you are liable for any fall out, not npm (because ToS, they still need to take it down reasonable fast, but they might be able to sue you).

(2): Within the constraints of the license.

Re:Dev corrupts NPM libs 'colors' and 'faker', breaking thou... by mschuster91 on Sunday January 09, 2022 @05:57PM (#29868260)
You can't legally retract opening up software source code under most if not all popular open source licenses.
Re:Dev corrupts NPM libs 'colors' and 'faker', breaking thou... by josephcsible on Sunday January 09, 2022 @06:07PM (#29868370)
It is indeed all, even if you ignore the "popular" qualifier. If a license could be unilaterally revoked, it would fail to meet the Open Source Definition for that reason.
Re:Dev corrupts NPM libs 'colors' and 'faker', breaking thou... by throw_m239339 on Sunday January 09, 2022 @06:14PM (#29868442)
open source =/= free software.

That's the first mistake you are making.

Re:Dev corrupts NPM libs 'colors' and 'faker', breaking thou... by vurpo on Monday January 10, 2022 @06:34AM (#29873240)
The differences between the two are extremely minimal, basically only relating to patent rights relating to the software. Go read https://www.gnu.org/philosophy/free-sw.en.html - the FSF's Free Software definition, and https://opensource.org/osd - the Open Source Definition (both by the respective parties that coined the terms and maintain them to this day) and see what the actual differences are. They're not many.
Re:Dev corrupts NPM libs 'colors' and 'faker', breaking thou... by josephcsible on Monday January 10, 2022 @02:32AM (#29871636)
While they are indeed sightly different, I fail to see how the differences are at all relevant in this context.
Re:Dev corrupts NPM libs 'colors' and 'faker', breaking thou... by wbl on Sunday January 09, 2022 @05:41PM (#29868102)
Why does a new version break projects without action by the project owners? In Go you would have to explicitly update to the broken version.
Re:Dev corrupts NPM libs 'colors' and 'faker', breaking thou... by bspammer on Sunday January 09, 2022 @05:52PM (#29868219)
Because npm install has the insane default behavior of adding a fuzzy qualifier to your package.json, for example ^6.0.2 means all of the following versions are accepted: 6.0.2, 6.0.9, 6.7.84
Re:Dev corrupts NPM libs 'colors' and 'faker', breaking thou... by K0nserv on Sunday January 09, 2022 @06:31PM (#29868579)
It’s not particularly insane. package.json and package-lock.json have different purposes, namely package.json specified intent e.g. I want a version that satisfies >=5.2.3 && < 6.0.0 and package-lock.json records the exact resolved version.

Off the top of my head Bundler, CocoaPods, Cargo, SPM, Pipfile(and various other Python dependency managers), and composer also all work like this.

Cargo even makes it implicit that a version like “1” means “^1.0.0” in Cargo.toml.

Re:Dev corrupts NPM libs 'colors' and 'faker', breaking thou... by samhh on Sunday January 09, 2022 @06:31PM (#29868577)
That's not an issue, that assists in quickly viewing wanted package upgrades. The problem is in not using a lockfile.
Re:Dev corrupts NPM libs 'colors' and 'faker', breaking thou... by jbverschoor on Sunday January 09, 2022 @07:11PM (#29868931)
Welcome to JavaScript, where every division is a bad one
Re:Dev corrupts NPM libs 'colors' and 'faker', breaking thou... by ldjb on Sunday January 09, 2022 @06:15PM (#29868456)
Very often, package installation is automated as part of a build pipeline. So if you want to build and deploy a new version of your software, you'll kick off the pipeline and that could potentially download a newer version of a package than was previously being used.

Incidents like this highlight that this may not be the best idea.

Re:Dev corrupts NPM libs 'colors' and 'faker', breaking thou... by jffry on Sunday January 09, 2022 @06:21PM (#29868507)
If you're using NPM without lockfiles, you're gonna have a bad time with discrepancies between trying things on your dev machine and building things in CI machines.

When you have a package-lock.json NPM will install exactly the same version of everything in your dependency tree, making the CI builds much more like what's on your dev machine (modulo architecture/environment changes)

Re:Dev corrupts NPM libs 'colors' and 'faker', breaking thou... by inlined on Sunday January 09, 2022 @05:59PM (#29868277)
Because of version locks. Normally you install “^X.Y.Z” which means any version at major X with at least minor Y and revision Z. For more conservative codebases you install “~X.Y.Z” which also locks the minor.

npm install will traditionally install the most recent packages that match your constraints. You need “npm ci” to use true version locks

Re:Dev corrupts NPM libs 'colors' and 'faker', breaking thou... by david_allison on Sunday January 09, 2022 @05:55PM (#29868246)
*I revoke my comment. Child comment is correct.
Re:Dev corrupts NPM libs 'colors' and 'faker', breaking thou... by K0nserv on Sunday January 09, 2022 @06:11PM (#29868409)
That's not true.

The first line of NPM install's documentation[0] says(emphasis mine):

> This command installs a package, and any packages that it depends on. If the package has a *package-lock or shrinkwrap file, the installation of dependencies will be driven by that*, with an npm-shrinkwrap.json taking precedence if both files exist. See package-lock.json and npm shrinkwrap.

What does happen is: if you have added a new package in package.json it will be installed based on the semver pattern specified there, or if you run npm install some-package@^x.y.z the same thing happens. Further, if you modify package.json by changing the semver pattern for an existing package that will also cause this behaviour.

Running `npm install` in a package that already has a package-lock.json will simply install what's in package-lock.json. `npm install` only changes the lock file to add/remove/update dependencies when it detects that package.json and package-lock.json disagrees about the specified dependenices and their semver patterns e.g. having foo@^2.3.1 in package.json and [email protected] in package-lock.json will cause foo to be update when running `npm install`.

0: https://docs.npmjs.com/cli/v6/commands/npm-install

Re:Dev corrupts NPM libs 'colors' and 'faker', breaking thou... by maxwell86 on Sunday January 09, 2022 @05:43PM (#29868138)
THat's why you specify the exact version of your dependencies.
Re:Dev corrupts NPM libs 'colors' and 'faker', breaking thou... by pc86 on Sunday January 09, 2022 @06:08PM (#29868385)
You're never going to be able to prevent that at a technical level. You can prevent it with workflow, though: 1) sync packages locally and build from those versions; 2) peg to a specific version and don't auto-update; 3) deploy to a test environment and not directly to production.
Re:Dev corrupts NPM libs 'colors' and 'faker', breaking thou... by xienze on Sunday January 09, 2022 @05:44PM (#29868139)
A key difference with Maven projects is that you specify exact dependency versions instead of “always use latest” or some variant of that, as is pretty common in the Node world.
Re:Dev corrupts NPM libs 'colors' and 'faker', breaking thou... by sverhagen on Sunday January 09, 2022 @09:09PM (#29869756)
This is not necessarily true, there are version ranges: https://www.baeldung.com/maven-dependency-latest-version

Admittedly, I don't think it has nearly as wide a usage as it has in the NPM world. Dependabot (I know I'm not the first to mention it, here, today) is probably more of a factor.

Still, it strikes me that this sort of "attack" (or mishap) is exceedingly rare in the Java ecosystem, while it's pretty common in the NPM world, and I don't immediately understand why that would be so.

Re:Dev corrupts NPM libs 'colors' and 'faker', breaking thou... by xienze on Sunday January 09, 2022 @11:21PM (#29870585)
I was not aware of that feature. To call it rare would be an understatement I think.

> while it's pretty common in the NPM world, and I don't immediately understand why that would be so.

I think it boils down to Node projects typically specifying dependencies in the form “any version >= X”, effectively “always use the latest.” Dependencies can therefore get bumped silently just by rebuilding, essentially. Whereas in the Java world updating dependencies is a deliberate process.

Re:Dev corrupts NPM libs 'colors' and 'faker', breaking thou... by bombcar on Sunday January 09, 2022 @09:21PM (#29869851)
We abuse jitpack.io and MASTER-snapshot to keep out Minecraft maven builds up to date.
Re:Dev corrupts NPM libs 'colors' and 'faker', breaking thou... by dvdcxn on Monday January 10, 2022 @07:28AM (#29873765)
With lock files, you will always be stuck with whatever version you first installed until you explicitly ask npm to upgrade, or delete your lockfile.
Re:Dev corrupts NPM libs 'colors' and 'faker', breaking thou... by julianlam on Sunday January 09, 2022 @05:30PM (#29867984)
npm does as well, they made this change after left-pad.

You also can't unpublish once a single person has downloaded the package, I believe.

Re:Dev corrupts NPM libs 'colors' and 'faker', breaking thou... by infogulch on Sunday January 09, 2022 @05:40PM (#29868101)
Immutability feels like the best approach here. Go's module system is pretty good in this respect: "proxy" is just a proxy that serves module code, and "sum" is an append-only transparency log of the hashes of all published versions. You can't "unpublish" from the log, but you can get code hosted on proxy removed for various reasons... which users can protect themselves against by running their own proxy. Go's module version resolution strategy means that the chosen module version never changes without explicit input from the user so no "publish a new version that breaks everyone's CI" issue.

All together I don't see how GP's "email php files around" is as any better than this system in any way.

Re:Dev corrupts NPM libs 'colors' and 'faker', breaking thou... by detaro on Sunday January 09, 2022 @05:34PM (#29868025)
How does that solve the issue here of new broken versions of packages being published?
Re:Dev corrupts NPM libs 'colors' and 'faker', breaking thou... by oblio on Sunday January 09, 2022 @05:44PM (#29868146)
That's another JS ecosystem widespread malpractice.

Autobumping versions, or version ranges as they're called in Maven land.

Dependencies should only use fixed versions and all updates should be manual.

You should only use auto-upgradable versions during development, and the package manager should warn you that you're using them (or your dependencies are).

Re:Dev corrupts NPM libs 'colors' and 'faker', breaking thou... by codebje on Sunday January 09, 2022 @05:59PM (#29868283)
If package A depends on package C at version 1.0 but package B depends on C at version 1.1, what version of C will be pulled in?

Dependency management is not as simple as only upgrading one direct dependency at a time after careful review.

The NPM ecosystem is particularly difficult to work with as it has deep and broad transitive dependency trees, many small packages, and a very high rate of change.

You either freeze everything and hope you don't have an unpatched vulnerability somewhere or update everything and hope you don't introduce a vulnerability somewhere.

Re:Dev corrupts NPM libs 'colors' and 'faker', breaking thou... by throw_m239339 on Sunday January 09, 2022 @06:33PM (#29868599)
> Dependency management is not as simple as only upgrading one direct dependency at a time after careful review.

Most package managers won't allow these stunts and conflicts have to be resolved UPSTREAM. NPM chose to go the "YOLO" way and will fetch every single version of a package that meets the dependency demands. Terrible design, but the purpose of that was growth for NPM, the company, not the best interest of the ecosystem.

Re:Dev corrupts NPM libs 'colors' and 'faker', breaking thou... by oblio on Sunday January 09, 2022 @06:18PM (#29868483)
There are package exclusions, package forcing and of course, full dependency tree checks where you review what everything pulls in.

The JS ecosystem will probably have to change but because it's so decentralized, that change will be orders of magnitude harder than, for example, PHPs transition from 3 (4, 5) to 7.

Re:Dev corrupts NPM libs 'colors' and 'faker', breaking thou... by throw_m239339 on Sunday January 09, 2022 @06:34PM (#29868611)
> The JS ecosystem will probably have to change but because it's so decentralized,

Is it? Everybody is pulling from Microsoft owned servers now, as Microsoft owns both Github and NPM.

Re:Dev corrupts NPM libs 'colors' and 'faker', breaking thou... by oblio on Monday January 10, 2022 @01:20PM (#29879077)
You're right in the package storage sense.

I don't think you're right in the builder/building practices sense.

Re:Dev corrupts NPM libs 'colors' and 'faker', breaking thou... by dvdcxn on Monday January 10, 2022 @07:29AM (#29873778)
I'm sorry but this is completely wrong. NPM has lock files which explicitly lockdown the version you have downloaded after your first install. These are commited to source control, so all subsequent installs will use the exact same version of dependencies, and nested dependencies too.

You need to ask npm to upgrade or delete your lock file and node modules to run into this issue.

Re:Dev corrupts NPM libs 'colors' and 'faker', breaking thou... by ses1984 on Sunday January 09, 2022 @05:45PM (#29868152)
You shouldn't blindly pull updates into production, how do you know if a non-malicious update breaks your app if you don't do any basic testing first?
[dead]
Re:Dev corrupts NPM libs 'colors' and 'faker', breaking thou... by stickfigure on Sunday January 09, 2022 @06:50PM (#29868742)
Re:Dev corrupts NPM libs 'colors' and 'faker', breaking thou... by oblio on Monday January 10, 2022 @02:43AM (#29871702)
The tone changes half way, it wasn't all parody, though :-)

Thought it would be clear enough.

Re:Dev corrupts NPM libs 'colors' and 'faker', breaking thou... by detaro on Sunday January 09, 2022 @05:34PM (#29868018)
npm also has immutable artifacts.
Re:Dev corrupts NPM libs 'colors' and 'faker', breaking thou... by oblio on Sunday January 09, 2022 @05:40PM (#29868095)
Yes, and my question is: why didn't they have them from day 1???

Or at least day 1000? Npm was launched in 2010, 11 years ago, and I'm quite sure immutable packages were implemented about 3 years ago.

Again, this is not rocket science, we knew the attack angles.