I wrote the above before caffeine really kicked in, so I neglected to add: pnp is itself achieved in part by obfuscating your entire dependency tree. That takes a loooot of trust, on a matter where trust has already exceedingly deteriorated. In hindsight, I regret even considering it.

Can you clarify what you mean by "obfuscating the dependency tree"?

They probably mean the idea that once you're in PnP, you can "kind of, sort of" peer into zipfs deps, but not in the same way that was possible in bare node_modules.

That said, I think yarn 2's PnP + zero installs (https://yarnpkg.com/features/zero-installs) is lovely with CI. Instead of tacking 40+ seconds to resolve deps every build, vending deps with PnP on is much cheaper than the node_modules equivalent.

A gigantic single file encoded version of `node_modules` is just modules I can’t go look in.

(Not a real edit): my last gig was with a lot of well prepared juniors, but they lacked the confidence to go look inside dependencies to find out what was happening. I tried to encourage setting breakpoints or logging or whatever felt comfortable in required packages. It was hard.

Turning that into a blob is even more discouraging.

As I mentioned elsewhere in the thread, Yarn v2 does have an "unplug" command that will extract a given dep into a folder for the time being. Does help with that use case.

And if you trust that they’re fundamentally the same thing, that’s a great escape hatch. I personally tried to use two new things together and discovered that one is transparent and one is opaque magic... and given the opportunity to do harm, I found the opacity of one alarming. I don’t trust yarn to manage dependencies in pnp, because what I saw in how they handled a special case was completely black box. Literally binary blob patches with no explanation of what it’s doing or why. Completely impossible to audit without reverse engineering or auditing the entire tool. Why would I trust “unplug” to do anything but misdirect?

What "binary blob patches" are you referring to?

FWIW, if I wanted to confirm whether an "unplugged" package had been modified, I'd just download the original tarball from NPM, extract it, and diff the two folders.

I mean the way that yarn 2 “installs” typescript is by patching it with some manually maintained base64 blob that (I assume) corresponds in some way to the base64 blob that pnp produces. Both are probably something you can reverse engineer... if that’s how you want to trust your package manager I guess? Idk I only learned that the patching was a thing because it failed when I tried to install an “unsupported” package. I was alarmed by trying to track down what was happening and saw the patch has no explanation. I was more alarmed when yarn2 tried to apply the patch even with pnp disabled.

Hmm. Okay, digging around in the Yarn repo, I see this "generate patch" setup code [0]. Looks like they're trying to cherry-pick some specific commits from the TS repo based on the TS version, and specifically apply them to the TS file.

The "base64" bit is referenced here [1].

I would assume this specifically relates to the fact that TS does not have native support for Yarn PnP as a filesystem approach. The Yarn team has been keeping an open PR against TS [2] and trying to convince the TS maintainers to merge it, but it hasn't happened yet.

A bit odd, and I can understand why you're concerned, but it also looks like there's a very understandable reason for this.

I would have assumed that this doesn't get applied if you install TypeScript via the Yarn v2 `node_modules` linker, but would have to try it out and actually see.

[0] https://github.com/yarnpkg/berry/blob/f384f0f40e87d636e4021b...

[1] https://github.com/yarnpkg/berry/blob/f384f0f40e87d636e4021b...

[2] https://github.com/microsoft/TypeScript/pull/35206

This blob is literally our open PR, applied to the various TS releases. You can rebuild it using `gen-typescript-patch.sh` (we actually have a GH Action that does this on CI, to prevent malicious uncontrolled changes), and the sources are auditable in my PR.

Note that it gets applied regardless of the linker since it would cause the cache to change when going from a linker to another, and we wanted to make the experience smoother, but that it's a noop for non-PnP environments.

It definitely applies the patch even with the node_modules linker. That was why I gave up and went back to yarn 1

Sorry if this makes it harder but I honestly recommend reading up on pnpm (https://pnpm.js.org/) before committing to npm7. Npm7 auto-installs peer dependencies(!) and pnpm has some remarkable advantages over npm or yarn.

Pnpm is indeed better than npm but I found it’s symlinking approach less compatible than yarn v2, (nextJS for example didn’t support pnpm until very recently) while also having less deterministic module resolution, creating version compatibility problems that disappeared with yarn v2.

I had already been meaning to look at it but had written it off because I wanted what yarn 2 was selling... but definitely gonna give it another look.