News for hackers. Stuff that matters.
Re:Pika/web: Web Apps Without the Bundler by segphault on Tuesday March 19, 2019 @05:17PM (#19435990)
This solution pulls in 155 packages from npm, including multiple versions of bullshit packages like "kind-of" that it can't deduplicate. One of the direct dependencies is a library that renders a loading spinner in command line interfaces, which itself pulls in over 20 transitive dependencies.

The reason that JavaScript build tooling is so complex and introduces so much overhead is that the ecosystem's culture is fundamentally broken. Every single one of these tools is a raging dumpster fire because it's built on top of all of these layers of low-quality interdependent crap.

I'm increasingly moving towards module-type script tags and standard ES modules everywhere, with no build step during development. It's still challenging to use node-targeted modules in this fashion, so I really do appreciate that people are working on finding ways to make it work. I just wish that we could do it without pulling in so much third-party code and the large surface area for failure and security problems that go with it.

Re:Pika/web: Web Apps Without the Bundler by curry-castaway on Tuesday March 19, 2019 @11:35PM (#19438683)
> One of the direct dependencies is a library that renders a loading spinner in command line interfaces, which itself pulls in over 20 transitive dependencies.

Just trying to understand, is this a bad thing?

Someone else made an open source CLI spinner library which also uses other people's existing open source libraries. This saves a lot of time and gives developers many good options.

Should Pika write and maintain its own custom CLI spinner animations? Are you saying the CLI spinners should be standardized in the next version of ECMAScript itself?

How is this worse than the same thing written in Python, for example? (I mainly use javascript, so maybe I haven't been exposed to the kinds of alternatives you're thinking about.)

Re:Pika/web: Web Apps Without the Bundler by intertextuality on Wednesday March 20, 2019 @12:09AM (#19438894)
It’s bad because this leads to a standard React project having over 2,000 dependencies.

The real question is: do you -really- need an external lib with 20 dependencies just to show a freakin’ loading spinner? Remaking the wheel is bad but so is never making truly simple things yourself, or just not using them.

What happens when a common package breaks? What happens if it gets hijacked and becomes a security vector that’s impossible to spot because it’s loaded as the 567th package in a dependency tree?

The answer here is to have a strong stdlib where do you don’t need to pull in 3rd party packages all the time for trivial things, and not including a million small packages in every single project.

Re:Pika/web: Web Apps Without the Bundler by curry-castaway on Wednesday March 20, 2019 @12:21AM (#19438957)
> The real question is: do you -really- need an external lib with 20 dependencies just to show a freakin’ loading spinner? Remaking the wheel is bad but so is never making truly simple things yourself, or just not using them.

So the problem is the sheer number of dependencies? What is a reasonable upper limit?

Yes, javascript should continue to standardize commonly used features, but avoiding dependencies doesn't seem to be a solution.

If anything, more dependencies are a good sign because they imply that other people have spent more time and effort on a solution than anything you'll be able to hand-roll for single-use.

It sounds like the root issue here is just dependency management. If our package managers were solving this issue well enough, there should be no practical difference between 2 big dependencies with significant functionality (and more code to review) or 20 tiny, easy-to-review dependencies.

Re:Pika/web: Web Apps Without the Bundler by yardstick on Wednesday March 20, 2019 @01:26AM (#19439273)
From a security perspective, minimising dependencies is preferred. I have to review at least monthly all our dependencies for published vulnerabilities and new versions.

We don’t allow automatic upgrading of packages/dependencies due to the risk of malicious code making it in (see https://www.npmjs.com/advisories for examples). Yeah there are companies that will help manage your vulnerability process but it’s still a lot of overhead and only grows as the number of dependencies grows.

There’s also the whole left-pad mess from a few years back which shows you always need local archived copies of any dependencies you use.

Re:Pika/web: Web Apps Without the Bundler by curry-castaway on Wednesday March 20, 2019 @12:00PM (#19443494)
> We don’t allow automatic upgrading of packages/dependencies due to the risk of malicious code

That's a good idea, how do you do that?

> you always need local archived copies of any dependencies you use.

Are you committing your dependencies? Or using a package manager with caching?

Re:Pika/web: Web Apps Without the Bundler by Torgo on Thursday March 21, 2019 @10:45AM (#19452892)
>I have to review at least monthly all our dependencies for published vulnerabilities and new versions.

Off-topic, but can you write about how you manage this without tons of manual work?

Re:Pika/web: Web Apps Without the Bundler by ownagefool on Wednesday March 20, 2019 @04:56AM (#19440189)
If you care about security, you should evaluate your dependencies.

This preferably means reading the code you're pulling in but it's unrealistic that we're going to read 2000 constantly changing dependencies for every deploy, so you need to establish trust some how.

Reputation of maintainers, dep CVE scanning, SAST and protective monitoring can all add additional assurance, but they won't protect you from a random hijacked npm module, and the more you include and from more people, the more likely you'll be affected by a zero day.

Having an entire community depend on tiny libraries that do nearly nothing exaperates this problem, and if you use something that almost nobody else does, you're unlikely to be saved by npm audit.

I don't use node daily driver, but I assume npm audit growing to include reputations and frameworks owning more of the dependency tree will help, but the users of the system also need to be considerate of the risks they take and the trust they place.

Re:Pika/web: Web Apps Without the Bundler by z3t4 on Wednesday March 20, 2019 @05:31AM (#19440381)
I used to have all npm modules in source control (SCM), until npm introduced tree shaking. I'm using ZFS which have both de-duplication and compression, so I gain very little by tree shaking. I wish there was a way to disable tree shaking in npm, it's really the source of all evil. Anyway, I reviewed all code diffs after each npm update, very little changed, eg. it wasn't that much work. But it's now impossible as npm moves files around. I also deleted a lot of unnecessary files. About 90% of the content of each NPM packages are not needed.

Another reason why I stopped hosting dependencies in SCM is native modules. I wish Node.JS itself could become a platform layer so that I wouln't have to use these native modules.

Another thing is compile to JS, where a tiny change in the source might cause a huge diff in the generated JS.

Re:Pika/web: Web Apps Without the Bundler by curry-castaway on Wednesday March 20, 2019 @12:06PM (#19443561)
I committed my dependencies until recently too, and I've been trying to figure out a better alternative. Are you doing something else now rather than committing dependencies? (Right now I'm just using a hodgepodge of npm scripts to lock things down.)
Re:Pika/web: Web Apps Without the Bundler by intertextuality on Wednesday March 20, 2019 @03:06AM (#19439709)
There is an enormous difference between 2 large packages and 20 small ones. I can see the main authors more clearly, and not have to worry about 20 different packages being compromised. Your statement makes no sense. More packages does not, in any way, shape, or form, correlate to “better quality”. The sole thing it shows is that the author pulled in more packages. That’s it. Whether it’s good or bad or secure or not secure is only determined by analysis. The author could easily have been super lazy and pulled in 18 small packages instead of writing small helper functions manually, etc. There is no defensible argument for more packages and bloat. None. As I said above, the core lib should focus on providing functionality so pulling in a fancy spinner library means you pull in ONE library, not 20.
Re:Pika/web: Web Apps Without the Bundler by jaabe on Wednesday March 20, 2019 @01:59AM (#19439426)
It’s fine from a development perspective where the job is to make something work.

It’s horrifying from an operations perspective where the job is to make sure everything works.

Developers can afford to ignore looking into dependencies, operations need to make sure every dependency is functional and safe.

If you write a piece of C# using the standard .Net library you can be fairly sure it’s safe and sound. If you write something using 2000 JS packages, you have to read through every one of them to be sure.

Re:Pika/web: Web Apps Without the Bundler by ownagefool on Wednesday March 20, 2019 @05:02AM (#19440228)
I disagree that it's the job of a random ops person to ensure that a developers dependencies are sane.

Putting the onus on the developer to do a good job with regards to secure development practices is an essential part of a wider system.

Re:Pika/web: Web Apps Without the Bundler by jaabe on Wednesday March 20, 2019 @07:06AM (#19440881)
Oh I agree with that completely, but it’s very easy for developers to get away with shitty practices in a lot of shops.
Re:Pika/web: Web Apps Without the Bundler by __oh_es on Wednesday March 20, 2019 @12:29PM (#19443809)
Agree with you but this goes the other way too - Ops not allowing security patching or upgrading of systems because "they work".
Re:Pika/web: Web Apps Without the Bundler by pier25 on Tuesday March 19, 2019 @06:04PM (#19436520)
I've said before and I'll say it again, we wouldn't be in this mess if we had a decent standard library and language to begin with.
Re:Pika/web: Web Apps Without the Bundler by preommr on Tuesday March 19, 2019 @08:32PM (#19437670)
The way it seems to work with JS is that the community develops a bunch of stuff and the best rises to the top. This then influences the official language spec. ES6 basically made a lot of the popular tools redundant because it was heavily inspired by them.

This is so incredibly important because the main development of the language is so poorly handled.

Re:Pika/web: Web Apps Without the Bundler by andreareina on Wednesday March 20, 2019 @11:58AM (#19443475)
And despite adding arrow functions, async/await, modules etc. we still don't have sprintf and a Math.round that can specify the number of places to round to.

I think that modern javascript is pretty good and I'm grateful for the syntax extensions that have effected that, but the standard library is crying out for some love.

Re:Pika/web: Web Apps Without the Bundler by rtfeldman on Tuesday March 19, 2019 @08:23PM (#19437618)
> a decent standard library and language

This is what a lot of people love about Elm: it's a lovely language with a first-rate standard library.

The Elm ecosystem is smaller than the JS ecosystem, but of course that's a hard requirement of a nicer foundation.

More info: https://elm-lang.org

Re:Pika/web: Web Apps Without the Bundler by toastal on Wednesday March 20, 2019 @01:40AM (#19439327)
Elm still needs `andMap` for `<*>`/`apply` on basically all core packages though.
Re:Pika/web: Web Apps Without the Bundler by h1d on Tuesday March 19, 2019 @07:42PM (#19437359)
Agree. JS standard library has been very immature and was so slow to improve as it was primarily targeted for browser and people thought that wasn't the place to do complicated stuff, not even md5.

I still have some array helper methods just to shuffle array or randomly pick one.

I think it's time to put everything in future JS.

Re:Pika/web: Web Apps Without the Bundler by zdragnar on Tuesday March 19, 2019 @06:29PM (#19436732)
We had several other languages once upon a time, but all were proprietary and most only worked through plugins. All were eventally removed from default installs in browsers due to awful security and mobile performance.
Re:Pika/web: Web Apps Without the Bundler by pier25 on Tuesday March 19, 2019 @07:28PM (#19437266)
I was a Flash dev for some 10 years and I can count with one hand the number of third party libraries I regularly used.

I don't disagree about the security issues, mobile performance, and energy consumption though, but as a dev I preferred the experience of writing AS3 apps by a long shot.

Re:Pika/web: Web Apps Without the Bundler by jnbiche on Tuesday March 19, 2019 @09:37PM (#19438066)
> if we had a decent standard library and language to begin with.

I use compile to JS languages when I can, but not sure what now you'd want for a frontend JavaScript standard library beyond what's available in ES6.

Re:Pika/web: Web Apps Without the Bundler by pier25 on Wednesday March 20, 2019 @11:12AM (#19442963)
Optional static types, enums, data binding, reactive primitive, etc.

Yeah TS has many of those, but TS is not JS and it introduces new problems.

Re:Pika/web: Web Apps Without the Bundler by est on Tuesday March 19, 2019 @10:09PM (#19438242)
js should not be used in web "app" to begin with.

Not every god damn webpage has to be as complicated as Gmail.

Re:Pika/web: Web Apps Without the Bundler by Scarbutt on Tuesday March 19, 2019 @09:38PM (#19438072)
JS standard library is npm, it has weaknesses but it is its biggest strength IMO.
Re:Pika/web: Web Apps Without the Bundler by inimino on Tuesday March 19, 2019 @09:59PM (#19438181)
A standard library should be well-documented, well-maintained, and trusted because it comes from a known source. This is everything npm is not.
Re:Pika/web: Web Apps Without the Bundler by Scarbutt on Tuesday March 19, 2019 @10:10PM (#19438253)
Library rot is a well known thing in the python/ruby/java/php std libs, although I always hear good things about Go's std lib, they had the advantage of a fresh start and have the resources to keep the std lib well maintained.
Re:Pika/web: Web Apps Without the Bundler by frou_dh on Tuesday March 19, 2019 @05:59PM (#19436459)
I like the philosophy of no required build step. Once you make that decision, it's like a guardrail keeping you away from the tooling circus.

Do you know of any listings of third-party ES Modules that are designed to be used directly? i.e. their full URLs being present in import statements that the browser sees at runtime. Even toy stuff would be interesting to look at.

Re:Pika/web: Web Apps Without the Bundler by MuffinFlavored on Tuesday March 19, 2019 @05:51PM (#19436382)
> with no build step during development

and then what do you do for production, webpack?

Re:Pika/web: Web Apps Without the Bundler by segphault on Tuesday March 19, 2019 @05:57PM (#19436436)
I use rollup with an extremely minimal configuration. No transpilers at all, just standards-based JavaScript.
Re:Pika/web: Web Apps Without the Bundler by anonytrary on Tuesday March 19, 2019 @06:09PM (#19436568)
Sorry if this is a dumb question -- but that means you don't write ES6?
Re:Pika/web: Web Apps Without the Bundler by segphault on Tuesday March 19, 2019 @07:28PM (#19437264)
At this point, ES6 support is sufficiently widespread that I don't really have a problem using it natively. Major features like the const, classes, template literals, and async functions are all at least at 90% market penetration.

I can get away with not supporting Internet Explorer, though I understand that it's not a trade-off that everybody is comfortable making.

Re:Pika/web: Web Apps Without the Bundler by ioulian on Wednesday March 20, 2019 @04:29AM (#19440072)
I think (thought) also like you, just use the latest es6 code to do everything. The workflow is very fast, I can write my app 20%-50% faster than es5. I develop mostly on Chrome. My application/website works.

Then we test the website on different browsers (we all work with OSX), FireFox works (of course), Opera works (is Chromium), Safari, naah, it's the new IE. We do some minor fixes for Safari.

When we send it to the client, nothing works, of course the client uses IE so we need to rewrite a lot of stuff to make it work for "most" people. (I live in Belgium, there are "a lot of" IE users still. And even if IE is not widely used, our clients always use IE :( )

But wait, why rewrite our application, this can be fully automated! We run our code through babel/webpack. It automagically works everywhere!

Now I understand there is some performance penalty by using transpilers, but using them leaves our code mostly bugfree on all browsers and the client is happy.

The big problem with those fancy new features is browsersupport. If every browser followed specs correctly, we wouldn't need these tools. They make the job of the developer easy, the bosses wallet full and the client happy.

Re:Pika/web: Web Apps Without the Bundler by Touche on Tuesday March 19, 2019 @06:20PM (#19436662)
It doesn't mean that, they might only support ES6 browsers (even IE11 has some ES6 support).
Re:Pika/web: Web Apps Without the Bundler by toastal on Wednesday March 20, 2019 @01:43AM (#19439341)
The support is really shallow, but using `let` and `const` is still superior to `var` where applicable.
Re:Pika/web: Web Apps Without the Bundler by anonytrary on Tuesday March 19, 2019 @06:31PM (#19436761)
Ah, thanks that's true... touche...