Assuming we survive the gap period where every country chucks what they still have at their worst enemies, I mean. I suppose we can always hit each other with animal bones.

Rivers caught on fire for a hundred years before the EPA was formed.

This is one force that operates. Another is that, in an effort to avoid depending on such a big attack surface, people are increasingly rolling their own code (with or without AI help) where they might previously have turned to an open source library.

I think the effect will generally be an increase in vulnerabilities, since the hand-rolled code hasn't had the same amount of time soaking in the real world as the equivalent OS library; there's no reason to assume the average author would magically create fewer bugs than the original OS library authors initially did. But the vulnerabilities will have much narrower scope: If you successfully exploit an OS library, you can hack a large fraction of all the code that uses it, while if you successfully exploit FooCorp's hand-rolled implementation, you can only hack FooCorp. This changes the economic incentive of funding vulnerabilities to exploit -- though less now than in the past, when you couldn't just point an LLM at your target and tell it "plz hack".

Also, even seemingly trivial libraries can have bugs. The infamous leftpad library didn't handle certain edge doses properly.

For supply chain security and bug count, I'll take a focused custom implementation of specific features over a library full of generalized functionality.

> Also, even seemingly trivial libraries can have bugs. The infamous leftpad library didn't handle certain edge doses properly.

This isn't really an argument in favour of having the average programmer reimplement stuff, though. For it to be, you'd have to argue that the leftpad author was unusually sloppy. That may be true in this specific case, but in general, I'm not persuaded that the average OSS author is worse than the average programmer overall. IMHO, contributing your work to an OSS ecosystem is already a mild signal of competence.

On the wider topic of reimplementation: Recently there was an article here about how the latest Ubuntu includes a bunch of coreutils binaries that have been rewritten in Rust. It turns out that, while this presumably reduced the number of memory corruption bugs (there was still one, somehow; I didn't dig into it), it introduced a bunch of new vulnerabilities, mostly caused by creating race conditions between checking a filesystem path and using the path for something.

Because it might grow in future and you want to allow flexibility for that, because it might be the input to or output from some external system that requires XML, because your team might have standardised on always using XML config files, because introducing yet another custom plain text file format just creates unnecessary cognitive load for everyone who has to use it are real-world reasons I can think of.

But really I was just looking for a concrete example where I know the complexity of the implementation has definitely caused vulnerabilities, whether or not the choice to use it to solve the problem at hand was sensible. I have zero love for XML.

The race conditions were indeed TOCTOU bugs. In a sense, the bugs were a result of incorrectly handling shared mutable data, though in this case the mutations were external to Rust.

https://corrode.dev/blog/bugs-rust-wont-catch/

    module.exports = leftpad;
    
    function leftpad (str, len, ch) {
      str = String(str);
    
      var i = -1;
    
      ch || (ch = ' ');
      len = len - str.length;
    
    
      while (++i < len) {
        str = ch + str;
      }
    
      return str;
    }

Both old and new versions return a string longer than `len` if the padding char is multiple characters, e.g. leftpad('a', 3, '&&&&') will be longer than 3. That feels like it shouldn't happen.

Have you read this old code? It's terrible and written with no care at all to security often in C. AI is much much better at writing code.

But I think most OSS code isn't like this -- even C code born long ago, if it's still in wide use, has been hardened by now. Examples: Linux kernel, GNU userland, PostgreSQL, Python.

There have been two LPE vulnerability and exploits in the Linux kernel announced today. After the one announced just last week. I don't think as much of the C code born long ago has been as carefully hardened as you think.

(Copy Fail 2 and Dirty Frag today, and Copy Fail last week)

You (anyone, not you personally) write that much code yourself and let's see how well you did in comparison.

Admittedly, not hard to do, but it could save some other folks.

If I'm to be offended by a single thing in your post that is calling me (names) - is AI Bro. This was undeserved, and cannot be farther from the truth. Not to miss the fact your comment is entirely off topic, and perhaps you see AI bros everywhere now.

* with internet access to FOSS via sourceforge and github we got an abundance of building blocks

* with central repositories like CPAN, npm, pip, cargo and docker those building blocks became trivially easy to use

Then LLMs and agents added velocity to building apps and producing yet more components, feeding back into the dependency chain. Worse: new code with unattributed reuse of questionable patterns found in unknowable versions of existing libraries. That is, implicit dependencies on fragments multitude of packages.

This may all end well ultimately, but we're definitely in for a bumpy ride.

Auto-installing random software is the problem. It was a problem when our parents did it, why would it be a good idea for developers to do it?

yolo!

I run a distro that often causes software like this to break because their silent automatic installation typically makes assumptions about Linux systems which don’t apply to mine. However I fear for the many users of most typical distros (and other OS’ in general as it’s not just a Linux-only issue) who are subject to having all sorts of stuff foisted onto their system with little to no opportunity to easily decide what is being heaped upon them.

Ruby gems are older than that, but I have no idea what capabilities it has/had.

(Obviously some developers are better or worse than others, so I presume your observation is assuming developer skill as a constant.)

1) As org size grows, it's the team's average quality that matters (so yes, large numbers).

2) Even with a single team, the velocity will increase to match the acceptable level of quality.

Management will push the accelerator until they get too many bugs, then it will be "we need fewer outages".

So, in an team+environment, you end up with a constant (in time) detection rate, which basically means a constant in time injection rate.

If the teams' velocity increases without increasing quality, the bug injection rate (and detection rate) will increase.

AKA if the AI is slightly worse, but 10x faster, stop carrying the pager. :)

We're seeing maintainers retreat from maintaining because the amount of AI slop being pushed at them is too much. How many are just going to hand over the maintenance burden to someone else, and how many of those new maintainers are going to be evil?

The essential problem is that our entire system of developing civilisation-critical software depends on the goodwill of a limited set of people to work for free and publish their work for everyone else to use. This was never sustainable, or even sensible, but because it was easy we based everything on it.

We need to solve the underlying problem: how to sustainably develop and maintain the software we need.

A large part of this is going to have to be: companies that use software to generate profits paying part of those profits towards the development and maintenance of that software. It just can't work any other way. How we do this is an open question that I have no answers for.

this is a cornerstone of modern software development. If it died, or if got taken over by a malicious entity, every single company on the planet would have an immediate security problem. Yet the experience of that maintainer is bad verging on terrible [1].

We need to do better than this.

[0] https://curl.se/docs/governance.html

[1] https://lwn.net/Articles/1034966/

>He emphasized that he has released curl under a free license, so there is no legal problem with what these companies are doing. But, he suggested, these companies might want to think a bit more about the future of the software they depend on.

There is little reason for minimal-restriction licenses to exist other than to allow corporate use without compensation or contribution. I would think by now that any hope that they would voluntarily be any less exploitative than they can would have been dashed.

If you aren't getting paid or working purely for your own benefit, use a protective license. Though, if thinly veiled license violation via LLM is allowed to stand, this won't be enough.

There's a bunch of problems with getting companies to pay for this, too - that sense of entitlement (or even contractual obligation), the ability to control the project with cash, etc.

I don't have any answers or solutions. But I don't think we can hand-wave the problem away.

Like you get when you buy e.g. MS products?

/s

I don't agree with any of that.

This opposed to closed off “products” that change at the whims of the company owning it.

There’s a lot of misconception about how the open source comes to be and very small part, still significant of course, of it was really created for the benefit of a community. There are exceptions, but dig the organisational culture and origins and you’ll see the pattern. Also, thousands of projects are made for the satisfaction of the author himself being highly intelligent and high on algorithmic dopamine.

Nuclear might be airgapped but what about water, power…?

Right now it kinda feels to me like "Open Source" is the Russian army, assuming their sheer numbers and their huge quantity of equipment much off which is decades old.

Meanwhile attackers and bug hunters are like the Ukrainians, using new, inexpensive, and surprisingly powerful tools that none of the Open Source community has ever seen in the past, and for which it has very little defence capability.

The attackers with cheap drones or LLMs are completely overwhelming the old school who perhaps didn't notice how quickly the world has changed around them, or did notice but cannot do anything about quickly enough.

Who exactly is the innocent little Ukraine supposed to be that the big bad open source is supposed to be attacking to, what? take their land and make the OSS leader look powerful and successful at acheiving goals to distract from their fundamental awfulness? And who are the North Korean canon fodder purchased by OSS while we're at it?

Yeah it's just like that, practically the same situation. The authors of gnu cp and ls can't wait to get, idk, something apparently, out of the war they started when they attacked, idk, someone apparently.

I wasn't intending to pass judgement on which side is the "innocent little" and which is the "big bad", but I (and the downvoters) clearly see the it obviously reads one specific way.

I wish I'd chosen a less contentious example of a unarguably good army that's 50 or 100 years old and is still using tactics and equipment from the 70s and earlier, fighting against a somewhat less clearly "good" army using new tools that barely existed 5 years ago and new tactics that the older army (and everybody else) has never seem before with the capability to create new weapons and adjust tactics at speeds previously thought impossible. But that war doesn't exist (at least not outside of blindly loyal Russia supporters).

For the record, I believe Russia is clearly on the side of evil and Ukraine is clearly on the side of good in this conflict.

Russia is the aggressor, but in your metaphor they are the defenders.

> any useful piece of software has been fuzz tested, property tested and formally verified.

That would require effort. Human effort and extra token cost. Not going to happen, people want to rather move fast an break things.

More people are producing more code because of easier tools. Most code is bad. But that's not the tools fault.

And in the end it is a problem of processes and culture.

I am not disagreeing in the main, but I wonder about the net effect. Again, this is total speculation on my part. If I vibe-slop a half dozen apps this week (and I might, just you watch), the overall raw code quality in the universe got worse. But if in the space of the same time, two major security holes got patched (assume there was no net amount of code changed), didn't things actually get better?

Even an intelligence agency doesn't have perfect opsec, and something could get mentioned offhand somewhere on a forum, but never get picked up until the LLM uses it.

Edit: and, ofc, what we're discussing here is Linux packages.

I have one server that has shell users, and I did the "yum update" and "reboot -f" dance last week.

Was that good enough? Oh no.

Here we go again!

We should have:

- OS level capabilities. Launched programs get passed a capability token from the shell (or wherever you launched the program from). All syscalls take a capability as the first argument. So, "open path /foo" becomes open(cap, "/foo"). The capability could correspond to a fake filesystem, real branch of your filesystem, network filesystem or really anything. The program doesn't get to know what kind of sandbox it lives inside.

- Library / language capabilities. When I pull in some 3rd party library - like an npm module - that library should also be passed a capability too, either at import time or per callsite. It shouldn't have read/write access to all other bytes in my program's address space. It shouldn't have access to do anything on my computer as if it were me! The question is: "What is the blast radius of this code?" If the library you're using is malicious or vulnerable, we need to have sane defaults for how much damage can be caused. Calling lib::add(1, 2) shouldn't be able to result in a persistent compromise of my entire computer.

SeL4 has fast, efficient OS level capabilities. Its had them for years. They work great. They're fast - faster than linux in many cases. And tremendously useful. They allow for transparent sandboxing, userland drivers, IPC, security improvements, and more. You can even run linux as a process in sel4. I want an OS that has all the features of my linux desktop, but works like SeL4.

Unfortunately, I don't think any programming language has the kind of language level capabilities I want. Rust is really close. We need a way to restrict a 3rd party crate from calling any unsafe code (including from untrusted dependencies). We need to fix the long standing soundness bugs in rust. And we need a capability based standard library. No more global open() / listen() / etc. Only openat(), and equivalents for all other parts of the OS.

If LLMs keep getting better, I'm going to get an LLM to build all this stuff in a few years if nobody else does it first. Security on modern desktop operating systems is a joke.

Those exploits are in kernel, and the userspace is only calling the normal, allowed calls. Removing global open()/listen()/etc.. with capability-based versions would still allow one to invoke the same kernel bugs.

(Now, using microkernel like seL4 where the kernel drivers are isolated _would_ help, but (1) that's independent from what userspace does, you can have POSIX layer with seL4 and (2) that would be may more context switches, so a performance drop)

Yes they would. Copyfail uses a bug in the linux kernel to write to arbitrary page table entries. A kernel like SeL4 puts the filesystem in a separate process. The kernel doesn't have a filesystem page table entry that it can corrupt.

Even if the bug somehow got in, the exploit chain uses the page table bug to overwrite the code in su. This can be used to get root because su has suid set. In a capability based OS, there is no "su" process to exploit like this.

A lot of these bugs seem to come from linux's monolithic nature meaning (complex code A) + (complex code B) leads to a bug. Microkernels make these sort of problems much harder to exploit because each component is small and easier to audit. And there's much bigger walls up between sections. Kernel ALG support wouldn't have raw access to overwrite page table entries in the first place.

> (2) that would be may more context switches, so a performance drop

I've heard this before. Is it actually true though? The SeL4 devs claim the context switching performance in sel4 is way better than it is in linux. There are only 11 syscalls - so optimising them is easier. Invoking a capability (like a file handle) in sel4 doesn't involve any complex scheduler lookups. Your process just hands your scheduler timeslice to the process on the other end of the invoked capability (like the filesystem driver).

But SeL4 will probably have more TLB flushes. I'm not really sure how expensive they are on modern silicon.

I'd love to see some real benchmarks doing heavy IO or something in linux and sel4. I'm not really sure how it would shake out.

I prefer it’s model of declaring this is what I want to use, any calls to code outside that error out.

- Pledge requires the program drop privileges. Process level caps move the "allowed actions" outside of an application. And they can do that without the application even knowing. This would - for example - let you sandbox an untrusted binary.

- Pledge still leaves an entire application in the same security zone. If your process needs network and disk access, every part of the process - including 3rd party libraries - gets access to the network and disk.

- You can reproduce pledge with caps very easily. Capability libraries generally let you make a child capability. So, cap A has access to resources x, y, z. Make cap B with access to only resource x. You could use this (combined with a global "root cap" in your process) to implement pledge. You can't use pledge to make caps.

To me it’s easier to get a program to let the system know what it needs vs. try to contain it from the outside.

Anyway, have a good one.

One approach is "Trust No Code" and the other is "Trusted code should run safely".

the first one sounds better on paper, but leads to a very complicated system. That said, I haven't worked with jails much or other forms of sandboxing. It just seems to me that to make software function you need escape hatches, and the more of those you have, well, now you're back to plugging exploits with a more complicated system.

It was interesting to me to hear that even though OpenBSD had designed their software to limit permissions even before pledge and unveil were released - upon release they found that a shocking amount of their software actually wasn't following their own rules.

At what point do you trust the system? And if you don’t trust any of it what are you trying to accomplish?

Re OpenBSD: I think it just shows we’re all human(fallible) at the end of the day :)

Yeah. Its yet another reminder that "being really careful" isn't an adequate security policy. Attackers only need to find 1 bug. Defenders need to protect everything. In large systems, you need defence in depth. Pledge? Yeah. NX? Yeah. Process isolation between subsystems? Yeah lets have that too. Static verification? Love it. Rust's borrow checker? Sure. We need it all.

https://blog.plan99.net/why-not-capability-languages-a8e6cbd...

But as pointed out by others, this particular exploit wouldn't be stopped by capabilities. Nor would it be stopped by micro-kernels. The filesystem is a trusted entity on any OS design I'm familiar with as it's what holds the core metadata about what components have what permissions. If you can exploit the filesystem code, you can trivially obtain any permission. That the code runs outside of the CPU's supervisor mode means nothing.

The only techniques we have to stop bugs like this are garbage collection or use of something like Rust's affine type system. You could in principle write a kernel in a language like C#, Java or Kotlin and it would be immune to these sorts of bugs.

This essay only addresses my second point - capabilities within a program. It doesn't address OS level capabilities at all.

But even in the space of programming languages, I find this essay extremely unconvincing. Like, you raise points like this:

> Here are some problems you’ll have to solve in order to sandbox libraries: What is your threat model? How do you stop components tampering with each other’s memory?

The threat model is left pad cryptolockering your computer via a supply chain attack. The solution is to design a language such that if I import leftpad, then call it, my computer can't get hacked.

You stop components tampering with each others' memory by using a memory safe language.

> its main() method must be given a “god object” exposing all the ambient authorities the app begins with

So what? The main function already takes arguments. I don't understand the problem.

Haskell already passes a type object as an argument to anything which does IO. They don't do it for security. Turns out having pure functions separated from non-pure functions is a beautiful thing.

Then there's these weird claims:

> Any mutable global variable is a problem as it may allow one component to violate expectations held by another.

You don't need to ban mutable global variables! Lets imagine we did this in safe rust. I think the only constraint is that a global variable can't be shared over the boundary between crates. But - nobody does that anyway. Even if you did share a global over a crate boundary, the child crate would still only be able to access it through methods on the type.

Sneaky developers could leverage globals to violate the security boundary. But it would be hard to do by accident. Maybe just, don't do that.

Your essay talks about some research project making a capability based java subset. And I understand that the resulting ergonomics weren't very good. But that isn't evidence that capabilities themselves are a bad idea. If a research student wrote a half baked C compiler one time, you wouldn't take that as evidence that C compilers are a bad idea. I do, however, accept that the burden of proof is on me to demonstrate that its a good idea. I hope that I can some day rise to that challenge.

> The filesystem is a trusted entity on any OS design I'm familiar with

Thats not how capability based microkernels like SeL4 work. The filesystem is owned by a specialised process. Other processes only modify files by sending messages to the filesystem process via a capability handle. If nobody created a writable file handle, the file can't be arbitrarily mutated by another module. Copyfail happened because in linux, any code can by default interact with the page table. One piece of code was missing access control checks. In capability based systems, its basically impossible to accidentally forget access control checks like that.

> The only techniques we have to stop bugs like this are garbage collection or use of something like Rust's affine type system. You could in principle write a kernel in a language like C#, Java or Kotlin and it would be immune to these sorts of bugs.

Copyfail is a logic bug. C#, Java or Kotlin wouldn't save you from it at all.

> The solution is to design a language such that if I import leftpad, then call it, my computer can't get hacked.

That requirement may seem clear right now, but the moment you talk to other people about your language you'll find there's no agreement on what "get hacked" means. Some people will consider calling exit(0) repeatedly to be "hacked" because it's a DoS attack, others will say no code execution or priv escalation happened, so that's not being hacked. Some will say that left-pad being able to read arbitrary bytes from your address space is being hacked, others will say no harm done and thus it wasn't being hacked. The details matter and you need to nail them down in advance.

It turns out for example that one of the top uses of the Java SecurityManager was just to stop plugins accidentally calling System.exit() and tearing down the whole process. It wasn't even a security goal, really.

> You stop components tampering with each others' memory by using a memory safe language.

That's not enough. See languages like Ruby or JavaScript, which are memory safe but not sandboxable due to all the monkeypatching they allow.

> Haskell already passes a type object as an argument to anything which does IO. They don't do it for security. Turns out having pure functions separated from non-pure functions is a beautiful thing.

But almost nobody uses Haskell, partly because of poor ergonomics like this! So if you want a language that gets wide usage and has a good library ecosystem, monads for everything probably isn't going to take off.

> If nobody created a writable file handle, the file can't be arbitrarily mutated by another module.

We're talking about critical bugs in the filesystem so what the FS processes idea of a file handle is doesn't really matter. If you can confuse or buffer overflow the FS process by sending it messages, you can then edit state inside that process you weren't supposed to be able to access, and as that process controls the security system for everything it's game over. Microkernels have no way to stop this, which is one reason very few operating systems move the core FS out into a separate process. You can't easily survive a crash of the core FS code, and it being exploited is equivalent to an exploit of the core microkernel anyway in terms of adversarial goals. So you might as well just run it in-kernel and reap the performance benefits.

> But almost nobody uses Haskell

Sad, but true

> partly because of poor ergonomics like this!

I'm somewhat dubious that's the reason, partly because I find such ergonomic excellent! Especially those provided by my capability system Bluefin: https://hackage.haskell.org/package/bluefin

The copyfail bug wasn’t a bug in the filesystem code. It was a bug in the crypto algorithm code, which wrote to the filesystem page table without checking if the process invoking it had permission to write to the passed file handle. In a monolithic kernel like Linux, every subsystem can access the memory of every other subsystem by default. It’s up to each subsystem to be careful. As we keep discovering, “be really careful” is not a successful security strategy.

A capability based OS like SeL4 is more secure. With SeL4, you would put the crypto algorithms and filesystem in separate user space processes. These processes would only communicate by RPC, by invoking capabilities. We can imagine how the copyfail scenario would play out: A user process has a capability representing its (read only) access to some privileged file on disk. It passes that capability to the crypto algorithm process. A bug - or even complete takeover - of the crypto algorithm process still doesn’t change that the file cap is read only. The crypto algorithm process doesn’t have direct access to the memory representing that file. It only has the read only file handle. All it can do with that handle is invoke it, which will only give it read access. Even with a bug in the crypto algorithms process, the OS would stay secure.

Yes, capability OSes aren't a magic bullet. A bug in the filesystem process could still result in filesystem corruption. But better is better. OS capabilities provide defence in depth. They would have prevented copyfail.

As far as I can tell, your argument against capabilities is that they might be slow. Some implementations have poor ergonomics. They don’t magically solve every possible security bug. You also, personally, used a bad implementation of capabilities this one time years ago in Java. Is that accurate?

You must see how unconvincing I find your argument. What are you even trying to do? Convince people to not explore different ideas in computer science? When I close my eyes I see an old man yelling: “Hey you kids! What are you doing up there, trying new things? You stop that right now!”

I don't think there are any good mainstream capability based programming languages. At least I've never seen one. Actually the SecurityManager is I think the best implementation that has existed. I've not yet seen a credible proposal that's better. Stuff like Mojo and SEL4 is at least deployed to production but that's not a programming language.

> What are you even trying to do? Convince people to not explore different ideas in computer science?

No. Please go read the opening of the article again, which says: "In this essay I want to show you the challenges that you’ll face if you want to walk that path. This isn’t meant to put anyone off, just to draw a map of the territory you’re about to enter and explain why it’s currently deserted."

Lots of people have proposed capabilities as some silver bullet over the years, yet real systems hardly use them. Anyone who is serious about their own ideas should want to understand why that is and that's the goal of the article. It doesn't say nobody can do better! The whole point of writing it, is the hope that someone will. But to do better you have to understand why existing systems failed. It wasn't (primarily) about performance.

The assumption here is that the FS is the root of trust for the kernel. (A claim I consider dubious, but what do I know about knowing things?) It's another way to say that if you don't harden your root of trust, you're SOL. Which, ok, fair enough. But that's frankly irrelevant because hardening the root of trust is table stakes. The system cannot be secured without it, regardless of the threat model.

All of the concerns about a definition of "getting hacked" falls out of ignoring the hardening of the root of trust. I don't wish to put words in your mouth, but my interpretation of the argument is essentially, "we can't have nice things because the root of trust cannot be hardened sufficiently to prevent all intrusions."

Iff the FS is the root of trust, and it is not possible to confuse the FS by sending it messages, then there is no game over. You have a root of trust that cannot be broken.

> Microkernels have no way to stop this, which is one reason very few operating systems move the core FS out into a separate process.

My reading of the history reaches a very different conclusion. First, the primary reason that very few operating systems in practice use a microkernel design is because Linus Torvalds believed it was too slow for early 90's hardware [1]. And everyone else just does whatever Linux is doing.

Second, security through surface area reduction (and more broadly, defense-in-depth) was always the point of the microkernel design [2]. Trivially, the principle of least privilege is how one arrives at a secure system. Monolithic kernels, to this very day, continue to prove that they cannot be secured in any practical manner. I can only assume we need things to get worse before kernel developers will tighten up and take security seriously.

> So you might as well just run it in-kernel and reap the performance benefits.

There's that same mentality. Apparently "speed at all costs" is the willful trading of security for performance. That position is just as flawed as trading essential liberty for temporary safety [3]. It doesn't matter how fast the thing is when the slightest bump always causes it to explode, killing everyone on board.

[1]: https://web.archive.org/web/20040210002251/http://people.flu...

[2]: https://www.cosy.sbg.ac.at/~clausen/PVSE2006/linus-rebuttal....

[3]: https://old.reddit.com/r/todayilearned/comments/k0c8o6/til_b...

You're completely right that the root of trust has to be secured. I argue that the core filesystem is indeed a part of the ROT, which is why e.g. Apple has put so much effort into making it immutable and fully tied to a cryptographic root hash that's checked by the secure boot process. Moving the FS out of the core kernel wouldn't change much though - if you have a bug in your FS code at runtime then you're just SOL even if everything is arranged in a Merkle tree.

The argument being made by josephg in the sibling comment is that in SEL4 or similar the page cache would be separated from the crypto code. And maybe he's right, but the better way to get the same outcome is to not have IPsec in the kernel rather than not have the core FS - as the latter is a ROT and IPsec isn't.

I disagree that the question of what "getting hacked" means is a reformulation of trust roots. A threat model isn't the same thing as a root of trust. The argument over what appears to be minor semantics is important because it scopes your goals and effort. One of the most common failure modes I've seen in security projects is not defining a threat model up front, often leading to an automatic fallback to "the threat model contains everything" followed by despondency and failure when it turns out to be impossible.

I don't think Apple or Microsoft care much about Linus' opinions tbh. Both NeXT/macOS and Windows NT started out as microkernel designs and all of them have oscillated back and forth over the years. The original concept was indeed far too slow and a lot of functionality went back to monolithic. Then over time some functionality got lifted back out e.g. the GUI subsystem on Windows. Core FS remains though in any OS as the cost/benefit ratio of moving it is so poor.

There is some truth to this idea, though. Setting aside the unsafe boundary, (FFI, direct MMIO access, etc.) a capability system in a programming language would solve some kinds of these problems. Not all; it doesn't solve logic bugs when a capability is in scope.

> It's always posited as obvious and easy

I do believe it's probably pretty obviously true, by now. But not at all easy.

> Moving the FS out of the core kernel wouldn't change much though - if you have a bug in your FS code at runtime then you're just SOL even if everything is arranged in a Merkle tree.

Perhaps, but that's only because traditional file systems are global state. A capability system turns that notion on its head specifically because global state is really the problem. The combination of capabilities and user mode file access would be quite a strong isolation boundary. The bug(s) would have to be "trivially flawed" in a way that these subtle exploits are not.

> A threat model isn't the same thing as a root of trust.

Ah, I didn't say that. I said (roughly) that security relies on a strong root of trust for every thread model. I think the distinction is important. They are not the same, but the thread model can be completely ignored (because it doesn't matter) until the root of trust is secured. In other words, a weak root of trust fails all threat models.

> I don't think Apple or Microsoft care much about Linus' opinions tbh.

True. macOS and NT are (or were?) "microkernel-ish" the last time I was in those weeds. No idea how they've evolved since.

You've made some good points, as well. I see where you are coming from.

But those claims have also got to be moderated. As no such thing has ever existed, we can't truly know how well it'd work in practice. Only experience can tell us that.

Global state is one of the key issues. Joe-E simply banned it, which is far too harsh and breaks almost everything. Mobile operating systems locked down filesystem access behind permissions and capabilities quite dramatically and were much more secure, but that came with a lot of 'vigorous' debate over owner control and power for productivity/pro-grade applications. macOS has taken an incremental approach and sandboxes off parts of the FS from apps whilst retaining what looks on the surface like a classical global shared state $HOME and / directory (although it's not).

macOS, iOS, Android and Windows have all been steadily moving code out of the kernel over the years. Apple doesn't run the core FS in a userspace process but every other FS that's not as performance sensitive is now a userspace daemon, for instance. They developed their own FUSE equivalent to do this. In Windows a lot moved out in Vista. Graphics, audio, printing, a lot of drivers are out of kernel now.

Linux has lagged behind quite badly in this respect partly because a microkernel design requires close cooperation between userspace and kernel space but the Linux design philosophy is that the kernel is a self-contained artifact.

All the arrogant asocial coder bros cast aside.

All the poorly reasoned shortcuts due to hustle culture and "git pull the world" engineering, startups aura farm on Twitter/social media about their cool sweatshop labor exploiting tech jobs...

Watching AI come around and the 2010s messes blow up in faces... chefs kiss

Hey it's all web-scale though! Good job!

What else do you expect, given the economic incentives on one side, and the immaturity of the discipline on the other? Writing robust software requires time, money and competence, in a purely empirical approach, since we have no fundamental theory of software. The pressure is for quantity and features in minimum time. The approaches are incompatible, and economics win every time.

Which is where the unserious emerges but in a subtle way; taking such unserious things so seriously is not serious behavior. It's anxious and paranoid, aloof and clueless behavior.

Secure in tech skills but unserious otherwise.

Lacking a broad set of skills will make office workers unable to grow a potato inherently paranoid about their job.

It was merely untenable due to hardware limits and now outdated software development patterns.

Big data SaaS companies were never the end goal. They were a stepping stone to AI. A lab to test AI theory.

So your runway and moat so to speak were never real. Merely temporary science research.

I don't think the wealth should go to billionaires. Nor do I think your life should be spent dancing like a monkey to their organ, while you convince yourself to soothe the soul its for a greater good.

Perhaps your country should engage in substance collective action. Because this whole time you were just a pawn of billionaires who don't know you exist. As such they never cared about providing you assurances. You were just cheaper labor.

Big companies have security roles on multiple levels, enforcing policies and not allowing devs to just install any package. That's not new but started maybe 15 years ago.

Yes, I mean crates like anyerror and syn.

Languages with rich standard libraries provide enough common components that it's feasible to build things using only a small handful of external dependencies. Each of those can be carefully chosen, monitored, and potentially even audited, by an individual or small team.

That doesn't make the resulting software exploit-proof, of course, but it seems to me much less risky than an ecosystem where most programs pull in hundreds of dependencies, all of which receive far less scrutiny than a language's standard library.

One idea I've been entertaining is to not allow transitive imports in packages. It would probably lead to far fewer and more capable packages, and a bigger standard library. Much harder to imagine a left-pad incident in such an ecosystem.

They're not either, every one of these projects contains a gigantic vendor/ folder full of unmaintained libraries, modified so much that keeping up with the latest changes is impossible so they're stuck with whatever version they copied back in 2009.

there's nothing stopping you from using python from 2009 except why would you want to do that to yourself - but the same strategy applies. the reference python implementation is written in C, after all.

The problem is that the UNIX shell model got very successful and is now also used on other platforms with poor package management, so all the language-level packaging system were created instead. But those did not learn from the lessons of Linux distributions. Cargo is particularly bad.

For example, I'm not sure if the world of windows freeware ever moved past this, but very often, the home page for a freeware package will look nearly identical to a page set up to deliver malware. Every package you download you wonder "is this the legit version?". Even push it further, there were multiple examples of sites that were previously trusted for software downloads(SourceForge and the installer debacle) that began packaging spyware or adware into downloads.

With either delivery method, you're not quite safe from supply chain attacks, but with the curated repo, you at least have a single source of packages where you can trust it 99% of the time.

I recall a decade ago listening to native app developers lamenting how web pages were inferior to native apps and gnashing their teeth at why browsers wouldn't learn the lessons of native apps. It was, and remains, a shocking display of self-unawareness to fail to understand why web pages, despite doing many things worse than native apps, managed to do blow native apps out of the water when it comes to doing the things that actually matter to users. This is how it feels listening to the above comment; you have failed to reflect on why both programming language authors and programming language users were pushed to using language-specific package managers in the first place, and you have failed to put forth any improvements to OS-level package managers that would allow them to address those underlying flaws.

Many Golang projects I see in the wild will import a number of dependencies with significant feature overlap with sections of the standard library, or even be intended as a replacement for them. So it seems that having an expansive stdlib isn’t sufficient to avoid deep dependency trees, it probably helps to some degree but it’s definitely not a panacea.

More or less the entire Debian apparatus is an organization devoted to being a C/C++ package manager, and while as an end-user it's adequate for installing applications it's still an enormous pain to use packages as libraries even with apt and friends. And once you get outside of apt, you're in an endless hellscape. People don't seem to understand that the real reason that people love Rust is not because of memory safety (let's be honest, most people are too short-sighted to care about that); it's because of Cargo.

I strongly doubt that. Especially with tools like pkg-config that let you generate the set of flags for a package. If anything I've seen more horrendous build scripts from people that are trying to be clever and trying to support everything under the sun.

Package managers aren't going anywhere. Even languages that historically bet on large standard libraries have been giving up on that over time (e.g. Java's stdlib comes with XML support but not JSON).

Unfortunately, LLMs are also not cheap enough to just create whole new PL ecosystems from scratch. So we have to focus on the lowest hanging fruits here. That means making sandboxing and containers far more available and easy for developers. Nobody should run "npm install" outside a sandbox.

We need a cultural shift toward code hygiene, which isn't really any different from the norms most cultures develop around food. It's a mix of crude heuristics but the sense of "eeew" is keeping billions of people alive.

Which is to say: Hiding the sausage-making is a core aspect of what makes supply chains profitable.

They dont wait for the cultures to come back negative to say yes either. They just eat what they are served.

If the restaurant has a foul smell and the food is served by a twitchy waiter who insists that the food totally free, I think most people will think twice.

That isn't a guarantee either, just last month someone compromised the Axios library.

Compare that with the average distro. You would have to compromise the developer infrastructure (repo or website) and publish a new version without them being aware while notifying the maintainer that’s its ok to merge the new package script in the distro repo. Hard to pull off in high profile projects.

Today I’m limiting the exposure to dependencies more than ever, and particularly for things that take few hundred lines to implement. It’s a paradigm shift, no less.

But being able to have agents implement pelr5 in rust and make it faster and more secure raises many questions towards the role of open source and consequences of security and supply chain risks.

Then I moved to another company where we had builds that access the Internet. We upgrade things as soon as they come out. And people think this is good practice because we're getting the latest bug fixes. CVEs are reviewed by a security team.

Then a startup with a mix of other practices. Some very good. But we also had a big CVE debt. e.g. we had secure boots on our servers and encrypted drives. We had a pretty good grasp on securing components talking to each other etc.

Everyone seems to think they are doing the right thing. It's impossible to convince the "frequent upgrader" that maybe that's a risk in terms of introducing new issues. We as an industry could really use a better set of practices. Example #1 for me is better in terms of dependency management. In general company #1 had well established security practices and we had really secure products.

For example, at one company I worked for, they created an ACL model for applications that essentially enforced rules like: “Application X in namespace A can communicate with me.” This ACL coordinated multiple technologies working together, including Kubernetes NetworkPolicies, Linkerd manifests with mTLS, and Entra ID application permissions. As a user, it was dead simple to use and abstracted away a lot of things i do not know that well.

The important part is not the specific implementation, but the mindset behind it.

An upgrade can both fix existing issues and introduce new ones. However, avoiding upgrades can create just as many problems — if not more — over time.

At the same time, I would argue that using software backed by a large community is even more important today, since bugs and vulnerabilities are more likely to receive attention, scrutiny, and timely fixes.

Sorry, if you are updating continuously, that there is at least one layer that you failed to build any security into.

And yes, they still thought they were doing the right thing.

Anyway, the point of parent and me wasn't that it was considered to be a "mistake", but people thinking they "are doing the right thing".

As for the parent comment about not using the lockfile for the production build, that’s just incredibly incompetent.

Maybe they should hire someone who knows what they are doing. Contrary to the popular beliefs of backend engineers online, you also need some competency to do frontend properly.

In this case what’s needed is „npm ci“ instead of „npm install“ or better „pnpm install —frozen-lockfile“.

Pnpm will also do that automatically if the CI environment variable is set.

When you are talking about checking your dependencies in the source tree, you are effectively pinning exact versions, and not using floating/tilde versioning syntax.

If you want a vendored deps model you can look at Yarn Plug and Play which does this via .zip files.

However, I would just stick with regular pnpm and installs.

Uh… no.

> setting up native binaries, or native modules linked against the specific Node version

So the majority of projects—those that don't use binary NodeJS modules—don't have a reason for sidestepping the primary VCS and going along with npm's shoddily designed overlay version control approach?

> However, I would just stick with regular pnpm and installs.

You're not answering the question. npm isn't bedrock, and pnpm certainly isn't. If you're going to introduce (mandate) the use of a tool in the workflow, you should be able to justify it by explaining your rationale for introducing it (and making everyone deal with the associated costs). You should at minimum be able to provide a lucid explanation of the tradeoffs. For good measure, you should be able to disprove the "NPM Null Hypothesis"; you should be able to state a straightforward answer to the question, "What problem is this supposed to be solving?"

this is on some ancient node 16 build i was trying to clean up ci for, so not very recent npm

In general, use of npm ci is usually sparsely documented - most node projects you can find just recommend using npm install during the setup, suggesting a failure in promoting it's availability (I only know of it because I got frustrated that the lockfile kept clogging up git commits whenever I added dependencies with what looked like auto-generated build-time junk).

1. Get instructions from somewhere, a web page, HOWTO, whatever, telling you to "pip install thing".

2. "× This environment is externally managed"

3. Spend awhile Googling how to get past this and just make the damn thing work while swearing.

This is from observing users in practice. Google "This environment is externally managed" and you'll get nothing but hits on (a) people asking what this means and how to make it go away and (b) various unsafe methods to make it so.

Turns out there is no equivalent to “npm ci” that doesn’t clear node_modules first, and you can’t call npm install to simulate NPM ci behavior (sans clean).

/s

I like to think people would agree more on the appropriate method if they saw the risk as large enough.

If you could convince everyone that a nuclear bomb would get dropped on their heads (or a comparably devastating event) if a vulnerability gets in, I highly doubt a company like #2 would still believe they're doing things optimally, for example.

No one in this thread proposed that, or anything that could be reasonably assumed to have meant that.

If you expose people to the true risks instead of allowing them to be ignorant, the conclusion that they might come to is that they shouldn’t develop software at all.

I would count myself as a "frequent upgrader" - I admin a bunch of Ubuntu machines and typically set them to auto-update each night. However, I am aware of the risks of introducing new issues, but that's offset by the risks of not upgrading when new bugs are found and patched. There's also the issue of organisations that fall far behind on versions of software which then creates an even bigger problem, though this is more common with Windows/proprietary software as you have less control over that. At least with Linux, you can generally find ways to install e.g. old versions of Java that may be required for specific tools.

There's no simple one-size-fits-all and it depends on the organisation's pool of skills as to whether it's better to proactively upgrade or to reluctantly upgrade at a slower pace. In my experience, the bugs introduced by new versions of software are easier to fix/workaround than the various issues of old software versions.

Well, you critisize people who run the latest software here. Two counter-arguments:

1) If you don't upgrade frequently, you end up with super stable debian stuck on ... ancient software. This in turn means that many more recent software, won't work, unless you recompile a lot. I had this issue with mesa for instance, then needing a more recent LLVM, spirv-components and so forth. No chance to have that easily on debian, unless you control what you compile. On my local system here I run gtk2, gtk3 and gtk4 just fine. Good luck having that with debian for recent version; even debian sid is slow compared to, say, gentoo or arch(linux) or void(linux) here.

2) Even debian systems would be vulnerable to copy.fail. So that strategy is also not automatically better.

Personally I am among the frequent update folks. I use ruby scripts to automatically update to the latest, in hope that the people who write code are not incompetent. There is no guarantee that newer software is automatically always better; it is a trade-off. I don't have the time and resource for infinite security audits. I need to get things done and this approach, different to the "everything is scary" crowd, works super-well for me. I use a versioned AppDir approach on linux though, so I don't run into many issues of "can not upgrade because of same .so name issue", so I can conveniently switch to other versions as-is, including the kernel. (Excluding ABI differences and glibc, but for about 98% of the programs this works very well. I am also not alone with the get-everything-working approach, see xserver or gtk2-ng: https://github.com/X11Libre/xserver https://git.devuan.org/Daemonratte/gtk2-ng - granted, for the linux kernel this does not work that well ... I think we need better strategies for the linux kernel, things such as copy.fail should not be possible. I have no good solution here, AI will find many more exploits. No clue how we can prevent this or mitigate this more easily. I was surprised when the local instructor showed us how easy it is to use python for gaining superuser access as-is.)

A long time ago I heard that Google reviews every single line of third party code they use, not sure if it's true or if they still do it.

Whether to do constant npm upgrades to keep the high-priority security issues count at zero (for what seems like about 15 minutes), or whether to hang back a bit to avoid catching the big one that everyone knows is coming real soon now.

Not enjoying npm at all.

Programming language packages issue only because we don't have zero trust for modules — no restrictions to open socket or file system. Issue is not count, pure function leftPad can't hurt you.

https://github.com/npm/cli/releases/tag/v11.10.0

This wasn't a nightmare wanting to happen, but an example of badly maintaned systems for the lowest common denominator.

Why is npm the only package ecosystem that has so many problems? What are the other package system owners doing better? Let’s start there, instead of blaming the victims.

Yes.

I just noticed that a Rust program I'm working on had acquired a plotter driver crate. A plotter driver? The program has no graphical output.

Turns out that "kdtree" has a dev dependency on a profiling library that pulls in a whole graphics system. Even in release mode, I get that, because I have debug symbols turned on, which activated dev dependencies.

Aargh.

Nope that doesn't happen. It's not compiled into your binary if it's a dev or build dependency. Cargo may have downloaded the crate source according to the lockfile and that's it, it shouldn't build anything unneeded.