The original technical debt, which cannot be paid off, was the decision to use a Von Neumann architectures instead of a Harvard architectures for CPUs.

Explain? (Genuine question because I wanna know)

Basically, have an explicit distinction between data and code at the architectural level, to enforce "Write xor Execute" in hardware. In this way, you would prevent e.g. a buffer overflow letting an attacker injecting shellcode that gets executed later by somehow pointing at it.

and we don't believe a sterilized virtual machine creates a similar situation?

any respective machine doesn't require full Turing completeness?

any ad-hoc machine allows for mutations and flexibility, which I'm sure is an added strength.

don't we already have the NX bit?

and it's not sufficient anyways. even with NX you could have the attacker control a function pointer you call. in any sufficiently complex program you'll have enough widgets to build anything you want just out of data.

code and data are the same thing. whatever inputs your system receives is "code" that your application interprets, just like your application is the code that the CPU interprets. turtles all the way down.

In a Harvard architecture, it's not possible to point the CPU at data.

What about return oriented programming?

So JIT is not possible on a Harvard architecture?

But that's just irrelevant confusion of terms.

There is no fundamental distinction between code and data. This separation does not exist as category in actual reality (and I vividly remember that being one of the bigger points GEB made, early on). They're fundamentally one and the same, and classifying some bits as data and some as code is a point of view that really depends on what you're doing and why.

As for Harvard architecture - every time you branch on your "data", you're effectively executing it, and your "code" can be seen as emulating a system where your "data" is the actual code.

How do you turn text into code in this architecture? Through the kernel?

you don't. I also believe that JITs would be impossible under strict Harvard.

Wait, how do you create programs then? How do you compile?

That doesn't even make sense, to be honest. Interpreted languages like Python aren't running machine code. They are taking data and executing machine code based on the data. Is Python safe?

A compiler writes machine code do a file. The OS then loads the code from the file into executable memory before it runs the program. The compiler isn't actually trying to execute the machine code it produces. This is why a JIT wouldn't work - the JIT bypasses the interpreter and produces machine code _that executes directly on the CPU_ inside of the same app.

I don't know any details about Python's architecture, but it's totally possible to write an interpreter for machine code; that's basically what an emulator is. The trade off is that pure interpreters tend to be very slow.

[delayed]

Yes, and that's my doubt. Compiling is turning data (text) into code (something executed by the CPU). I thought the point of the hardware is that data can't become code. If data can become code, then data is code, and it doesn't matter if the data is compiled first or interpreted at runtime. There doesn't seem to be a way to prevent "data" from becoming "code".

My understanding is that all hard-Harvards are special-purpose ROM-driven systems with static payloads developed on von Neumann computers.

This sort of thing is much more common in microcontrollers, where the code is cross-compiled on another computer and written to flash memory through a debug interface.

In practice, it's usually possible to write to the embedded flash memory via software, but it's much more difficult and disruptive than a simple CPU memory access.

According to Wikipedia, Modified Harvard Architecture is a common thing and appears to have properties of both Harvard and Von Neumann architectures. It's probably worth discussing a specific device, OS, and programming language rather than having a vague discussion where both parties are constantly describing different goalposts.

It's not that hard to imagine that in a specific embedded device the kernel / compiler have access to write to the instruction memory space while user space programs don't have any API / permissions to do that.

I see. I guess this does solve the issue of a program managing to insert arbitrary machine code on an address and point the CPU at it, or replacing the memory where the machine code already exists. But I think there already exists OS-level protections against writable executable memory?

It also doesn't solve one important attack vector: the existing executable has logic that can be used maliciously (e.g. a function that sends a file to the internet), and this logic is executed based on data on memory. In this case you only need to manipulate the data, without manipulating the "code", in order to exploit it.

I'm not sure I understand why would you need special hardware for that.

I think that the principle of "code is data" is that data influences the algorithms either way, so there is not much of a point in separating the two.

> I think that the principle of "code is data" is that data influences the algorithms either way, so there is not much of a point in separating the two.

This is clearly wrong in the sense that you allow your users to submit data to your website via a form, but you shouldn't trust that data in an `eval(...)` statement.

There is value in logically separating data / instructions. Having a physical separation may reduce some categories of bugs / security problems. Obviously as you rightly point out that the memory full of instructions has to get populated sometime, but when, how often, by whom, and with what safeguards are all relevant questions to the overall security of the model.

I think we generally agree, but you are taking the "code is just data" entirely too far to the point that you are blinding yourself to the value of the gradations in the middle.

But.. alter data that gets executed (interpeter?), I mean, how does this work even in theory? All data is potentially code?

All data is potentially code only in a Von Neumann architecture, but not in a Harvard architecture, where there's a separation of data and code with different memory access to each.

[delayed]

> All data is potentially code only in a Von Neumann architecture, but not in a Harvard architecture, where there's a separation of data and code with different memory access to each.

Says you. I wrote a program for a Hardvard architecture chip (PIC) that read bytecode from the data memory and executed it. Updates were made by replacing the data block with a new data block.

In practice, there is no way to separate data and instructions. After all, JS downloaded inside a webpage is literally data, and it gets executed.

To clarify what intrasight is talking about: A Harvard architecture CPU has two physically separate memory busses for code and data. This makes it possible to build a computer with a program memory that cannot be altered through the data bus. In practice, this isn't normally done -- even on Harvard architecture microcontrollers, the flash and RAM are connected to both busses because memory is expensive. And having inalterable program memory doesn't save you from stack-smashing attacks that alter a return address.

That won’t help with Lisp programs. ;)

Good point, but not much infrastructure runs on Lisp

PHP would still be a C interpreter even if you disabled JITs.

Really, we never should have left the Garden of Eden. Things were great there, other than the maternal mortality rate.

Given that there were no mothers in the Garden (other than arguably Adam who gave birth from his rib), the maternal mortality rate was undefined behavior (0/0). Which checks out -- what did they do during all their time in the Garden, anyway? I guess now we know that we cannot know, and that Lilith must have been of the nasal variety.

This argument is much worse than "we should have made functional programming the standard back in 196X" (which might some merit but overdone imo). The Harvard architecture is just a hardware separation of instructions and data - the argument is virtually an urban legend. Hardwave code/data separations are relatively easy, have existed at time but change nothing since dynamic language just makes its code into data and the self-modification is unhindered.

see: https://www.computer.org/csdl/magazine/an/2022/03/09779481/1...

Separating the call stack (and probably inventing a very specific "the only valid reads/writes in this area are the ones made by the CALL/RET instructions" permission bit for it) from the data stack would make almost all buffer overflows useless for taking over the control flow: the normal STORE instruction simply can't overwrite the return address, period.

Of course, the interpreters would still be vulnerable, but even for them having separate (and distant) stacks make this particular vulnerability much harder to exploit. And again, a generalized technique having different processor instructions for manipulating differently "painted" memory regions would help there too — at least, it would be there so the implementers of the interpreters could reach for it if they decided to.

In the end, of course, nothing can save you from having a program that insists on literally doing eval(fetch(url=input())), or curl "$(read ; printf "%s" "$REPLY")" | "$SHELL", or something else to this effect.

Modern CPUs already split code and data between two types of L1 caches. So they are technically both "Von Neumann" and "Harvard" simultaneously.

It's also missing the point because modern buffer overflow exploits no longer write code into the stack, they overwrite the return address on the stack and assemble jumps to the end of functions known as "return oriented programming", but the underlying issue still applies to function pointers on the heap.

Max AI-glazing perspective in the title, as expected. We need to fix bugs that no mere humans could successfully find and exploit. Thankfully we have AI that can fix those bugs that wouldn’t be a problem without superhuman code agents.

But there are potential upsides like the already mentioned possible nation state exploits (sleeper exploits).

They would absolutely be a problem at some point, the timeline is just uncertain. That means defenders have to fix them all just in case.

That’s a legitimate and sensible opinion to have. But FYI I wasn’t inventing my hyperbolic retelling about bots-protecting-from-bots. See paragraph “Jana says actually exploiting the bug requires carefully working out a way to jump to a specific point in the memory for a particular program running on the server”.

the equilibrium state would be having found a good pattern for making patterns good. we probably aren't too far from that, so i think your worry may show up in the short term but not the long one

You're thinking of provable code? Or something else?

it appears they are thinking of a universal 'framework' for all things we may consider insecure as of present

that would do it too, and i suppose people are working on that. but even without that, there arent infinite low hanging fruit problems to find, and any given code can just sort of be play-doh'd into other code. so the llms will need a few idk years worth of rounds of being trained on the things that pop up that wed never have the time to harden, and things will settle. like, its hard patching stuff up quick right now but society hasnt fallen in to chaos.

[delayed]

> It will make adding features easier once the debt is paid off.

I'm not so sure: DJB-style secure microkernel/microservice architectures are hard to extend, and I have little hope that today's monolithic code bases are securable without retreating to a small, proven, and well-tested TCB--even by ASI.

> Is it forcing us to "pay off" technical debt or is it starting us on a compulsory endless treadmill of feature-neutral fix-work?

My problem with even talking about this is that there's absolutely no evidence for it. The idea that coming up with a technology to find holes in the hulls of boats starts with a flurry of holes being found and a bunch of patching to be done does not even hint at the theory that there are an infinite number of holes in all boats, and now that we can find them we will have to spend the rest of our lives patching them.

Code was bad, and we knew it was bad. We would have found most of these bugs if instead of coming up with LLMs, a bunch of countries got together and decided that they were going to put $10B into hiring programmers to do nothing but review all the code in the world for bugs. With AI, we managed to do it for only a trillion dollars.