Somehow I always assumed RISC-V already had MSI for its PCI-Express use cases, which is essential to me.

Yes, the RISC-V AIA has been a standard for some months now.

That is shockingly young, like the GP I had assumed MSI would've been designed in when the PLIC was designed (i.e. ≤ 2019). The PLIC doc says something about "Interrupt Gateways", did those not make it?

Out of curiosity, if anyone can answer: There are many RISC-V CPU cores. Are there implementations of full systems, including PCI-E host, on an FPGA platform?

The article seems to claim that AIA is still a work-in-progress thing?

AIA is still a work in progress. The last draft release was 18 days ago, and it is at version 0.3.0-31. Verify here: https://github.com/riscv/riscv-aia/releases

As the article mentions, there was a major change between 30 and 31 where the SETEIENUM/SETEIPNUM as well as the CLREIENUM/CLREIPNUM CSRs were removed. The issue on GitHub reads that they were removed since the architecture review board asked for them to be removed.

A manufacturer can use a draft spec, but it may be rendered outdated sooner rather than later.

tangential question: if i wanted to poke around with the risc-v isa (like add an instruction) and then wanted to add support to an emulator, compiler and operating system for that instruction, is there an emulator, compiler and operating system that are best suited for this sort of isa experimentation?

The CHERI team had the same problem: https://github.com/CTSRD-CHERI/cheribuild. You'll need something along the lines of "./cheribuild.py sdk-riscv64 freebsd-riscv64 disk-image-riscv64 run-riscv64", bit of disk space, and a free couple of hours for it to finish building everything; at the end you'll get a login prompt for the guest.

QEMU TCG is not that difficult to understand, and has quite comprehensive support for RV64 already, so you could add a new instruction there. As for compiler/kernel support, that's probably a lot more difficult, but it's hard to say what would be involved (eg. deep changes, or simple __asm__ statements) without knowing what kind of feature you'd be trying to add.

what if i said risc-v isn't as important (to start), but rather just a dead simple isa with a dead simple emulator, compiler and operating system (like something that can take cross-compiled binaries and run them with basic input and output).

like i'd like to experiment with novel ideas in the area, but ideally would like to avoid getting into the complexity of real compilers, operating systems and emulators until the ideas are looking proven...

i can go looking more for stuff myself (maybe toy mips stuff used in schools or something) but this seems like something that people would do and therefore there'd be known tooling?

I built a RISC-V emulator in python focused on extensibility and debugability for this exact purpose. It should be dead simple to use with plain RISC-V Assembly files. It has a userspace-only mode and a minimal privileged mode: https://github.com/AntonLydike/riscemu

Adding new instructions is as simple as adding a method to a class, as long as you are reading straight assembly files. If you want to parse ELF-files, you also have to add the opcodes for your command to the decoder (which is currently not very well documented). If you'd like to do that, feel free to open an issue and I'll work you through it.

I also built a minimal kernel, which works on the emulator, but it doesn't support IO and is very rough around the edges in general.

If you're asking "do simple teaching OSes/toolchains exist?", of course they do (Minix is the most famous one). If you want risc-v specifically, with a quick search I found xv6, which seems to have been put together for https://pdos.csail.mit.edu/6.S081/2021/ .

Another alternative is to look at something like PicoRV32. There's no "OS" as such that can run on PicoRV32, there is only a few kb of RAM, so single C files that you write from scratch. But it's a simple project and fun for hacking. You can put it on a cheap Lattice FPGA using a completely open source toolchain. A fun way to get into hardware / FPGAs for sure.

If you're into the open ISA idea but find the big guys a bit intimidating, you might have fun with OpenRISC. At least lately I've had a blast hacking on it. The kernel and QEMU implementations are very simple, and Stafford Horne is fun to talk with.

If you just want to test the idea without any major complexity, like "I just want to add an instruction for int8 dot products" or whatever, you can write your own emulator for RV32IMC, in M-mode, in a few hundred lines of most modern programming languages. There are a bunch of test suites to check the base ISA works, and you can use a C compiler to target bare metal, and use GCC's .insn directive to embed custom opcodes inline. Run a bunch of tests this way. It's actually very easy to control the whole stack in cases like this.

Once you get into actually needing to boot a system like Linux, you're going to need more support from the emulator, and at that level you'd either want to look into extending the Spike emulator, or QEMU itself. Like, features that interact with different privilege levels or whatever are just complex, there isn't any way around just working on the source directly. I realize that's not a very good answer.

There are simpler kernels and whatnot you could also probably use. I mean, I would hope so. Honestly, a lot of it depends on the specifics of what you're looking at trying to implement. Simple instructions that work in M/U mode are the best place to start.

I suggest trying to write your own emulator first. It'd surprise you at how easy the base RISC-V IMC ISA is, and you can add as many custom instructions as you like, and you'll have a C compiler (and Rust too, actually) to start with.

Emulator is dead easy other than bit twiddling initially.

The compiler aspect would depend because although compilers try to construct the code generator automatically (e.g. by matching on a tree) sometimes it requires a special case to do good code.

What are these MSI things useful for? (The article shows how to interact with it, but not what you can do with it)

I know nothing about hardware

Interrupts are a way for peripherals to signal to the CPU something happened (packet is received, audio buffer has been played and I need more data, etc. the application is device dependent). Once a cpu receives an interrupt it jumps to a predetermined interrupt service routine the OS sets up to address the situation the peripheral signaled (usually ultimately in a driver). Traditionally those are carried by individual discreet connections to the CPU ie. a set of wires signaling an interrupt level. MSI is a way to virtualize this and have interrupts in band with the rest of the bus (Ie. Within the address space of the bus) instead of needing specific wires for each one.

ordinarily, you would have to dedicate a physical pin on your device to listen for an interrupt signal from another device. MSI allows you do use existing data busses, like PCI, to send and listen for events or matched patterns on that bus, removing the need for a physical pin to signal an interrupt (in some cases)

Note that using the existing data connection (PCI-e is point to point, not a bus anymore) helps solve some race conditions. DMA writes from a device to memory may be buffered by bridges, but interrupt pins are likely not; so if the device wrote to memory and signals that to the host via an interrupt pin, the interrupt may arrive before the write. A message signaled interrupt uses the same mechanism as the DMA write, if the DMA write is buffered, the interrupt write will wait behind it.

Presumably the addresses for this is configurable right? Some systems put DRAM at 0. If a system has a fixed address it's probably best to describe in the spec how to discover it in the device tree

What is the max latency of these signals?

Depends on the system architecture. But a lot of complex systems have been message signaled internally for a very long time, hypertransport based systems being a prominent early example in the desktop/server space.

Indeed there are no interrupt pins in PCI express at all, you only have a choice between special "Legacy/Compatibility" Assert_INTx message packets, or plain Memory Writes for MSI/MSI-X. But both are carried as in-band PCIe packets on the high speed lanes.

RISC_V is getting better and better. Now we need a raspberry-pi like thing with all these goodness.