I know of a few projects looking in that direction, each optimising for different things, and none getting near the capability of LLVM, which is going to take some time. I spoke with some of the core MLIR developers about this, and they're generally open to the notion, but it's going to take a lot of volunteer effort to get there, and it's not clear who the sherpa will be, especially given the major sponsors of the LLVM project aren't in a particular hurry. If you're interested in this feel free to look our paper up in a week or two, we've had a bit of trouble uploading it to arxiv but should be ready soon.

https://2025.cgo.org/details/cgo-2025-papers/39/A-Multi-Leve...

Here's a quick pres from the last dev meeting on how this can be leveraged to compile NNs to a RISC-V-based accelerator core: https://www.youtube.com/watch?v=RSTjn_wA16A&t=1s

MLIR is less a specific IR and more a generic framework for expressing your own custom IR (which is itself composed of many pluggable IRs)--except these IRs are renamed "dialects." One of the MLIR dialects is the LLVM dialect, which can be lowered to LLVM IR.

In the future, it's plausible that dialects for hardware ISAs would be added to MLIR, and thus it would be plausible to completely bypass the LLVM IR layer for optimizations. But the final codegen layer of LLVM IR is not a simple thing (I mean, LLVM itself has three different versions of it), and the fact that GlobalISel hasn't taken over SelectionDAG after even a decade of effort should be a sign of how difficult it is to actually replicate that layer in another system to the point of replacing existing work.

Earlier compilers were a pipeline of specialized IRs. I used to think that MLIR was an acknowledgment that this specialization was necessary. Ok, it is necessary. But MLIR's real contribution is, as you say, a generic framework for expressing your own custom IR.

All the important bits of MLIR are closed source and there’s no indication that’ll change anytime soon.

The big players have their own frontend, dialects, and mostly use LLVM backends. There’s very little common usable infrastructure that is upstreamed. Some of the upstreamed bits are missing large pieces.

What important bits are those? I can't imagine what you have in mind here; my current job and my previous job have both revolved around MLIR-based compilers, and it has never seemed to me that there is anything missing. I wonder if you might be expecting MLIR to do a job it's not really meant for.

> There’s very little common usable infrastructure that is upstreamed

Hmm I wonder what all that stuff is then that's under mlir/lib?

Like what are you even talking about? First of all there are literally three upstream frontends (flang, ClangIR, and torch-mlir). Most people use PyTorch as a frontend (some people use Jax or Triton). Secondly, downstream users having their own dialects... is basically the whole point of MLIR. Core dialects like linalg, tensor, memref, arith are absolutely generically useful. In addition many (not all) MLIR-based production quality compilers are fully open source (IREE, Triton) even if wholly developed at a for-profit.

I'd be interested to learn about such closed-source important bits and invite them to MLIR workshop / open developer meeting. Having worked on the project essentially since its inception, I am quite positive that the bits the original MLIR team considered important are completely open source.

Certainly, there are closed-source downstream dialects, that was one of the actual design goals of the project, but they are rarely as useful as one might think. I'd expect every big company with a hardware to have an ISA/intrinsic-level dialect, at least as a prototype, that they won't open source for the same reason they won't open source the ISA.

What I find sad is the lack is that end-to-end flows from, e.g., PyTorch to binaries are usually living outside of the LLVM project, and often in each company's downstream. There is some slow motion to fix that.

They solve different problems. MLIR is not a backend, but a toolkit for defining intermediate representations ("dialects") and the passes which optimize them or lower from one to another. You can use MLIR to organize everything your compiler does between its front-end and its back-end; MLIR doesn't care where the IR comes from or what you ultimately do with it.

MLIR does include a couple dozen built-in dialects for common tasks - there's an "scf" dialect which defines loops and conditionals, for example, and a "func" dialect with function call and return ops - and it so happens that one of these built-in dialects offers a 1:1 representation of the operations and types found in LLVM IR.

If you choose to structure your compiler so that all of your other dialects are ultimately lowered into this MLIR-LLVM dialect, you can then pass your MLIR-LLVM output through a converter function to get actual LLVM-IR, which you can then provide to LLVM in exchange for machine code; but that is the extent of the interaction between the two projects.

> Here's my question: It seems inevitable that people will eventually port all LLVM's smarts directly into MLIR, and remove the need to shift between the two. Is that right?

Theoretically, yes -- taking years if not decades for sure. And set aside (middle-end) optimizations, I think people often forgot another big part of LLVM that is (much) more difficult to port: code generation. Again, it's not impossible to port the entire codegen pipeline, it just takes lots of time and you need to try really hard to justify the advantage of moving over to MLIR, which at least needs to show that codegen with MLIR brings X% of performance improvement.

MLIR maintainer here, or however close one can be given that we don't have a clear ownership structure. This has been discussed repeatedly in the community, and it is likely that many things will get eventually ported/reimplemented, but there is no strong push towards that. Lower level parts of the stack, such as register allocation / machine IR / instruction selection are where LLVM has seen a lot of investment are unlikely to move soon. At least not in a generic way.

There was a keynote at the LLVM developer meeting a couple of years ago presenting the differences and the likely evolution from somebody not involved in MLIR that does the lay of the land.