superlopuh

I'm curious and not an expert here, do you know why the TTFT is so much worse on Mac? To elaborate, the article just says that this step is compute bound, but I'm wondering whether it is just that simple or if it might also be less optimised in MLX?

I raised this in person to a number of array language implementors (and Connor Hoekstra) last year and they weren't familiar with interaction nets. I'm not sure that I was successful in convincing them that this was worth looking into, partially because I'm not yet personally convinced that this is worth looking into.

In my experience a culture where teammates prioritise review times (both by checking on updates in GH a few times a day, and by splitting changes agressively into smaller patches) is reflected in much faster overall progress time. It's definitely a culture thing, there's nothing technically or organisationally difficult about implementing it, it just requires people working together considering team velocity more important than personal velocity.

Missing Muna[0][1], I'm curious how it would compare on these benchmarks.

[0]: https://www.muna.ai/ [1]: https://docs.muna.ai/predictors/create

Also seems like the repo is now private, so I can't open an issue, or reproduce the numbers.

I'm surprised that the `isinstance()` comparison is with `type() == type` and not `type() is type`, which I would expect to be faster, since the `==` implementation tends to have an `isinstance` call anyway.

We've been relying on TypeForm (an experimental feature in Pyright) in xDSL. Since there are some Astral members commenting here: are there any plans to support TypeForm any time soon? It seems like you already have some features that go beyond the Python type spec, so I feel like there may be hope

In MLIR, there are two representations of memory, `tensor` and `memref`, which enables you to do some high-level things[0] in SSA before "bufferizing" to memrefs, which are eventually lowered to LLVM pointers.

[0]: https://mlir.llvm.org/docs/Dialects/TensorOps/

Yep, although never in a project of a similar size. One advantage of the Python setup is that the types are ignored at runtime, so there's no overhead at startup/compilation time. Although it's also a disadvantage in terms of what you can do in the system, of course.

Python with Pyright in strict mode. I work on a ~200kLOC fully typed Python project [0] and am having fun.

[0]: https://github.com/xdslproject/xdsl

Can someone familiar with performance of LLMs please tell me how important this is to the overall perf? I'm interested in looking into optimizing tokenizers, and have not yet run the measurements. I would have assumed that the cost is generally dominated by matmuls but am encouraged by the reception of this post in the comments.

I really want to switch to Zed from Cursor but the battery usage for my Python project with Pyright is unjustifiable. There are issues for this on GitHub and I'm just sad that the team isn't prioritising this more.

Beautiful

I love that language and frequently show it to people. I'm sad to see that my local install doesn't work any more. I actually used it to solve a puzzle in Evoland 2 that I'm relatively sure was added as a joke, and is not solvable in a reasonable time without a solver. I'm actually doing a PhD in compilers right now, and would love to chat about sentient if you have the time. My email is [email protected].

I just get a bunch of stars.

I once used Sentient[0] to solve a similar puzzle in a different game that I think made a puzzle that required a constraint solver as a joke on the player. I'm a bit saddened to see that the repo hasn't been updated in 6 years, and a node update broke the binary installed on my computer, I find it a much more ergonomic environment than Prolog, hopefully someone else will pick up the mantle.

[0]: https://sentient-lang.org/

It's a good thing I only ever write legacy code.

My experience with Zig is that it's also a plausible replacement for C++

I know of a few projects looking in that direction, each optimising for different things, and none getting near the capability of LLVM, which is going to take some time. I spoke with some of the core MLIR developers about this, and they're generally open to the notion, but it's going to take a lot of volunteer effort to get there, and it's not clear who the sherpa will be, especially given the major sponsors of the LLVM project aren't in a particular hurry. If you're interested in this feel free to look our paper up in a week or two, we've had a bit of trouble uploading it to arxiv but should be ready soon.

https://2025.cgo.org/details/cgo-2025-papers/39/A-Multi-Leve...

Here's a quick pres from the last dev meeting on how this can be leveraged to compile NNs to a RISC-V-based accelerator core: https://www.youtube.com/watch?v=RSTjn_wA16A&t=1s

The typing version is still useful when you want to communicate that the result conforms to a certain interface, which doesn't include mutability in the case of Set, but not the exact type.

Edit: I see that he imported the typing Set, which is deprecated, instead of collections.abc.Set, which is still useful, so your comment is correct.

As far as I'm concerned, this is the fix: https://github.com/astral-sh/uv

We use Textual as an interactive way to explore compilation pipelines in xDSL [0]. As our compiler is written in Python, it was the perfect tool to build a UI in the same language as the existing codebase. After starting the `xdsl-gui` project, we found Marimo [1], a reactive notebook for Python, which also lets users build apps in Python. It's been interesting to compare these two, especially in the way they handle state and propagate updates. For now we're using both tools, but it might make sense to centralise at some point. Both of these frameworks like immutable data structures, which I find is a positive incentive to use immutability throughout the code, and has been good for the rest of the project.

[0]: https://xdsl.dev/ [1]: https://marimo.io/

Here's a keynote talk by Andrea Liu, the lead of the project, it's a much better resource about one of the most exciting things going on in ML right now:

https://youtu.be/7hz4cs-hGew?si=64O3Q7g-qeRQ0Td4

If they wanted to, why wouldn't they make a modified, specialized, server rack version of their hardware?

Definitely doesn't look as clean as the example above, or as clean as the Swift equivalent of

    await x + y

The same could be said of a lot of things. For example in hash maps, you might have a cliff in performance if your hash function is not good for your data distribution, but you'll still happily use the default ones until you're really sure that they're not the right tool for the job. I also feel like this depends a lot on your language philosophy, some languages generally accept the cliffs, some try to expose enough of an API for you to work around the cliff if you feel like you know what you're doing, like custom allocators etc.

I have some personal hunches about how to have better guarantees about these properties but I feel like it's ok for this to not be solved with the v1.

Datalog is a strong contender: https://www.learndatalogtoday.org/chapter/1

This is basically my PhD thesis proposal, I don't think there's any fundamental technological problem here, just that for a graph to be efficient to process you need high-level optimisations that can take mathematical properties of graphs into account. For that you need to either reimplement a compiler into your framework, or be integrated into an existing compiler, both obviously take a lot of work.

Some comments here mention GraphBLAS, which is the big breakthrough in decoupling the layout of the graph from an efficient implementation of an algorithm, but none mention MLIR-GraphBLAS [0] which is the most promising integration into a compiler that I've seen.

I still think it's early days, I wouldn't throw in the towel quite yet :)

[0]: https://mlir-graphblas.readthedocs.io/en/latest/index.html

I work on a reimplementation of MLIR in Python called xDSL, and ported a part of MLIR's Toy tutorial, you can try it interactively online at xdsl.dev/notebooks, or by cloning the github repo and running them locally github.com/xdslproject/xdsl