Show HN: Jacq, a DSL that compiles model and kernel descriptions to CUDA
https://github.com/ImpossibleComputing/jacq
I was working (with agents) on testing out various attention algorithm ideas I had, and I found that the existing infrastructure/libraries made it more difficult than it should be.So I created Jacq, a DSL and compiler. A jacq file describes a model’s configuration, weights, layers, and pipeline. Jacq turns it into a standalone CUDA inference runtime and a Python weight exporter. Kernel descriptions can also generate CUDA harnesses.
The easiest way to try it is the TinyLlama example in the README. Building, validating, visualizing, and generating CUDA only require Rust. Of course, running inference does require an NVIDIA GPU, CUDA, Python dependencies, and weights. Jacq doesn't support other GPU vendors (AMD, etc) or Metal, yet (PR's welcome!)
There's also an experimental vLLM patch to compile and patch your jacq file into a vLLM build, as well.
Feedback very welcome!
Jacq also handles some of the repetitive setup stuff: tensor allocation, seeded inputs, launch wiring, timing, and all the output dumps for generating comparisons and what not.
Triton is a much broader framework that can do a lot more, frankly. Jacq is pretty narrow at this time. But, by being narrow, it makes the experiment cycle really fast.