Show HN: Glowstick – type level tensor shapes in stable rust
https://github.com/nicksenger/glowstick
Hi HN,In the past few years I've become more interested in machine learning. Since I'm sure the same is true for many here, I wanted to share this project I've been working on: glowstick uses type-directed metaprogramming to keep track of tensor shapes in Rust's type system and determine which operations are permitted or not at compile time.
I find Rust has a lot of strengths when it comes to ML applications, but waiting until runtime to find shape related issues feels a bit strange since normally I don't run the code all that often while developing. Given Rust has fancy types available, I figured I'd try my hand at using them to address this.
I've added integration crates for the two ML frameworks I use most frequently, candle and burn, and included examples of implementing llama 3.2 in each using typed shapes for much of the model internals and inference loop. Mixtures of static and dynamic dimensions should be supported well enough for most applications at this point, though there are of course still improvements to be made.
Any feedback is appreciated!
Anything similar in here?
If not—actually, optimizing compilers are pretty okay nowadays anyway. I wonder if you’ve tried just seeing what Rust will do automatically with different optimization levels?
That being said, it seems reasonable that you could make some optimizations like this if you had deeper integration of these types with a framework or similar. It's not something I've explored personally, sounds interesting though.
https://web.archive.org/web/20111015133833/http://www-staff....
This was used in his shape aware language FiSh, for dealing with multidimensional arrays. Shape compatibilities were statically type checked, if I recall correctly. Shapes were also used to optimize the loops.
[Programming in FISh] https://link.springer.com/article/10.1007/s100090050037
[Towards Dynamic Shaping] https://www.researchgate.net/publication/265975794_Towards_D...
The glowstick shape is also a type-level list of integers, and I could definitely see how other shapes might be useful in different situations.
Doing this sort of thing in Rust is a bit of a stretch for sure, but it makes the outcome easy to apply which is nice.