News for hackers. Stuff that matters.
Re:ZML - High performance AI inference stack by gwenzek on Tuesday September 17, 2024 @11:48AM (#41568948)
Hi co-author here. Zig is way simpler than C++. Simple like in an afternoon I was able to onboard in the language and rewrote the core meat of a C++ algorithm and see speed gains (fastBPE for reference).

Coming from Python, the hardest part is learning memory management. What helps with ZML is that the model code is mostly meta programming, so we can be a bit flexible there.

We have a high level API, that should feel familiar to Pytorch user (as myself), but improves in a few ways

There are no comments on this comment yet.