ZML - High performance AI inference stack

Re:ZML - High performance AI inference stack by onurcel on Tuesday September 17, 2024 @11:40AM (#41568837)

First of all, great job! I think the inference will become more and more important.

That being said, I have a question regarding the ease of use. How difficult it is for someone with python/c++ background to get used to zig and (re)write a model to use with zml?

[ Parent | on HN ]

Re:ZML - High performance AI inference stack by gwenzek on Tuesday September 17, 2024 @11:48AM (#41568948)

Hi co-author here. Zig is way simpler than C++. Simple like in an afternoon I was able to onboard in the language and rewrote the core meat of a C++ algorithm and see speed gains (fastBPE for reference).

Coming from Python, the hardest part is learning memory management. What helps with ZML is that the model code is mostly meta programming, so we can be a bit flexible there.

We have a high level API, that should feel familiar to Pytorch user (as myself), but improves in a few ways

[ Parent | on HN ]

Re:ZML - High performance AI inference stack by steeve on Tuesday September 17, 2024 @11:43AM (#41568884)

pretty easy, usually the hardest part is figuring out what the python code is doing

[ Parent | on HN ]