News for hackers. Stuff that matters.
Re:Show HN: We made our own inference engine for Apple Silicon by dcreater on Tuesday July 15, 2025 @12:05PM (#44572592)
Somewhat faster on small models. Requires new format.

Not sure what the goal is for this project? Not seeing how this presents adequate benefits to get adopted by the community

Re:Show HN: We made our own inference engine for Apple Silicon by koakuma-chan on Tuesday July 15, 2025 @12:39PM (#44573065)
Written in Rust is a big one for me.
Re:Show HN: We made our own inference engine for Apple Silicon by worldsavior on Tuesday July 15, 2025 @01:32PM (#44573715)
It's utilizing Apple ANE and probably other optimization tools provided by Apple's framework. Not sure if llama.cpp uses them, but if they're not then the benchmark on GitHub says it all.