Show HN: A Zero-Copy 1.58-bit LLM Engine hitting 117 Tokens/s on single CPU core
https://github.com/r3-engine/r3-engine
The Project: I am building R3-Engine, a from-scratch, local AI inference engine for Microsoft's bitnet-b1.58-2B-4T. It is written in 100% Safe Rust, natively cross-compiles to Wasm SIMD128, and uses Zero heap allocations in the execution loop.The Physics: By mapping a 64-byte aligned .r3 file directly from NVMe to CPU L3 Cache (Zero-Copy) and using AVX-512 VPOPCNTDQ for branchless math, the...