Re:Self-Hosting Open Source LLMs: Cross Devices and Local De...
by 3Sophons on Tuesday November 21, 2023 @02:54AM
(#38360680)
Faster is compared to Python. Portable, more secure and lightweight are compared with Python and other native solutions. In terms of benchmarks, Rust / C++ is 50,000x faster than Python; WasmEdge runtime + portable app is 30M compared with 4G Python and 300MB llama.cpp Docker image that is NOT portable across CPU or GPU; Wasm sandbox is more secure than native binary.
[ Parent
| on HN ]