News for hackers. Stuff that matters.

A comment on Rust is a lot better than Python to run ML in production.

Re:Rust is a lot better than Python to run ML in production by mectors on Tuesday December 17, 2024 @12:32PM (#42443291)
Python Docker images with CUDA, Python, Pytorch,... are 5GB to 10GB of third-party code. Here is an example of bringing this down to 1.13GB thanks to WasmEdge, 10MB of LlamaEdge API Server [compatible with ChatGPT] and 1.17GB for TinyLlama.
[ Parent | on HN ]
Re:Rust is a lot better than Python to run ML in production by verdverm on Tuesday December 17, 2024 @01:50PM (#42444081)
If you are not actually running ML code, of course you can remove Python and ML libraries. You could get this down to MB by removing WASM and using a Go binary

apples and oranges

[ Parent | on HN ]