Rust is a lot better than Python to run ML in production

Posted by mectors on Tuesday December 17, 2024 @12:32PM
from the mectors.medium.com dept.

https://mectors.medium.com/ai-explained-mlops-de1523a0afc0

( 2 comments | 3 points | on HN )

Re:Rust is a lot better than Python to run ML in production by mectors on Tuesday December 17, 2024 @12:32PM (#42443291)

Python Docker images with CUDA, Python, Pytorch,... are 5GB to 10GB of third-party code. Here is an example of bringing this down to 1.13GB thanks to WasmEdge, 10MB of LlamaEdge API Server [compatible with ChatGPT] and 1.17GB for TinyLlama.

[ Parent | on HN ]

Re:Rust is a lot better than Python to run ML in production by verdverm on Tuesday December 17, 2024 @01:50PM (#42444081)

If you are not actually running ML code, of course you can remove Python and ML libraries. You could get this down to MB by removing WASM and using a Go binary

apples and oranges

[ Parent | on HN ]