Re:Lm.rs: Minimal CPU LLM inference in Rust with no dependency
by J_Shelby_J on Friday October 11, 2024 @04:24PM
(#41813366)
Yeah, hard to not be overly verbose. “No massive dependencies with long build times and deep abstractions!” Is not as catchy.
[ Parent
| on HN ]
Re:Lm.rs: Minimal CPU LLM inference in Rust with no dependency
by 0x457 on Friday October 11, 2024 @04:53PM
(#41813626)
No dependencies in this case (and pretty much any rust project) means: to build you need rustc+cargo and to use you just need resulting binary.
As in you don't need to have C compiler, python, dynamic libraries. "pure rust" would be a better way to describe it.
Re:Lm.rs: Minimal CPU LLM inference in Rust with no dependency
by littlestymaar on Friday October 11, 2024 @05:34PM
(#41813984)
It's a little bit more than pure Rust: to build the library there's basically only two dependencies (rayon and wide) which bring only 14 transitive dependencies (anyone who's built even simple Rust program knows that this is a very small number).
And there's more, Rayon and wide are only needed for performance and we could trivially put them behind a feature flag and have zero dependency and have the library work in a no-std context actually, but it would be so slow it would have no use at all so I don't really think that makes sense to do except in order to win an argument…