Re:Should you use Rust in LLM based tools for performance?
by pjmlp on Tuesday October 01, 2024 @12:02PM
(#41710200)
Yet another reason to use native compiled languages with bindings to the same C and C++ libraries.
If using C++20 onwards, then it is relatively easy to have similar high level abstractions, one just needs to let go of Cisms that many insist in using.
Here Rust has clearly an advantage that it doesn't allow for copy-paste of C like code.
Naturally D and Swift with their safety and C++ interop, would be an option as well.
Re:Should you use Rust in LLM based tools for performance?
by __coaxialcabal on Tuesday October 01, 2024 @12:42PM
(#41710841)
Have you had any success using LLMs to rewrite Python to rust?
Re:Should you use Rust in LLM based tools for performance?
by throwup238 on Tuesday October 01, 2024 @02:35PM
(#41712374)
They’re very good at porting code between languages but going from a dynamically typed language with a large standard library to a static one with a large library ecosystem requires a bit more hand holding. It helps to specify the rust libraries you want to use (and their versions) and you’ll probably want to give a few rounds of feedback and error correction before the code is ready.
Re:Should you use Rust in LLM based tools for performance?
by nicce on Tuesday October 01, 2024 @12:59PM
(#41711082)
> Python is an incredibly easy interface over these native libraries, but has a lot of runtime costs.
It also means that many people use Python while they don't understand that what part of the code is actually fast. They mix Python code with wrappers to native libraries, and sometimes the Python code slows down the overall work substantially and people don't know that fault is there. E.g use Python Maths with the mix of Numpy math bindings, while they can do it with Numpy alone.
It gets worse because Python doesn't expose you to memory management. This initially is an advantage, but later on causes bloat.
Python is an incredibly easy interface over these native libraries, but has a lot of runtime costs.