News for hackers. Stuff that matters.
Re:Should you use Rust in LLM based tools for performance? by oersted on Tuesday October 01, 2024 @12:15PM (#41710425)
Indeed, but Python is used to orchestrate all these lower-level libraries. If you have Python on top, you often want to call these libraries on a loop, or more often, on parallelized multi-stage pipelines.

Overhead and parallelization limitations become a serious issue then. Frameworks like PySpark take your Python code and are able to distribute it better, but it's still (relatively) very slow and clunky. Or they can limit what you can do to a DSL (often SQL), but you can't to much serious data work without UDFs, where again Python comes in. There are tricks but you can never really avoid the limitations of the Python interpreter.

There are no comments on this comment yet.