Re:Classic Data science pipelines built with LLMs
by timr on Sunday February 09, 2025 @08:27AM
(#42990533)
Inappropriate tools are always an option? I can cut a cake with a jackhammer, but....
Anyway, like I said, there are certainly good applications of LLMs, and this is probably one? I wouldn't describe "do market research on prices" as a traditional "data pipeline", but that's just me, I guess.
Re:Classic Data science pipelines built with LLMs
by daxfohl on Sunday February 09, 2025 @05:27PM
(#42994725)
I think you'd tell the LLM to design the pipeline, not be the pipeline. That way you can see exactly what it's done and tweak as needed. Plus should be way more cost effective.
Anyway, like I said, there are certainly good applications of LLMs, and this is probably one? I wouldn't describe "do market research on prices" as a traditional "data pipeline", but that's just me, I guess.