News for hackers. Stuff that matters.
Re:Classic Data science pipelines built with LLMs by ekianjo on Sunday February 09, 2025 @07:52AM (#42990361)
This would require massively more compute than regular pipelines...
Re:Classic Data science pipelines built with LLMs by plaidfuji on Sunday February 09, 2025 @08:00AM (#42990398)
(1) that delta will decrease quickly, and (2) corporations will gladly pay for compute over headcount to maintain fragile data pipelines
Re:Classic Data science pipelines built with LLMs by ekianjo on Sunday February 09, 2025 @08:13AM (#42990465)
The price will go down, but LLMs reaching 100% accuracy and reliability is another story. We are nowhere close right now.
Re:Classic Data science pipelines built with LLMs by timr on Sunday February 09, 2025 @08:22AM (#42990508)
> (1) that delta will decrease quickly

Is your data pipeline o(n^3) in the number of tokens? If not, then no, it won't.

Re:Classic Data science pipelines built with LLMs by galgia on Sunday February 09, 2025 @07:58AM (#42990390)
If your problem is compute, you are already optimizing. This is here for all the steps before you start thinking latency-compute. Not all use cases are made equal.