Deep Learning’s Diminishing Returns – Cost of improvement becoming unsustainable

Re:Deep Learning’s Diminishing Returns – Cost of improvement... by mrtranscendence on Friday June 16, 2023 @04:07PM (#36363083)

The amount of computation in the world devoted to deep learning, in aggregate, might be a small slice of total computation. But in terms of costs borne by individual organizations who train such models, deep learning could be extremely substantial. Particularly if it takes millions in additional investment to achieve only marginal reductions in error.

[ Parent | on HN ]

Re:Deep Learning’s Diminishing Returns – Cost of improvement... by dekhn on Friday June 16, 2023 @04:35PM (#36363466)

How is that different from any other capital-intensive industry. Basically my point is that there is nothing specific to DL in these articles/complaints, people are just jumping on the bandwagon.

[ Parent | on HN ]

Re:Deep Learning’s Diminishing Returns – Cost of improvement... by mrtranscendence on Friday June 16, 2023 @10:14PM (#36366519)

For me, the takeaway is that training state of the art models may become cost prohibitive even for large companies. There may be a practical limit to how complex models can become. This is at least plausibly interesting to know.

[ Parent | on HN ]