@orwin has claimed that SOTA LLMs have already hit that diminishing return where spending more money on a SOTA LLM today does not add more value than a non-SOTA LLM (assuming high value tasks).

I never said there will never be a diminishing return. I'm challenging the statement that we've already hit.

Note: We're still scaling chip nodes. It's still worth it for TSMC and chip design companies to invest hundreds of billions into every new chip node every 2-3 years. This is after decades of scaling already.