It's kind of like back when Moore's Law / Dennard Scaling began hitting the scaling/thermal wall, bringing the semiconductor "Golden Era" to an end. They started over-volting and over-clocking the chips to within an inch of their lives to extract more compute when the "Magic Scaling Unicorn" stopped being magic. Now days instead of a new node sometimes being as much as twice the performance at half the cost, most new nodes are ~10-15% more gates OR ~10-15% more speed for ~20% higher cost per gate or clock.

Same is happening in LLMs. Sadly, the Golden Era is over.