I'm not sure I believe in FOOM, but I don't think the hardware argument is so cut and dry. LLMs have had vast improvements to capabilities attributable purely to software improvements.

Considering how calorically efficient the human brain is when it comes to the breadth and depth of general problems it can solve, I don't feel like LLM software improvements have hit a theoretical limit yet.