> There's a line of thought that if we're impressed with what we have, if it just gets bigger maybe eventually 'reasoning' will just emerge as a side-effect. This is somewhat unclear and not really a strategy per se. It's kind of like saying Moore's Law will get us to quantum computers. It's not clear that what we want is a mere scale-up of what we have.

Reasoning ability really does seem to emerge from scale:

https://yaofu.notion.site/How-does-GPT-Obtain-its-Ability-Tr...