>> I think one missing step is to fuse training and inference regime into one, just as in animals.

This has always been an important missing piece. Without it ChatGPT is just a natural language interface to the information it was trained on. Still useful but unable to learn (aside from context).