Generative AI is a technology that moves the frontier of what is possible. This comes with many opportunities to solve problems that were not previously solvable by a reasonably sized team.

Transformers have some interesting properties like in-context meta learning. This is indeed something to be excited about.

But to say transformers can be AGI by simply increasing context windows is a doubtful proposition.