OpenAIs o1 isnt really going that far though. Its definitelly better in some areas, but not overall better.

Im wondering if we can abstract chain of thought further down into the computation levels to replace a lot of matrix multiply. Like smaller transformers with less parameters and more selection of which transformer to use through search.