Has anyone thought about what it would actually take to make gpt learn at an exponential rate? GPT4 from 3 was already starting to see diminishing returns. GPT3 cost $4.6 million dollars to train. 4 cost $100 million, an order of magnitude more, and required an enormous amount of data. All these training algorithms are O(n^2). Do we actually expect these things to grow exponentially or is it more realistic to expect we're going to spend a lot of compute and treasure only to see disappointing results?