I think TinyStories is a promising direction. I just wish we had an alternative to GPT-4 because supposedly you can't use it to train other models.

Or maybe clarification of people publicly saying they are going to ignore that restriction.