AI chatbot example, you're chatting and then there could be some Mark down, latex, images maybe base64 encoded webp, audio files, we can type all this stuff do all the validations and such, understand it better and there's some gains to be had doing that, then likely someone like Facebook comes along with this Giant byte Transformer expert system with all of the data specific optimisations, that's kind of the bitter lesson but I guess the main point is that the problem itself of how to best communicate with AI is not necessarily solved so you can get bogged down typing the best possible ways to do it but it's a moving target.

Same with a lot of systems like say a search system that tracks data using the best embedding, well we don't know what that best embedding is in terms of price performance and encoded knowledge perf it's just a moving target.

Send me the system that tracks important metrics effecting the stock market, or a weather system etc... the sensors are all updating and then an entirely new thing comes along like Starlink that helps us track the weather in totally new ways