The "bitter lesson" being that you need teams of dedicated scientists, engineers designing increasingly better training to squeeze more out of existing compute? I love how people think bitter lesson was just "oh we added one more GPU and now the model is better than our hand-crafted approach". When really its just the opposite -- models with better approaches were necessary, and models at 10% the size are showing nearly identical performance when some attention is paid to quality. The "savory lesson" is, don't listen to the bitter lesson, it was only true when the 'hand-crafted features' experts were using were vastly over-aligned to human superstition, but if the priors are well-aligned mathematically, it really does help to have a better strategy (and more compute may not help). The reverse-bitter-lesson every trader and investor knows: throwing more compute can often give worse results (e.g. memorization), not better.