This hasn’t been my experience. I don’t think this knowledge has been lost and it’s often still applied when incentivized, such as when quality output of machine learning models serves an immediate business need, or in scientific applications of machine learning. By your reference to web scraping I assume your referring to datasets for large language models, but I don’t think the kitchen sink approach is applied out of ignorance, but out of a principled decision to forgo data hygiene in order to collect datasets large enough to feed such large models. Also, strictly speaking, one only needs to apply the critical approach when constructing and using validation datasets, and it’s not uncommon for larger noisier training datasets to outperform smaller more rigorous ones when models trained on both are tested on carefully constructed validation datasets.