These are concepts I believe are very important to internalize if you work with machine learning. Fundamentally we are making predictions (ie. guessing) on the nature of entirely unknown information. So there is a certain inherent impossibility to the task in the general sense. It shouldn't always work.
https://en.wikipedia.org/wiki/Problem_of_induction
Another closely related thing is the No Free Lunch Theorem.
https://en.wikipedia.org/wiki/No_free_lunch_in_search_and_op...
These are concepts I believe are very important to internalize if you work with machine learning. Fundamentally we are making predictions (ie. guessing) on the nature of entirely unknown information. So there is a certain inherent impossibility to the task in the general sense. It shouldn't always work.