I tried to get into machine learning and started out simple. A network of two inputs, and one output. The function to learn was a heart-shaped SDF. I generated training samples from the +/-1 range of coordinates.

For such a relatively simple thing it either didn't converge, or when it did, it got decent error rates. But try feeding it a coordinate outside its training range and it very very very quickly starts outputting total nonsense. This exercise taught me that despite whatever they keep telling us, this shit will never generalize.