I imagine that’s because it’s difficult for the model to decide which features are important. Understanding my human bias in looking for an intuitive explanation, the “circle” in this example looks most like the circle in the picture it identified as most similar. I imagine salience assignment is difficult without either more examples or injecting prior knowledge.

Yea I was thinking something similar, the alignment of the eyes is also most similar to that class as well.

The reason we find the frown to be most important is our understanding that this is a common symbolic representation of an emotion.

If we were to look at it as just an abstract shape, I think we'd come to the same conclusion as the algorithm though.

Is it really that hard to determine what changes more geometrically?

:) :( :| :O :D :/ ;)

wouldnt the solution be more accurate if you partitioned the picture down into smaller segments and then just use the algorithm on each part?