Especially pandoras metrics just don't cut the subtleties of "good taste". E.g. listening to something like Bach, with many interprets, there will be just a few that play it to my liking, even though most superficial (every single note) stays the same.

Another problem might be the inability to split a personality. Many engines take the 7 people that behave most like you as a reference, but might compromise when trying to find people that like exactly the genres you like. (if you try to find people that listen to the same set of genres your pool of candidates gets so small, that the similarities within the genres are very few).