I’m not an expert, but I’ve always felt the use of softmax everywhere is a contributing factor. It’s basically saying “you HAVE to pick one” rather than “which one would you pick?”