If we gave voice to every harsh judgment that comes to mind, we would never stop criticizing other people. There's a tension between the need to see oneself in a positive light and the desire to think positively of others.

If you manage to keep these feelings to yourself, I see no cause for shame.