You can be an asshole without being an aggressive asshole. Suppose the concert guy is about to shit is pants, he’s therefore very much in a hurry and communicating that non verbally by bumping into people but also trying to avoid confrontations because that would slow him down.

As to unintentional racism/sexism, it’s impossible to treat thousands of different groups equally with every action. The layout of entrances to a shopping mall is going to disadvantage some group because of where people live. Calling that racism or sexism devalues the meaning of the term into irrelevance. So no, there’s still actual racism and that’s what the term is referring to.