It’s not literal guilt like in a criminal trial, it’s a collective responsibility.

“White guilt is a belief that white people bear a collective responsibility for the harm which has resulted from historical or current racist treatment of people belonging to other racial groups” (Wikipedia)

Here’s an analogy: if I see litter on the ground in my neighborhood, I’m not guilty of littering. But if I willfully ignore it and downplay it, rather than putting in extra effort to pick it up, my neighborhood will still have litter.