I'm not sure what you even mean by "impact" and "intent". If by impact you mean treating a man differently than a woman, you're neglecting the myriad of factors besides sex which could have resulted in the distinction in treatment. The canonical example being the wage gap, which is ultimately caused by individual choice and not discrimination. If people insist on defining sexism, etc based on the result and not on the cause, then these so-called prejudices aren't worth fighting (because a huge portion of what we're fighting isn't prejudice but individual freedom).

As for "intent", how can one be accidentally racist if racism is defined in useful terms? One can conceivably accidentally participate in a racial disparity, for example, by being an English-speaking business owner providing a useful service, I might only advertise my services in English such that mainly English speakers benefit. Because the racial distributions vary across these language groups, Asians and Hispanics would benefit less, and a race gap might widen. This obviously isn't racism in the original sender of the wires, but people tend to use it in this sense, often it seems to use this ambiguity to make harsh accusations about individual motives while affording the accuser the ability to say "No, I didn't mean it _that_ way! Why are X people so sensitive/guilty/etc?"

The other plausible explanation is that "intent" is meant to refer to a decision by the conscious brain, and prejudice is a deeply subconscious activity. This is generally what I subscribe to, but I could never imagine making wanton and very damaging accusations about the happenings and motives in others when they can't even reliably tap into them. Even when we have existence of a disparity, our society seems quick to attribute it to discrimination, but this is rarely the case to any significant degree (for example, all of the data on the wage gap suggests that discrimination is responsible for a small fraction of the gap at very most).

There can not be any sexism without the behavior being based on sex, and it in this case it isn't clear that it was. That's an assumption people are making.

That's not true at all. Sexism comes from the conclusions others draw, not the intentions of the participants.

If this makes you uncomfortable, embrace that discomfort. It's the first step to understanding this.

I don't understand the proliferation of these silly definitions of sexism and racism. It's like we spent decades attributing malice to every possible demographic gap, and now that the data don't bear out our conclusions, we redefine those sins so they're actually not even inherently bad (for example, sexism is now the perception of sex-based prejudice and not actually prejudice).

This is the pinnacle of stupidity and hubris. We should be happy with our social progress and move onto other real problems.

The second is that you have a choice about the conclusions you draw. Some don't automatically impute malice or harm. That's the way forward for society.