It is interesting. Technically it should be the same amount of energy transferred but without localised damage (eg, concentration of force through a knuckle into an area of bone or tissue). The head will still be jarred, so your point is pretty solid.

Regardless, though, in this specific example - if one of these guys throws a stray fist to the mouth or jaw (not so much for body shots because they're not exactly doing much of a job of it), its likely to break something, and if they are boxing in a training only setting or socially they're not going to be taking enough blows for concussion / brain damage to be a real issue.