Pedestrians have a button to press to let the signal know they exist. Until we invent car-hands, that seems equivalent enough.

In a number of those systems, the buttons are a placebo that are wired to nothing. [1] [2]

[1] - https://www.nytimes.com/2016/10/28/us/placebo-buttons-elevat...

[2] - https://www.nytimes.com/2004/02/27/nyregion/for-exercise-in-...

In Japan there is sometimes a button for cyclists/motorcycles to push to get the lughts to change. I assumed this was because there is a sensor in the road to detect cars, but now I'm not so sure.

Not strictly true, a number of different traffic systems detect cars using detection methods such as cameras and radar/microwave.

Actually in Japan pedestrian sensors have been deployed. https://www.kyosan.co.jp/product/traffic04.html

If one set of people are waiting unnecessarily, we should make another set do so also?

If the pedestrian was about to get green (or blue) but a car was detected and the pedestrian was forced to wait then that's not fair is it?

We already prioritize cars enough, pedestrians usually have to press a button to trigger the crossing so they're always waiting but cars don't get punished for getting to the lights a second after the sequence starts.

Excactly.

Close to where I live there are a couple of funny examples:

In one place the lights are red by default for everyone. Cars are detected from 100m or so away, so they barely need to slow down. Pedestrians need to press a button and usually wait for a while.

At another crossing the lights are green by default for the bigger road. A sideroad has car-detection that is ofter rather quick. Pedestrians and cyclists need to press a button and usually wait for a long time.

Just because you don't sit in a car it's ok to wait that much longer.

A random pedestrian is just as likely to be helped by a light change as hurt by it. You provide an excellent example of motivated zero-sum reasoning though!