Because unlike speed acceleration is not relative.

Acceleration can be measured without reference to anything else. If you are in a box and accelerating you can measure it - it feels exactly like gravity.

He feels that gravity, they don't.