That's right. That's why when this experiment is actually done, the mirror is typically rigidly mounted to an optical bench, which is sitting on the surface of a planet. If the mirror were freely floating in zero G, the outcome would be different. It is a worthwhile exercise to calculate how small the mass of the mirror would have to be before you would actually notice a difference in the results.