Center cube is not moved, but it is rotated. However, since it does not have any colored faces then it's rotations are meaningless.

If the cubes have correct locations, none of them need to have colored faces. The location is enough.

(At least, that's true of the 26 outer cubes. You can't get them all into place without simultaneously aligning them correctly. I don't actually know if correct alignment of the center cube is also required, but it'd be my first guess.)

That's provably false - I witnessed it many times when solving the cube myself. Colored faces determine orientation, in addition to location. In the Rubik solving method that I know (a simple method for amateurs, not remotely close to professional speed cubing methods) there's actually a late stage where ALL the cubes are in their correct locations, except some of the third layer corners might have a wrong orientation - there's a dedicated sequence of turns that allows to solve that.

By the same logic, there are 6 more invisible cubes, 1 above each face. Where does it end?

The center cube is obviously part of the mental model of the cube, the Platonic object that the physical object is supposed to represent.

Nobody believes there are invisible cubes outside the Rubik's cube. That's not the same logic; that's you trying and failing to imagine a problem with the idea that cutting a cube into three parts along each of its three axes will generate a hidden central subcube.