Because the entire Milky Way is mostly at rest relative to itself.

If you are charging full speed ahead into its center, you are going against just about everything, including energetic particles no doubt coming the from the crowded center of the galaxy.