This is not correct. Momentum is conserved by the spurious solution and there's still an equal but opposite force on another body (the body producing the gravitational force).

I think this example just illustrates a case where the Newtonian model of reality simply does not describe reality itself