The distant stars are important because they make space flat. Remember that general relativity says mass bends space, or more accurately is the same thing as space.

So if all the mass was nearby, space would bend inward around the current point and it would fold in on itself - the universe would be like the inside of a sphere.

Because the mass is distributed evenly at the largest scales, it instead continues flatly in all directions.