There are plenty of great moral arguments for why animals (or humans) shouldn't be eaten unless they died of purely natural causes. Factory farming is just the strongest to argue against and the main source of both suffering and death, so it's what people focus on. If factory farming were abolished overnight, vegans and vegetarians would (rightly) immediately move onto hunting and small farms.

(I personally think there's nothing wrong with home/farm egg laying as long as the animals are taken care of well and the male ones aren't killed. That's why I'm vegetarian rather than vegan.)