I am under the impression it is much less that "most decided" and much more that agricultural based societies were incredibly effective at displacing hunter gatherer societies.

Some combination of much higher birth rates, ecological destruction, and centralized governments among other factors made agricultural societies more successful in the Darwinian sense.

You see this in the history of the Europeans vs Plains Indians. Very few military successes for the Europeans, but ultimately the Europeans displaced the original inhabitants of the Great Plains almost entirely. The Comanches never "chose" to become agricultural. They were forcefully pushed out over generations.