Strikes me that this is about comparative level of development. As long as large inequalities exist then richer countries can feed off the labour of less developed countries. When those countries develop sufficiently they can't be [ab]used in that way.
This is a big issue in UK at the moment as we've kicked out loud of foreign workers, and now have harvest locally produced food, in part because farmers won't pay wages at UK subsistence levels.
Later in the fourth century there were lots of shenanigans around securing the grain supply for the capital which in that period had shifted to north africa. I don't think we can rule out soil degradation as the reason for the shifts in the productive regions across ancient history, like how Paestum turned into a swamp due to soil erosion.
This consolidation was partially driven by war debts; Roman citizens accrued debts during the wars of conquest, while the generals won the spoils with which they could purchase the land of indebted citizen farmers. It was also driven by economies of scale; the larger estates were more efficient, had lower costs of operation, and had the market contacts to grow the crops that generated the best profit when shipped.
Of course this means Italy basically stopped making food when senators could sell oil at higher margins, and began to depend on Sicily and Egypt to produce the grain for Rome, with the latter eventually becoming the personal property of the emperor. This would cause serious problems later, but the senators profiting neither knew nor cared.