The horse itself cost at least a third of a labourer’s yearly income (which was not really disposable), and that does not include a sufficiently large pasture or equivalent fodder.

Not only that, but those horses would have been work implements, like a tractor today, you would usually not get those horses out to go to church or see your family a village over. If only because most of the time the horse would be at work in the fields anyway.