We have more food surplus now than ever before. But we also have more distractions than ever before.
Buddha: he survived without a safety net, that's my point.
Having kids and dependents makes it harder, but not impossible. You may have to leave them. Again, OP never said it would be easy.
Epicurus owned a garden and the leisure to sit in it.
Diogenes I can kind of buy about but he still lived off a slave-owning city that had surplus to spare.
Also none of them had kids or other dependents to take care of.