"Where Buffet lets lower priority tasks slide or delegates them they still get done by his employees. If you do it then your house is going unmaintained or a relationship is getting neglected etc."

Perhaps that's the lesson.

I'm reminded of the startup aphorism "You don't have to be good if you're great." It's really a statement about market dynamics. If you are "great" (the monopoly provider of a service, asset, or technology that other people value) then you will attract capital and people who are "good" (hardworking and skilled people who specialize in the non-monopoly fields you need). You can then trade access to your skill for access to theirs, and do so with great leverage because there are many more of them than you.

If you don't have a monopoly skill or asset, then - assuming you want to take advantage of these market dynamics - your first priority should be to acquire one. And yes, that means that your house is going unmaintained and your relationships are getting neglected. So what?