One way to think about offices is that they are appealing for much the same reason that people demonstrably prefer to live in urban areas instead of rural areas. Over time, people flock to cities because of the efficiencies of having many people in a concentrated area. In many ways, the same thing is true with office jobs.
People living in suburbs or exurbs shouldn't expect to have a dozen eateries within a few blocks of their home, but it's quite common to have those options at work. The economics works because there are many more people per square mile or meter at the office than at home.
Personally, I keep groceries in my house, and almost never go out for lunch, but most people eat lunches out, and offices make that easier.
(it hints at that there is something wrong with such logic).
The pure fact that it provides increased economic activity and therefore is able to support different jobs and the second/third order consequences of those makes perfect sense to me.
In your stay at home model, what would all those people do instead?