It saves money if you cast off things you don't need, such as an extra car (my wife and I now car share), business casual clothing, eating out, &c, and if you live somewhere cheap. I live somewhere cheap (Louisiana), so I have a house with a couple of spare bedrooms from which to work.
But if the company is assisting remote work, e.g. paying for coworking space with an allowance that is probably still cheaper. Consider a team of 10... Even at a $200 / mo allowance for coworking that it still only 20~30% of what it would cost per month for a decent space in most major cities and there's no long term commitment.
So I think when you scale it out for small to medium teams it is cheaper and would be cheaper for a lot of startups that weren't already in a shared space.
Even if every employee worked in their own coworking space, the advantage is that they can choose one within walking distance of their front door.
No-one needs to cause carnage to their family life if they are forced to change jobs multiple times. The idea that to find a job, you have to relocate and separate your children from their friends, clubs and schooling is quite horrifying really.
Depending on your locale and situation, you might get tax breaks and/or company subsidies for a home office and equipment as well.
http://www.joelonsoftware.com/articles/FieldGuidetoDeveloper... (Kind of old, but still applicable.)
Supplying an allowance to rent somewhere with a good workspace or set up an office is still mostly cheaper than renting an inner-city space with private offices... and then you don't have issues with VCs thinking you're burning their money unnecessarily.
Don't forget that, in working from home, you do want to have an office; its just a home office. For a lot of people, that means paying more for a home with an office room. Overall, working from home doesn't save money, it saves time (which is, at least to me, worth more than the extra money is costs to work from home).