There's some paid services like http://osmbuildings.org but they're just as limited... and paid!!
I also didn't want to rely on a paid provider: that's why I chose Leaflet/OSM instead of Google maps, which has polygons for buildings. I felt pretty vindicated a few months later when they (apparently) raised their prices ~1400%, though!
Microsoft did release this (https://blogs.bing.com/maps/2018-06/microsoft-releases-125-m...), and I thought it'd have a lot more coverage. But their blog post makes it sound like it's already been integrated with OSM, which doesn't seem to have those hoped for polygons.
I was wary about generating houses just from the street vectors at first, and wasn't sure how I would even do it, but it ended up working out! They also made translating positions within units (like (1,1) for the back corner) convenient, which makes navigating agents around a lot easier.
So I'm less disappointed about the lack of building polys in OSM than I was at the beginning. I might say it kind of benefited me, even!
The MS buildings haven't been merged yet.
And that sounds like a good idea. If I don't do it soon, feel free to implement it as an extension and I'll link to it from the AgentMaps repo!
There will be traffic on https://lists.openstreetmap.org/listinfo/imports and some pages on the OpenStreetMap wiki at some point.
It's pretty easy to do:
http://overpass-turbo.eu/s/BDR
(Note that Overpass Turbo is an IDE for Overpass-API, you can directly query the api without using Turbo)