From the sound of it, I assumed it was building up a 3d world scene graph composed by objects supplied by independent processes. Maybe I'm reading too much into it.

Yeah I think the "physically based 3D renderer" is a bit misleading because it makes you think "ah, like Blender or Unreal Engine 4", but actually I think they just mean they are doing window translucency and drop shadows in a physically correct way (e.g. there is an actual light source) rather than faking it. It is still used to render mostly-2D windows.

(This is my assumption anyway - I haven't tried running it)

I think you assumption is spot on.

Android already does physically correct soft shadows in real time.

The initial design of material would have allowed way more space manipulations, like making a view rotate on any axis.

So the shadow implementation allows to remain correct even in this kind of case.

It is kinda sad that the final design forbids this, a simple raster shadow would have been enough to just signify the elevation.