tl;dr I think you need a small set of tempo-spatially grounded meanings (though not necessarily agent-related) and you can bootstrap everything from that using only textual knowledge.