I often wonder how much modern IT infrastructure is simply this mess of 'we have no idea how it really works' blackboxes strung together with API calls.

I suspect you're right about how much of a true understanding they (at google) still have of the behaviour of their search engine.