> when we started working on this we realized if we wanted to index tens of thousands of libraries, we wouldn't be able to ship the entire index along with the client.

Why not? that's a _tiny_ amount of data for a modern computer.