The demo featured in this PR takes about 800-1000ms total to produce search results. How much of that is the actual API is not known. Typically, https request to an API in the cloud will cost you at least 50ms of network latency, more likely 100ms-200ms. If you are running vector search on premise you will obviously not have this overhead.
Text embeddings typically work for short text as well as paragraphs (paragraph embeddings are usually mean/max of word embeddings anyway) simply because most commercial use cases demand handling of short text input (because nobody is inputting a paragraph into a typical search box; what use is a news search if you can not type a single word like 'Biden' or 'gpt3' into it).
Typical Google Cloud Bigtable P(50) is <= 4ms to a GCE VM in the same region.
Source: I work on Google Cloud Bigtable.
EDIT: I should clarify that applies to point reads on an established connection using gRPC-over-HTTP/2 from e.g. Java/golang/C++ client, and doesn’t include cold connection set-up / uncached auth / TLS negotiation, but those are well into P(99.999) territory if you’re doing millions of QPS (which applies to most of our large customers).
The offering is similarity search, not a search engine. They offer image to image as another comparison point.
I was obviously talking about a general use case where a user considers using an API like this vs running Faiss, and their server can be anywhere (a use case that is more common to me personally).
(1) Google's offering returns with-in <5ms, in my experience. (2) the demo is for paragraphs, not short text. You're putting mismatched data into the input, of course it's not going to work. Try a paragraph as suggested.