awadallah

I am CEO and one of cofounders of Vectara. We are very proud of the release of this open source eval model. We certainly would like to add more LLMs to the scorecard, and would love to collaborate with others to make the evaluation model even more accurate. Please reach out to bader@ or simon@ if interested.

What are the limitations and challenges of Boomerang in terms of scalability to a large corpus with tens of millions off questions?

How does Boomerang handle the trade-off between speed and accuracy? Does it sacrifice the quality of the results for faster response time?

yes, and for voice, video, 3d objects, the digital knowledge at large.

We, humans, are preconditioned to be linear in our extrapolation (as opposed to exponential) thanks to our hunter ancestors (and FPS games!). It is very clear that the rate of advancement of Large Language Models is super-linear, if not exponential.

Hence, I indeed predict that keyword search will be completely supplanted in the next 5 years as a mechanism for search.

Of course we will still need to do lookups for ISBNs and generic ids, but that isn't keyword search, that is index lookup functionality.

Case in point: take a look at Meta Research's Contriever model (https://github.com/facebookresearch/contriever), which already matches keyword techniques in efficacy without any supervision.

This is only the beginning, come build the future with us, we see it very clearly :)