supo

Founder here, happy to answer questions about self-hosting & serverless. One pattern we saw is people overflowing from their own-cloud setup to serverless for spiky & non-private workloads on synthetic data, wide-model evals etc.

The environment allows for separation of the useful good ones from the "only in it for the cash" ex-investment banking types ^_^

Open-source k8s cluster for serving a wide catalog of small language models across search, OCR & data processing.

by that same logic, why would you not strive to push all the signals you have available into the ANN search? sure, some will have reduced resolution vs using a heavy reranker, but surely the optimal solution is to use the same signals in both stages and just add resolution in the second stage? the more they are aligned, the fewer candidates you need -> better latency & lower cost.

one thing to remember is that bm25 is purely in the domain of text - the moment any other signal enters in the picture (and it ~always does in sufficiently important systems), bm25 alone can literally have 0 recall.

Here is a quick overview, doesn't really explain the deep details though: https://www.youtube.com/watch?v=ikYsr6nvbdE

Basically think mixture of experts, but each expert is an encoder with it's own input tokenization/feature engineering part and then there is some machinery to parametrize these encoders based on natural language queries and stitch their outputs together to form unified vectors for objects in the index and the queries coming in.

Given that the whole framework is Apache 2.0, you can also check https://github.com/superlinked/superlinked and the docs: https://docs.superlinked.com/ https://docs.superlinked.com/concepts/overview

We have some proprietary tech besides that - mainly embedding models for things like location and then executors for the Superlinked DAG that help you run it on GPU-accelerated Spark and something analogous to that for streaming workloads - those proprietary things are how we make money, we call it Superlinked Cloud.

This article focuses on ways to make "pre-fetching" more accurate, reducing or eliminating the need for reranking to improve latency/cost but also sometimes quality - for example if you use a text cross-encoder to rerank your structured objects, you'll find that those rerankers don't actually understand much of the numbers, locations and other data like that.

If this was a solved problem, every e-com website would already run some variant of https://arxiv.org/pdf/2209.07663 for all their shopping surfaces and some version of "deep research" on every search query. They don't, so the problem isn't solved. And a big part of the problem is still the integration of behavioral, structured and unstructured signals into the underlying retrieval of products and content - which is what the article is about.

Also, the company is doing fine, not sure where that last sentence came from ^_^

It allows faster experimentation because you can't do things like partial embedding updates and reasonable schema migrations on your vector search index - if you could, you'd experiment in retrieval... and with better retrieval you don't have to move 100s or 1000s of candidates from a database and pay a ton for a ranker inference on every query (not even mentioning the latency impact of that).

If you could wave a magic wand and push all the ranking signals into retrieval and that index would be fast to update and not that expensive to operate - you would do that and you would delete all your reranking systems, wouldn't you?

We are mostly focused on natural language search in e-commerce/marketplace/travel settings right now. There we have a production deployment that drove $15M of incremental revenue through personalization of a shopping feed for a fashion e-commerce business. Happy to share more if you are interested, feel free to email me at [email protected]!

We focus on text, images, numerical, categorical and timestamp-properties for the objects you vectorize with Superlinked. The performance will depend on which models you chose to use with the framework, your queries etc. Happy to elaborate if you could describe your use-case :-)

Supported VDBs: Currently MongoDB, Redis Search and Qdrant https://docs.superlinked.com/run-in-production/index-1

As you say, AstraDB also offers vector search (you can see it at https://superlinked.com/vector-db-comparison) but we haven't built the integration with them yet. We focus on fewer but higher quality integrations in general. Do you like AstraDB? Any advantages vs competition that you care about?

In terms of the plan you mentioned Astra+Redis - you don't have to do that, you can just use redis for both key-value store and vector search.

A table comparing the features available with different vector databases. The project is open-source and developed by a community of practitioners with points of contact from the different vector databases to validate.

And you decided to not work remotely and 10x your income instantly, because you'd miss your office and colleagues. Got it!

Any apps that can benefit from social interactions. A good example is DuoLingo. They built a social+ product for learning languages, have 50m MAUs and $250M revenue!! There are thousands of other learning apps, but DuoLingo is by far the best one out there.

The removal of barriers favors those who can take advantage of the newly acquired access and information.

Your life reflects this capacity as much as the environment that let it manifest itself:)