RAG at scale: Synchronizing and ingesting billions of text embeddings

Re:RAG at scale: Synchronizing and ingesting billions of tex... by bluelightning2k on Tuesday October 10, 2023 @03:48AM (#37829524)

Good article BUT I can't fathom that people would use a managed service to generate and store embeddings.

The openAI or replicate embeddings APIs are already a managed service... You would still need to self managing it all just into a different API.

And dealing with embeddings is the kind of fun work every engineer wants to do anyway.

Still a good article but very perplexing how the company can exist

[ Parent | on HN ]

Re:RAG at scale: Synchronizing and ingesting billions of tex... by ddematheu on Tuesday October 10, 2023 @04:08AM (#37829615)

Some engineers find it fun, other might not. Same as everything.

IMO the fun parts are actually prototyping and figuring out the right pattern I want to use for my solution. Once you have done that, scaling and dealing with robustness tends to be a bit less fun.

[ Parent | on HN ]

Re:RAG at scale: Synchronizing and ingesting billions of tex... by raverbashing on Tuesday October 10, 2023 @04:08AM (#37829618)

Sounds like the same people who use langchain's "Prompt replacement" methods instead of, you know, just use string formatting

https://python.langchain.com/docs/modules/model_io/prompts/p...

[ Parent | on HN ]