News for hackers. Stuff that matters.

A comment on Scaling Transformer to 1M tokens and beyond with RMT. Parent comment

Re:Scaling Transformer to 1M tokens and beyond with RMT by tough on Monday April 24, 2023 @10:04AM (#35687817)
It's quite easy if you go the vector embedding route, and don't output any info the authenticated user doesn't have access too

The info lives in a database, not the LLM' model weights, you just use the LLM to understand natural langauge questions, find relevant documents (the user has access to) and providing relevant bits from them to the context to allow responding with them (That's how all systems who show sources work afaik, braid, bing, and mdx docs based stuff)

For outsiders? That's just public FAQ's

You're the CEO? You can ask anything.

[ Parent | on HN ]
Re:Scaling Transformer to 1M tokens and beyond with RMT by DebtDeflation on Monday April 24, 2023 @10:26AM (#35688061)
The idea is to get away from Snippet + URL and instead just get a natural language answer to your question that uses knowledge from that internal corpus.
[ Parent | on HN ]