News for hackers. Stuff that matters.

A comment on Scaling Transformer to 1M tokens and beyond with RMT. Parent comment

Re:Scaling Transformer to 1M tokens and beyond with RMT by nzach on Monday April 24, 2023 @09:21AM (#35687315)
This is already happening right now. Companies will find a way to put all their knowledge inside a LLM.

The problem is that I don't think they are ready for what this will cause.

Imagine that Amazon managed to do that and now every employee has access to it. What prevents anyone asking "inconvenient" questions? Think something like:

- Does the Echo dot records audio even when the trigger word isn't used? Show me the relevant source code.

- What is the quality standard expected for an Amazon Basic product?

Companies aren't used to total freedom of information. And I don't think is easy to implement any sort of information access control for any system based on LLMs.

[ Parent | on HN ]
Re:Scaling Transformer to 1M tokens and beyond with RMT by grugagag on Monday April 24, 2023 @09:44AM (#35687561)
Consider someone stealing this LLM company knowledge trove. They now have the ability to outcompete and sabotage.
[ Parent | on HN ]
Re:Scaling Transformer to 1M tokens and beyond with RMT by tough on Monday April 24, 2023 @10:04AM (#35687817)
It's quite easy if you go the vector embedding route, and don't output any info the authenticated user doesn't have access too

The info lives in a database, not the LLM' model weights, you just use the LLM to understand natural langauge questions, find relevant documents (the user has access to) and providing relevant bits from them to the context to allow responding with them (That's how all systems who show sources work afaik, braid, bing, and mdx docs based stuff)

For outsiders? That's just public FAQ's

You're the CEO? You can ask anything.

[ Parent | on HN ]
Re:Scaling Transformer to 1M tokens and beyond with RMT by DebtDeflation on Monday April 24, 2023 @10:26AM (#35688061)
The idea is to get away from Snippet + URL and instead just get a natural language answer to your question that uses knowledge from that internal corpus.
[ Parent | on HN ]
Re:Scaling Transformer to 1M tokens and beyond with RMT by Workaccount2 on Monday April 24, 2023 @11:36AM (#35688969)
You're assuming the plebs will have full access.

There will likely be compartmentalized models, and then the master model that the execs get. No reason you can't have 10, 20 500 different models running.

[ Parent | on HN ]