News for hackers. Stuff that matters.
Re:Employees Are Feeding Sensitive Biz Data to ChatGPT, Rais... by somenameforme on Tuesday March 28, 2023 @01:08AM (#35336179)
One problem with the current way these models are being trained is that they have no idea of what they're saying. It's just a recursive guess the next word type algorithm. I would not expect the confidence levels of any given fragment, let alone an average, to be a meaningful predictor of truth.

Also completely spitballing, I expect that a big chunk of OpenAI's 'secret sauce' is simple processing layers above and beyond the model. If you input gibberish to llama does it give you an output? If OpenAI is artificially tokenizing inputs (as opposed to just sending inputs straight to the software), it would both dramatically limit the input domain, thus improving output tuning, as well as give "it" the ability to say when it doesn't know something. I put "it" in quotes since that response would not becoming from the LLM, but from the preprocess tokenization system returning an error code in natural language.

I think there's some weak indirect evidence for this in the service itself, since incoherent inputs are instantly rejected, whereas even simple queries take dramatically longer to output even the first word. It's like the input is not even being sent to the LLM software for processing.

Re:Employees Are Feeding Sensitive Biz Data to ChatGPT, Rais... by bob1029 on Tuesday March 28, 2023 @04:08AM (#35337324)
This is a very helpful observation.

I've been debating the idea of building tiers or layers of models to accomplish the same.

It very well could be that this go/no-go pre-processor is simply another ML model trained on a binary classification task. Stack a few of these and you can wind up with some interesting programming models.

Re:Employees Are Feeding Sensitive Biz Data to ChatGPT, Rais... by a_bonobo on Tuesday March 28, 2023 @04:45AM (#35337589)
This would also explain the ease at which ChatGPT gets rid of escapes/bad prompts - they have an additional layer that assesses whether the question could be, for example, racist, and then spits out a 'Sorry as a language-model I am not trained to answer this kind of question'. No need to retrain the main 14B transformer model.