News for hackers. Stuff that matters.
Re:Introducing Structured Outputs in the API by Der_Einzige on Tuesday August 06, 2024 @04:06PM (#41174763)
You should also mention that before you had done custom alignment accounting for this feature, that it was an excellent alignment breaker (therefor a big no-no to release too early)

For example, if I ask an LLM to generate social security numbers, it will give the whole "I'm sorry Hal, I can't do that". If I ban all tokens except numbers and hyphens, prior to your "refusal = True" approach, it was guaranteed that even "aligned" models would generate what appeared to be social security numbers.

Re:Introducing Structured Outputs in the API by ethical_source on Tuesday August 06, 2024 @05:37PM (#41175645)
And if LLMs can generate plausible social security numbers, our civilization will fall /s

Christ, I hate the AI safety people who brain-damage models so that they refuse to do things trivial to do by other means. Is LLM censorship preventing bad actors from generating social security numbers? Obviously not. THEN WHY DOES DAMAGING AN LLM TO MAKE IT REFUSE THIS TASK MAKE CIVILIZATION BETTER OFF?

History will not be kind to safetyist luddites.

Re:Introducing Structured Outputs in the API by Terretta on Tuesday August 06, 2024 @06:22PM (#41176062)
I'm less concerned with the AI teams lobotomizing utility, more concerned with damage to language, particularly redefining the term "safe" to mean something like "what we deem suitable".

That said, when zero "safety" is at stake might be the best time to experiment with how to build and where to put safety latches, for when we get to a point we mean actual safety. I'm even OK with models that default to parental control for practice provided it can be switched off.

Re:Introducing Structured Outputs in the API by consteval on Wednesday August 07, 2024 @12:40PM (#41183034)
I don't think it's about better off or worse off, it's about PR and perception.

The wider consumer base is practically itching for a reason to avoid this tech. If it gets out, that could be a problem.

It was the same issue with Geminis image gen. Sure, the problem Google had was bad. But could you even imagine what it would've been if they did nothing? Meaning, the model could generate horrendously racist images? That 100% would've been a worse PR outcome. Imagine someone gens a mistral image and accredits it to Google's model. Like... that's bad for Google. Really bad.