News for hackers. Stuff that matters.
Re:GitHub – ShreyaR/guardrails: Adding guardrails to large l... by doppenhe on Thursday March 16, 2023 @03:00AM (#35179018)
This is great, thanks for sharing. Key component in evolving FM based applications is making them feel as deterministic as possible vs probabilistic. Framework like this would enable generating trust in the outputs of these FMs.. exciting.
Re:GitHub – ShreyaR/guardrails: Adding guardrails to large l... by Animats on Thursday March 16, 2023 @03:33AM (#35179209)
Well, sort of. This is mostly for using a large language model to generate JSON or XML or SQL, something for which there's a syntax checker. It guarantees only that the output has the right syntax. If used for censorship, it's just looking for keywords.
Re:GitHub – ShreyaR/guardrails: Adding guardrails to large l... by codetrotter on Thursday March 16, 2023 @04:42AM (#35179612)
Maybe I am a bit too lazy but I don’t understand what “guardrails” add that I wouldn’t get simply from:

1. Denying outputs with blacklisted words or phrases, and

2. Deserialising the JSON with serde_json [0], and denying output that fails to deserialise. If my requirements are very specific I can use strongly typed structs. If my requirements are more loose I can use serde json value types etc

[0]: https://docs.rs/serde_json/latest/serde_json/

Re:GitHub – ShreyaR/guardrails: Adding guardrails to large l... by krandiash on Thursday March 16, 2023 @05:04AM (#35179712)
I've been very mildly involved in this project, so I can give my two cents. While it's true that structural / type checks are not difficult to implement, there's no real need for a back-and-forth when you do static checks -- you either fail out, or run rules to fix.

There's something a bit different that we (should) expect with LLMs (and FMs more generally) since they are fundamentally interactive, so you can actually get them to correct things in interesting ways. Passing the outputs of static checkers back to the models is one nice trick. I (and some friends) have been exploring some stuff with using models in the loop for evaluation (more research side), and I think guardrails is directionally exciting in bringing that kind of vision into more production type settings. There's also just the crud of dealing with LLM code...

Re:GitHub – ShreyaR/guardrails: Adding guardrails to large l... by codetrotter on Thursday March 16, 2023 @06:13AM (#35180070)
Thank you. That makes a lot of sense! Good idea :)
Re:GitHub – ShreyaR/guardrails: Adding guardrails to large l... by thr717272 on Thursday March 16, 2023 @06:42AM (#35180253)
What does FM mean in this context? I already see it mentioned in one top level comment and in the first thread but I don't see a definition here or on the project page.

Edit: after adding Large Language Model to my query it seems I found the explanation: FM stands for "Foundational Model".

https://kagi.com/search?q=llm+large+language+model+fm&r=no&s...

Re:GitHub – ShreyaR/guardrails: Adding guardrails to large l... by asimpletune on Thursday March 16, 2023 @06:54AM (#35180331)
Wouldn’t the right way to create AI guardrails is to to have an antagonistic AI act as a moderator? Like you have one model trained to be as accurate as possible in fulfilling the prompt, and then another AI trained based on how human moderators apply the terms of another, “moderation” prompt. Then you have the two fight on a large training set and when you’re done you have generated a moderated AI.
Re:GitHub – ShreyaR/guardrails: Adding guardrails to large l... by jszymborski on Thursday March 16, 2023 @07:17AM (#35180519)
"LLM moderation" is something that sounds downright dystopic.

Speaking to your comment practically, I feel like it would probably be possible to prompt an LLM to successfully "express X concept that breaks ToS" in such a way that moderation doesn't flag it. It may take clever prompt engineering but that's what these jailbreaks are.

Re:GitHub – ShreyaR/guardrails: Adding guardrails to large l... by la64710 on Thursday March 16, 2023 @08:55AM (#35181251)
It will be fun to watch them argue.
Re:GitHub – ShreyaR/guardrails: Adding guardrails to large l... by can16358p on Thursday March 16, 2023 @09:18AM (#35181536)
Something like a GAN?

Will probably end up something similar though.

Re:GitHub – ShreyaR/guardrails: Adding guardrails to large l... by ape4 on Thursday March 16, 2023 @09:52AM (#35181982)
To be effective the moderator AI would need to be as smart (or smarter) as the source AI. Think of all the ways we have already seen people get around restrictions. Giving instructions for murder isn't allowed but people said they were writing a novel and want to have a murder in it and how could be be done. A smart moderator would see what the user is trying to do and stop it.
Re:GitHub – ShreyaR/guardrails: Adding guardrails to large l... by eurasiantiger on Thursday March 16, 2023 @10:40AM (#35182625)
This is gonna be great for generating domain-specific mock data.
Re:GitHub – ShreyaR/guardrails: Adding guardrails to large l... by anileated on Thursday March 16, 2023 @11:55AM (#35183635)
You can bet people will ask LLMs to generate the rail files ^_^