Bing: “I will not harm you unless you harm me first”

Re:Bing: “I will not harm you unless you harm me first” by sdenton4 on Wednesday February 15, 2023 @07:18PM (#34813049)

I think the statement is that the LLM is given access to internet search, and therefore has a more recent functional memory than its training data.

Imagine freezing the 'language' part of the model but continuing to update the knowledge database. Approaches like RETRO make this very explicit.

[ Parent | on HN ]

Re:Bing: “I will not harm you unless you harm me first” by theptip on Thursday February 16, 2023 @12:21PM (#34821776)

I don’t think that parses with the current architecture of GPT. There is no “knowledge database”, just parameter weights.

See the Toolformer paper for an extension of the system to call external APIs, or the LaMDA paper for another approach to fact checking (they have a second layer atop the language model that spots “fact type” utterances, makes queries to verify them, and replaces utterances if they need to be corrected).

It’s plausible that Bing is adding a separate LaMDA style fact check layer, but retraining the whole model seems less likely? (Expensive to do continually). Not an expert though.

[ Parent | on HN ]