The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

Re:The LLama Effect: Leak Sparked a Series of Open Source Al... by og_kalu on Sunday April 09, 2023 @08:48PM (#35507977)

There is such a thing as a text only GPT-4 lol. It wasn't trained to be multimodal from scratch. First a text only version was trained and then it was made multimodal somehow ( the details are unknown but making a text only LLM multimodal isn't new e.g Palm, Flamingo, Blip-2, Fromage). The text only version exists and is what the microsoft researchers had access to.

[ Parent | on HN ]

Re:The LLama Effect: Leak Sparked a Series of Open Source Al... by whimsicalism on Monday April 10, 2023 @03:59PM (#35516713)

That would make sense to me, but AFAIK the existence of text-only trained GPT-4 is not publicly reported? Or I missed this.

[ Parent | on HN ]

Re:The LLama Effect: Leak Sparked a Series of Open Source Al... by lucubratory on Thursday April 13, 2023 @11:11AM (#35556699)

It has been, it was in the Microsoft research paper "Sparks of AGI". You can watch the lead author of the paper, Sebastien Bubeck, present it here: https://youtu.be/qbIk7-JPB2c

It's a good video for understanding GPT-4 as a "What are we sure that LLMs are technically capable of?" exercise. As he notes in the video right at the start, the model was made safe and thus has significantly lower performance in the public release, so the examples he shows aren't replicable in the different model the public has access to.

[ Parent | on HN ]

Re:The LLama Effect: Leak Sparked a Series of Open Source Al... by whimsicalism on Tuesday April 18, 2023 @06:42PM (#35621469)

I see that you are probably referring to the claim at 4:30... but I'm not sure he is actually saying that the early model had no text capability or if it merely was not something they were given access to.

[ Parent | on HN ]