Llama 3-V: Matching GPT4-V with a 100x smaller model and 500 dollars

Re:Llama 3-V: Matching GPT4-V with a 100x smaller model and ... by abbaselmas on Wednesday May 29, 2024 @05:02AM (#40509917)

https://ollama.com/library/llava-llama3 llava-llama3

[ Parent | on HN ]

Re:Llama 3-V: Matching GPT4-V with a 100x smaller model and ... by CGamesPlay on Wednesday May 29, 2024 @06:19AM (#40510360)

This appears to be a Llava model which was then fine-tuned using outputs from Llama 3. If I understand correctly, that would make it Llama-2-based.

[ Parent | on HN ]

Re:Llama 3-V: Matching GPT4-V with a 100x smaller model and ... by GaggiX on Wednesday May 29, 2024 @06:31AM (#40510422)

>fine-tuned using outputs from Llama 3.

Llama 3 outputs text and can only see text, this is a vision model.

>that would make it Llama-2-based.

It's based on Llama 3, Llama 2 has nothing to do with it. They took Llama 3 Instruct and CLIP-ViT-Large-patch14-336, train the projection layer first and then later finetuned the Llama 3 checkpoint and train a LoRA for the ViT.

[ Parent | on HN ]

Re:Llama 3-V: Matching GPT4-V with a 100x smaller model and ... by qeternity on Wednesday May 29, 2024 @08:37AM (#40511292)

That is someone else who has just used the Llava name.

It is not by the original group who have published a series of models under the Llava name.

[ Parent | on HN ]