Llama 3-V: Matching GPT4-V with a 100x smaller model and 500 dollars

Re:Llama 3-V: Matching GPT4-V with a 100x smaller model and ... by lanceflt on Tuesday May 28, 2024 @05:49PM (#40505978)

- Llava is not the SOTA open VLM, InternVL-1.5 is https://huggingface.co/spaces/opencompass/open_vlm_leaderboa...

You need to compare the evals to strong open VLMs including this and CogVLM

- This is not "first-ever multimodal model built on top of Llama3", there's already a Llava on Llama3-8b https://huggingface.co/lmms-lab

[ Parent | on HN ]

Re:Llama 3-V: Matching GPT4-V with a 100x smaller model and ... by valine on Tuesday May 28, 2024 @06:49PM (#40506424)

Very curious how it performs on OCR tasks compared to InternVL. To be competitive at reading text you need tiling support, and InternVL does tiles exceptionally well.

[ Parent | on HN ]

Re:Llama 3-V: Matching GPT4-V with a 100x smaller model and ... by hovering_nox on Wednesday May 29, 2024 @03:14AM (#40509330)

I think CogVLM2 is even better than Intern at OCR (my usecase is extracting information from an invoice)

[ Parent | on HN ]

Re:Llama 3-V: Matching GPT4-V with a 100x smaller model and ... by mkesper on Wednesday May 29, 2024 @08:26AM (#40511207)

After some superficial testing I with bad quality scans you can find on kaggle I can not confirm that. CogVLM2 refuses to handle scans that InternVL-V1.5 still can comprehend.

[ Parent | on HN ]

Re:Llama 3-V: Matching GPT4-V with a 100x smaller model and ... by gigel82 on Tuesday May 28, 2024 @06:03PM (#40506102)

Like InternVL, no llama.cpp support severely limits its applications. Close to GPT4v performance level and runnable locally on any machine (no need for a GPU) would be huge for the accessibility community.

[ Parent | on HN ]

Re:Llama 3-V: Matching GPT4-V with a 100x smaller model and ... by yieldcrv on Tuesday May 28, 2024 @08:04PM (#40506847)

I’m going to be saying First Ever AI something for the next 15 years for clout and capital, not going to be listening to anybody’s complicated ten step funnel if they’re not doing the obvious

[ Parent | on HN ]

Re:Llama 3-V: Matching GPT4-V with a 100x smaller model and ... by abrichr on Tuesday May 28, 2024 @10:32PM (#40507809)

Thank you for the link! Our initial testing suggests MiniCPM outperforms InternVL for GUI understanding: https://github.com/OpenAdaptAI/OpenAdapt/issues/637#issuecom...

(InternVL appears to hallucinate more.)

[ Parent | on HN ]