Re:PaliGemma: Open-Source Multimodal Model by Google
by seaal on Wednesday May 15, 2024 @03:47PM
(#40371499)
LM Studio is pretty nice and simple to setup.
Re:PaliGemma: Open-Source Multimodal Model by Google
by sroussey on Wednesday May 15, 2024 @04:56PM
(#40372260)
Can probably run on your iPhone
Re:PaliGemma: Open-Source Multimodal Model by Google
by resource_waste on Wednesday May 15, 2024 @05:03PM
(#40372324)
CPU isnt realistic, regardless what the marketers tell you.
Re:PaliGemma: Open-Source Multimodal Model by Google
by simonw on Wednesday May 15, 2024 @05:14PM
(#40372436)
iPhones have GPUs. I've successfully run Mistral on mine using this app: https://llm.mlc.ai/docs/deploy/ios.html
Re:PaliGemma: Open-Source Multimodal Model by Google
by resource_waste on Wednesday May 15, 2024 @08:41PM
(#40374047)
Please don't say 'what is technically possible'
It blurs what is realistically useful.
Also 'GPUs' No. This is a major red flag man. They do not. The marketers told you this, and you believed them.
Re:PaliGemma: Open-Source Multimodal Model by Google
by simonw on Wednesday May 15, 2024 @08:50PM
(#40374097)
I don't get it.
The iPhone does have a GPU, and it can be used to accelerate AI workloads.
This isn't theoretical. You can see this for yourself by installing https://apps.apple.com/us/app/mlc-chat/id6448482937 and turning off your wifi and running the quite capable Mistral 7B Instruct LLM on your phone.
I've even used it to answer simple questions while I was offline and only had my phone with me.
What am I missing here?
Re:PaliGemma: Open-Source Multimodal Model by Google
by resource_waste on Thursday May 16, 2024 @08:36AM
(#40377573)
What like 50 token responses?
Buddy I'm cranking out 4k.
Also LOL at the GPU thing. Its insane how well Apple marketing worked.
Re:PaliGemma: Open-Source Multimodal Model by Google
by simonw on Thursday May 16, 2024 @08:49AM
(#40377718)
I didn't say an iPhone was as good as a desktop with a dedicated graphics card. I said that an iPhone has a GPU and is capable of running a small but capable LLM.
Most people don't know that's possible yet.
Why do keep indicating that iPhones don't have a GPU?
Re:PaliGemma: Open-Source Multimodal Model by Google
by regularfry on Wednesday May 15, 2024 @05:22PM
(#40372514)
I can and do run phi-2/3 on cpu, which are in the same size class.
Also, I imagine there isnt some oobabooga/automatic1111 for this?