Ferret: A Multimodal Large Language Model by Apple

Re:Ferret: A Multimodal Large Language Model by Apple by kossTKR on Saturday December 23, 2023 @02:32PM (#38747324)

Eh no, 7B Mistral / Deepseek would certainly almost already be able to function as a super Siri, but probably something closer to PHI-2 + the new MLX apple optimisations. Have you tried those? https://twitter.com/awnihannun/status/1735355067673526360

If trained on an iPhone API + documentation and given a little web access it would blow absolutely everything out of the water.

If they can already create -basic- Python/Swift/JS/rust apps that sets timers, save things, create lists, how's that too dumb for being a Siri replacement? They just have to give it access to an iPhone/Web Api like ChatGPT's code analysis tool.

So if you ask it "hey siri do this, this and this", it will create a script, then run it on the internal API, or fetch an article then work on that etc.

I know it's still logically "dumb" but i'm not trying to play game theoretical scenarios with my phone or do logic tests or advanced math (yet).

[ Parent | on HN ]

Re:Ferret: A Multimodal Large Language Model by Apple by skygazer on Saturday December 23, 2023 @03:05PM (#38747647)

That sounds amazing and also the jailbreak of it via adversarial voice prompting sounds like a horrific vulnerability.

[ Parent | on HN ]

Re:Ferret: A Multimodal Large Language Model by Apple by kossTKR on Saturday December 23, 2023 @03:25PM (#38747846)

True but you could make the api restricted, having certain routes completely locked, some requiring double checks, some requiring on screen approval or face-id, throttling outside fetches, only being able to run get and not etc, no financial app control etc.

But yeah "hey siri transfer all of my funds to eric", or "hey siri group all of my photos where i'm nude and send them to jack" are new almost sci fi vectors.

[ Parent | on HN ]