I made an app that runs Mistral 7B 0.2 LLM locally on iPhone Pros

Re:I made an app that runs Mistral 7B 0.2 LLM locally on iPh... by wahnfrieden on Sunday January 07, 2024 @08:18PM (#38907103)

I made a free / mostly open source one for iOS that lets you edit the system prompt

https://chatonmac.com

[ Parent | on HN ]

Re:I made an app that runs Mistral 7B 0.2 LLM locally on iPh... by ricktdotorg on Sunday January 07, 2024 @10:25PM (#38907890)

trying this out!

BTW and FYI i need to reduce the font size on my iOS device to be smaller than i like in order to use your add/replace API key key pages. if the font is "larger than normal" i can't see/focus on the box to enter or paste in the API key. just increase your iOS system font size to trigger this. thanks in advance for fixing, will try out the app!

[ Parent | on HN ]

Re:I made an app that runs Mistral 7B 0.2 LLM locally on iPh... by wahnfrieden on Sunday January 07, 2024 @11:08PM (#38908154)

Thanks for the detailed report - will fix asap, along with releasing the macOS v1.0. I've just soft launched this so far but have more to come so please let me know anything else.

[ Parent | on HN ]

Re:I made an app that runs Mistral 7B 0.2 LLM locally on iPh... by ionwake on Sunday January 07, 2024 @09:13PM (#38907505)

Amazing! Does it submit any data online ?

[ Parent | on HN ]

Re:I made an app that runs Mistral 7B 0.2 LLM locally on iPh... by wahnfrieden on Sunday January 07, 2024 @10:04PM (#38907771)

No.

I definitely do not want any liability of user-generated content or PII or similar. I have no analytics, besides the standard Apple opt-in crash/reporting (not using any 3rd-party service and not sending anything to my own servers).

It downloads configuration from GitHub and HuggingFace directly. It also has OpenAI integration, directly to their servers via BYOK.

[ Parent | on HN ]