Llama.cpp: Full CUDA GPU Acceleration

Re:Llama.cpp: Full CUDA GPU Acceleration by Void_ on Tuesday June 13, 2023 @03:31AM (#36306810)

And Whisper Memos for iOS https://whispermemos.com/

[ Parent | on HN ]

Re:Llama.cpp: Full CUDA GPU Acceleration by b33f on Tuesday June 13, 2023 @03:46AM (#36306934)

The really nice part of Whisper is being able to use it offline and on-device, it seems whisper memos is uploading your audio and notes to a server of unknown security, confidentiality etc.

I like Aiko for on-device transcription both in macOS and iOS https://apps.apple.com/us/app/aiko/id1672085276

[ Parent | on HN ]

Re:Llama.cpp: Full CUDA GPU Acceleration by Void_ on Tuesday June 13, 2023 @04:42AM (#36307392)

Whisper Memos uses OpenAI API. The upside is that it uses the largest model - that would take 2GB on your iPhone.

[ Parent | on HN ]