News for hackers. Stuff that matters.
Re:Llama 3.2: Revolutionizing edge AI and vision with open, ... by refulgentis on Wednesday September 25, 2024 @07:09PM (#41652825)
I'm writing a Flutter AI client app, integrates with llama.cpp. I used a PoC of llama.cpp running in WASM, I'm desperate to signal the app is agnostic to AI provider, but it was horrifically slow, ended up backing out to WebMLC.

What are you doing underneath, here? If thats secret sauce, I'm curious what you're seeing in tokens/sec on ex. a phone vs. MacBook M-series.

Or are you deploying on servers?

There are no comments on this comment yet.