Performance of llama.cpp on Apple Silicon A-series

Re:Performance of llama.cpp on Apple Silicon A-series by alberth on Tuesday December 19, 2023 @08:01PM (#38704244)

How is “Edge Computing” any different than what the GP was stating about current cloud hosted (eg OpenAI)?

[ Parent | on HN ]

Re:Performance of llama.cpp on Apple Silicon A-series by falcor84 on Tuesday December 19, 2023 @09:12PM (#38704682)

There are different possible approaches here, but for me, the main benefit would be of having control over the model(s) used. Other than that, low latency would be crucial for effective voice interaction.

[ Parent | on HN ]