Performance of llama.cpp on Apple Silicon A-series

Re:Performance of llama.cpp on Apple Silicon A-series by moneywoes on Tuesday December 19, 2023 @07:35PM (#38704017)

would love to learn more and help out on NPCs llms. What sort of performance hit are you seeing?

[ Parent | on HN ]

Re:Performance of llama.cpp on Apple Silicon A-series by joegibbs on Sunday December 24, 2023 @12:10AM (#38751312)

Sure, here's my writeup about it: https://jgibbs.dev/blogs/local-llm-npcs-in-unreal-engine. There's barely any performance hit, only a slight hitch when it starts generating a response.

[ Parent | on HN ]