News for hackers. Stuff that matters.
Re:Run LLMs on Apple Neural Engine (ANE) by conradev on Saturday May 03, 2025 @03:04PM (#43881308)
It essentially never makes sense to run on the CPU and you will only ever see enthusiasts doing it.

Yes, hammering the GPU too hard can affect the display server, but no, switching to the CPU is not a good alternative

Re:Run LLMs on Apple Neural Engine (ANE) by kamranjon on Saturday May 03, 2025 @03:10PM (#43881335)
Not switching to the CPU - switching to the ANE (Neural Cores) - if you read the research papers Apple has released - the example I gave is pretty much how it's being used - small image classification models running on the ANE, alongside a graphics app that needs the GPU to be free.
Re:Run LLMs on Apple Neural Engine (ANE) by conradev on Saturday May 03, 2025 @07:37PM (#43883247)
Oh, yes, I misread! It’s great for that