News for hackers. Stuff that matters.

jvstokes

Re:Run LLMs on Apple Neural Engine (ANE) on Saturday May 03, 2025 @03:03PM (#43881303)
But coreml utilizes ANE, right? Is there some bottleneck in coreml that requires lower level access?
This is cool, and really makes sense for large organizations. Do you foresee a release for smaller enterprises (something as simple as a lightweight aws integration?)