Deploying Transformers on the Apple Neural Engine

Re:Deploying Transformers on the Apple Neural Engine by vletal on Wednesday June 08, 2022 @10:00AM (#31667463)

I wonder how approachable it would be to optimise a custom model for ANE. According to the code examples at the bottom, the current implementation seems to be a custom model, so no generic solution.

Anyway, it seems that we are at dawn of deploying mode cool models which formerly required cloud computation to the hands of the users. Really cool!

Are we going to see more federated learning being pushed to user devices or is it a dead branch only useful for a few use cases?

[ Parent | on HN ]

Re:Deploying Transformers on the Apple Neural Engine by lupex on Saturday June 11, 2022 @04:29AM (#31702897)

To be honest, I would expect Apple to provide an automl optimization service for this.

Architectural model search is not something new. Apple has the unique access to their chip and a large enough customer base to make it feasible.

[ Parent | on HN ]