Re:Deploying Transformers on the Apple Neural Engine
by amelius on Wednesday June 08, 2022 @10:00AM
(#31667468)
Suprised to find out that there is not a simple abstraction layer that makes it the same across all platforms.
[ Parent
| on HN ]
Re:Deploying Transformers on the Apple Neural Engine
by nmfisher on Wednesday June 08, 2022 @01:27PM
(#31670895)
That was the idea behind ONNX, which has been somewhat successful.
But if you’ve ever had to deploy these models, you’ll know it’s never as simple as promised. Different libraries use different ops, have weird export requirements or even noticeable performance issues.
Re:Deploying Transformers on the Apple Neural Engine
by criddell on Wednesday June 08, 2022 @10:10AM
(#31667603)
I would have been more surprised to find out there was. Simple is rarely goes along with ML / AI libraries.
Any abstraction layer will add overhead and probably present a lowest common denominator interface. The audience willing to accept those costs is small.
Re:Deploying Transformers on the Apple Neural Engine
by amelius on Wednesday June 08, 2022 @10:48AM
(#31668171)
> Any abstraction layer will add overhead and probably present a lowest common denominator interface. The audience willing to accept those costs is small.
Eh ... lots of people use Tensorflow or Torch.
Re:Deploying Transformers on the Apple Neural Engine
by hprotagonist on Wednesday June 08, 2022 @11:00AM
(#31668336)
which together are about 5 separate abstraction layers that don’t agree with each other. (tf 1,2, keras, torch, torch functional, …)
so we add onnx and trt on top! that’ll help, right?