News for hackers. Stuff that matters.
Re:Apple: Transformer architecture optimized for Apple Silicon by atleastoptimal on Friday March 24, 2023 @01:37AM (#35285444)
Local inference is a pipe dream. There's no way any of these companies will allow consumers to interact with their flagship models on anything but a black box api.
Re:Apple: Transformer architecture optimized for Apple Silicon by kergonath on Friday March 24, 2023 @03:14AM (#35285999)
The black box could well be a local library. Functionally, what would be the difference between using something like AIKit and using a REST API over the network?

Local has a lot of advantages as well (latency, privacy, etc).