News for hackers. Stuff that matters.
Re:OpenXLA Is Available Now by sorenjan on Thursday March 09, 2023 @07:13AM (#35080460)
Will this make it easier to ship ML models to consumers? Let's say I'm making a photo editor, can I ship trained models for various image effects and generation using this, and it will run on the client's best available hardware on Windows, Mac OS, Android, etc?
Re:OpenXLA Is Available Now by sanxiyn on Thursday March 09, 2023 @07:27AM (#35080525)
Sadly, I don't think so. Android already has NNAPI, but this post doesn't mention NNAPI at all. This seems focused on running on servers, instead of running on user devices.
Re:OpenXLA Is Available Now by sorenjan on Thursday March 09, 2023 @08:13AM (#35080825)
That would be a shame. Where does Apple fit in then, are they running custom ML frameworks in their cloud?
Re:OpenXLA Is Available Now by scotttodd on Thursday March 09, 2023 @12:20PM (#35083743)
The post itself doesn't get into too many details about edge deployment, but we're building the core technology to scale down to embedded systems without compromising on feature support. In fact, resource constrained devices really benefit from ahead-of-time compilation to improve binary size and finely control memory usage. We (IREE) published a paper focusing on such embedded device uses: https://arxiv.org/abs/2205.14479.
Re:OpenXLA Is Available Now by mikaraento on Thursday March 09, 2023 @01:07PM (#35084475)
We've shared some plans for client-side inference (TFLite) support in https://www.w3.org/2022/11/17-webmachinelearning-minutes.htm.... The presentation was for WebML but mentions non-WebML work. It's not shipping yet though.

(I work for Google and I work on client-side StableHLO, but I don't speak for Google).

Re:OpenXLA Is Available Now by JacquesPienaar on Thursday March 09, 2023 @01:23PM (#35084712)
That's a very good question and definitely something of interest. Note, that the compiler is only part of this story (as Mika also mentioned here). With OpenXLA we want to be able to take advantage of the best of what each platform can provide and opsets like StableHLO are meant to provide a portability layer while being expressive enough that targeting specialized hardware efficiently is possible. If you look inside the openxla/iree repo (as well as iree/iree-samples and iree/iree-jax repos, paper Scott cited or sers of IREE like Shark (https://github.com/nod-ai/SHARK#quick-start-for-shark-stable...)) you'll see some example.