scotttodd
Re:OpenXLA Is Available Now
on Thursday March 09, 2023 @12:20PM (#35083743)
The post itself doesn't get into too many details about edge deployment, but we're building the core technology to scale down to embedded systems without compromising on feature support. In fact, resource constrained devices really benefit from ahead-of-time compilation to improve binary size and finely control memory usage. We (IREE) published a paper focusing on such embedded device uses: https://arxiv.org/abs/2205.14479.
Re:OpenXLA Is Available Now
on Thursday March 09, 2023 @12:15PM (#35083674)
You can today, though we're still narrowing some performance and feature set gaps. There's a downstream distribution of IREE called SHARK that runs Stable Diffusion and other models on AMD GPUs via Vulkan: https://nod.ai/sd-rdna3-ces2023/
Re:OpenXLA Is Available Now
on Thursday March 09, 2023 @12:11PM (#35083617)
The WebML WG has at least looked at StableHLO (and various other MLIR dialects), yeah. StableHLO is one of the first dialects in that ecosystem to focus directly on stability (and not just being a compiler IR), so it could be a good choice for runtimes / APIs that want to consume graphs of high level ML ops.
In IREE, we have prototypes targeting Wasm and WebGPU with ahead-of-time compilation, and we'd like to see more hardware exposed as compute devices via Vulkan/WebGPU (possibly leveraging extensions for computations like matrix multiplication).