haolu7
Re:AITemplate, a revolutionary new inference engine by Meta AI
on Tuesday October 04, 2022 @12:40AM (#33076750)
No, you don't need to train the model differently to use it with AITemplate. Here is an intro example to do inference with AITemplate with a very simple PyTorch model: https://facebookincubator.github.io/AITemplate/tutorial/how_.... For more advanced examples, check out https://github.com/facebookincubator/AITemplate/tree/main/ex...
Re:AITemplate, a revolutionary new inference engine by Meta AI
on Monday October 03, 2022 @04:38PM (#33073369)
AITemplate-PyTorch Stable Diffusion is the fastest stable diffusion inference solution by pushing image generation below one second on A100 (batch 1: 0.7s / 25 steps, 1.3s / 50 steps; batch 3: 1.6s / 25 steps, per image 0.55s; batch 16 7.9s / 25 steps, per image 0.49s) for the first time, 2.57X faster than Keras' XLA-based GPU compilation solution.
More benchmark numbers and repro at: https://github.com/facebookincubator/AITemplate/tree/main/ex...