Re:AMD Alveo V70 AI inference accelerator card
by marcyb5st on Tuesday January 10, 2023 @03:07PM
(#34330750)
I hope they change their minds. At least now that generative models are becoming somewhat popular. I'd love to be able to get an AMD card to run generative models, but to the best of my knowledge, they only run on Nvidia hardware
[ Parent
| on HN ]
Re:AMD Alveo V70 AI inference accelerator card
by nl on Tuesday January 10, 2023 @06:28PM
(#34332954)
No personal experience, but you can actually get Stable Diffusion to run on AMD cards.
It uses DirectML on Windows: https://gist.github.com/averad/256c507baa3dcc9464203dc14610d... This is thanks to Microsoft, not AMD.
On Linux you can use ROCm: https://www.videogames.ai/2022/11/06/Stable-Diffusion-AMD-GP...
The horrible install processes and what a mess this is is all down to AMD.
Re:AMD Alveo V70 AI inference accelerator card
by hedgehog on Tuesday January 10, 2023 @06:37PM
(#34333039)
I don't have any experience with DirectML but it sounds promising.
Re:AMD Alveo V70 AI inference accelerator card
by hedgehog on Tuesday January 10, 2023 @06:36PM
(#34333024)
I wouldn't hold my breath, and anyway at this point NVIDIA has faster chips and more supported software all the way down the stack. My previous startup tried to solve some of these problems and we built what is as far as I know still the only reasonably complete device-portable deep learning framework. Today something like an RTX 3070 is a good budget option for small experiments and you can always lean on a cloud provider if you need more compute temporarily. Hard to beat a TPU pod when you're in a hurry.