Re:PyTorch Library for Running LLM on Intel CPU and GPU
by captaindiego on Wednesday April 03, 2024 @08:46AM
(#39916732)
Are there any Intel GPUs with a lot of vRAM that someone could recommend that would work with this?
[ Parent
| on HN ]
Re:PyTorch Library for Running LLM on Intel CPU and GPU
by Aromasin on Wednesday April 03, 2024 @09:01AM
(#39916880)
There's the Max GPU (Ponte Vecchio), their datacentre offering, with 128GB of HBM2e memory, 408 MB of L2 cache, and 64 MB of L1 cache. Then there's Gaudi, which has similar numbers but with cores specific for AI workloads (as far as I know from the marketing).
You can pick them up in prebuilds from Dell and Supermicro: https://www.supermicro.com/en/accelerators/intel
Read more about them here: https://www.servethehome.com/intel-shows-gpu-max-1550-perfor...
Re:PyTorch Library for Running LLM on Intel CPU and GPU
by goosedragons on Wednesday April 03, 2024 @08:49AM
(#39916760)
For consumer stuff there's the Intel Arc A770 with 16GB VRAM. More than that and you start moving into enterprise stuff.
Re:PyTorch Library for Running LLM on Intel CPU and GPU
by ZeroCool2u on Wednesday April 03, 2024 @11:53AM
(#39919065)
Which seems like their biggest mistake. If they would just release a card with more than 24GB VRAM, people would be clamoring for their cards, even if they were marginally slower. It's the same reason that 3090's are still in high demand compared to the 4090's.