News for hackers. Stuff that matters.

A comment on Intel announces Arc B-series "Battlemage" discrete graphics with Linux support. Parent comment

Re:Intel announces Arc B-series "Battlemage" discrete graphi... by pixelpoet on Tuesday December 03, 2024 @04:40PM (#42311841)
Isn't 2666 MHz ECC RAM obscenely slow? 32 cores without the fast AVX-512 of Zen5 isn't what anyone is looking for in terms of floating point throughput (ask me about electricity prices in Germany), and for that money I'd rather just take a 4090 with 24GB memory and do my own software fixed point or floating point (which is exactly what I do personally and professionally).

This is exactly what I meant about Intel's recent launch. Imagine if they went full ALU-heavy on latest TSMC process and packaged 128GB with it, for like, 2-3k Eur. Nvidia would be whipping their lawyers to try to do something about that, not just their engineers.

[ Parent | on HN ]
Re:Intel announces Arc B-series "Battlemage" discrete graphi... by timschmidt on Tuesday December 03, 2024 @04:43PM (#42311885)
I don't think anyone's stopping you, buddy. Great chat. I hope you have a nice evening.
[ Parent | on HN ]
Re:Intel announces Arc B-series "Battlemage" discrete graphi... by ryao on Wednesday December 04, 2024 @12:15AM (#42314647)
Yes and no. I have been developing some local llama 3 inference software on a machine with 3200MT/s ECC RAM and a Ryzen 7 5800X:

https://github.com/ryao/llama3.c

My experience is that input processing (prompt processing) is compute bottlenecked in GEMM. AVX-512 would help there, although my CPU’s Zen 3 cores do not support it and the memory bandwidth does not matter very much. For output generation (token generation), memory bandwidth is a bottleneck and AVX-512 would not help at all.

[ Parent | on HN ]