Re:AMD records its highest server market share in decades
by treprinum on Monday August 12, 2024 @01:33PM
(#41227107)
MKL runs on Ryzens just fine and newest Ryzens all have AVX512 which all new desktop/laptop Intels lack. "AI" on CPU is a gimmick, everybody does it on GPU anyway (outside 128GB RAM M3 Max that has some LLM use, but it's still due to shared memory with GPU).
Re:AMD records its highest server market share in decades
by Der_Einzige on Monday August 12, 2024 @01:37PM
(#41227144)
Certain workloads aren't total "gimmicks" (i.e. small embeddings models), and often those CPU optimizations are on things that absolutely do take advantage of the CPU well (i.e. graph analysis) or on traditional ML algorithms like random forests (intel optimizations for sklearn) which are still important and do still run well on CPUs.
Not to mention that most people have more RAM than they do VRAM.
Running MKL on your datacenter grade AMD processor technically violates some license and it relies tricks like this - https://danieldk.eu/Posts/2020-08-31-MKL-Zen
Technically, if your found to be allowing this as a cloud provider by Intel, they have grounds to sue you, so yes, you can use MKL on a Ryzen at home, but you put yourself at risk for lawsuits by doing this in a data center at scale
Re:AMD records its highest server market share in decades
by 0xDEADFED5 on Monday August 12, 2024 @05:40PM
(#41229632)
I'm doing fast AVX512 embeddings on Ryzen, and fast ONNX AVX512 reranking on Ryzen. Though I do the actual heavy lifting on GPU, doing all the RAG stuff in CPU is helpful. AI on CPU is still mostly a gimmick, but as models get smaller and more capable it's becoming less of a gimmick.
Re:AMD records its highest server market share in decades
by treprinum on Monday August 12, 2024 @07:00PM
(#41230329)
Yeah, but the "AI" on CPU means some basic NPU doing (sparse) matrix multiplication, not AVX512.
Re:AMD records its highest server market share in decades
by 0xDEADFED5 on Wednesday August 14, 2024 @08:16PM
(#41252058)
I'm handling dense, sparse, and colbert vectors generated by BGE-M3 model on CPU, pretty sure that counts as "AI"
MKL, OpenAPI, Intel Optimization for sklearn, for pytorch, etc (there are many more). No equivilants exist from AMD but NVIDIA has an equivalent for all of these (i.e. CuPy).
Using AMD for AI stuff only makes sense if you never believe that you may want to do any kind of ML-like workload on or with the CPUs assistance. This is true for many folks though, but for those it's not true for, they will buy blue until Intel no longer exists.