News for hackers. Stuff that matters.

A comment on Tencent Hunyuan-Large. Parent comment

Re:Tencent Hunyuan-Large by Tepix on Tuesday November 12, 2024 @02:37AM (#42113541)
How many cores do your CPUs have? Are you using the 64 core EPYC 9334 mentioned in the linked page? Do that many cores provide a speedup versus having fewer cores?
[ Parent | on HN ]
Re:Tencent Hunyuan-Large by DrPhish on Wednesday November 13, 2024 @09:50PM (#42132657)
Yes I have the same engineering sample. I can only keep 48 cores fed with max memory bandwidth given current llama.cpp architectural constraints
[ Parent | on HN ]