Nvidia's latest AI PC boxes sound great – for data scientists with $3k to spare

Re:Nvidia's latest AI PC boxes sound great – for data scient... by derekp7 on Monday March 31, 2025 @11:10AM (#43535945)

You can also get an Epyc 9115 for $800, motherboard for $640, and 12 16-GiB ddr5-6400 dims for $1400, that gives you 614.4 GiB/sec, for around $2800. You may also want to add in a small GPU to do prompt processing (inference on a CPU is memory bandwidth bound, prompt processing is processing bound).

[ Parent | on HN ]

Re:Nvidia's latest AI PC boxes sound great – for data scient... by dcrazy on Monday March 31, 2025 @11:17AM (#43536019)

How does CPU-based inference compare to GPU-based inference, performance-wise? And aren’t these machines likely to be used for training?

[ Parent | on HN ]

Re:Nvidia's latest AI PC boxes sound great – for data scient... by fransje26 on Monday March 31, 2025 @04:24PM (#43539515)

In which world do you get 614GiB/s memory bandwidth with an Epyc?

I think the best you can dream of is 480.0 GB/s, so 447 GiB/s.

[ Parent | on HN ]

Re:Nvidia's latest AI PC boxes sound great – for data scient... by derekp7 on Monday March 31, 2025 @07:15PM (#43541018)

I was going by the number of memory channels the CPU spec says it supports (12). But apparently I was wrong, as that gets bottlenecked by the number of CCDs on the chip. In which case you would need to go with a much higher end epyc processor, and then there are other limits. So much for napkin math

[ Parent | on HN ]