News for hackers. Stuff that matters.

A comment on Building a $5k ML Workstation with Tiitan RTX and Ryzen ThreadRipper [video]. Parent comment

Re:Building a $5k ML Workstation with Tiitan RTX and Ryzen T... by colincooke on Wednesday July 15, 2020 @11:27AM (#23846607)
Yeah you're not wrong, but it's a bit misleading. This allows you to run faster, but it does it by allowing you to use a larger batch size (arguably not best practice but your mileage will vary). Memory pooling is a bit different in that you can treat the combined cards as a single card from TF/pytorch.
[ Parent | on HN ]
Re:Building a $5k ML Workstation with Tiitan RTX and Ryzen T... by ivalm on Wednesday July 15, 2020 @03:06PM (#23850096)
But batch size is prob least problem since you can do data parallelism (send half batch to each gpu, combine on cpu).

I think only model bigger than gpu mem is where you really wish for nvlink on v100s.

[ Parent | on HN ]