News for hackers. Stuff that matters.

A comment on Jeff Dean Responds to EDA Industry about AlphaChip. Parent comment

Re:Jeff Dean Responds to EDA Industry about AlphaChip by phonon on Sunday December 01, 2024 @07:45AM (#42288182)
I assume you mean Fig. 6 here?[0]

But that was explicitly limited to 8 hours for all setups. Do they have another paper that shows that you can't increase the number of hours of a smaller GPU setup to compensate?

[0]https://dl.acm.org/doi/pdf/10.1145/3505170.3511478

[ Parent | on HN ]
Re:Jeff Dean Responds to EDA Industry about AlphaChip by wholehog on Monday December 02, 2024 @01:35AM (#42293693)
They also changed the ratio of RL experience collectors to GPU workers (~1/20th the RL experience collectors, 1/2 the GPUs). I don't know what impact that has --- maybe each GPU episode has less experience? Maybe that makes for an effectively small batch size and therefore more chaotic training? But either way, why change things when you can just match them exactly?
[ Parent | on HN ]