Re:Tesla: Artificial Intelligence and Autopilot
by rajnathani on Sunday August 22, 2021 @02:58AM
(#28263489)
This other comment here mentions that the chips on the sides (i.e. sides of the grid composite of chips) have access to DRAM: https://news.ycombinator.com/item?id=28254892
Re:Tesla: Artificial Intelligence and Autopilot
by panick21 on Friday August 20, 2021 @09:00AM
(#28245469)
> - They only use SRAM, no DRAM. This is a huge mistake
This seems pretty interesting. They must have had a reason for this approach. Maybe it doesn't matter as much for their usecase.
They clearly had the option of adding DRAM if they wanted to. Why not do it?
- Communication speed is one of the biggest bottlenecks to large models, so their bandwidth of 4TBps is very smart.
- They claim a 1.3x perf/watt improvement, which is not really that great for ASICs compared to GPUs. Perf/watt is probably the most important number in datacenters.
- They only use SRAM, no DRAM. This is a huge mistake, which limits their model size. You can only fit a ~10GB model inside a single tile, versus 80GB models for a single A100 GPU.
- Software / compiler stack is as or more important than the hardware itself, because it dictates how much real performance you can squeeze out of the chips. I think Tesla will need to heavily focus on this area before getting anywhere close to real-world GPU performance.