Chiplet ASIC supercomputers for LLMs like GPT-4

Re:Chiplet ASIC supercomputers for LLMs like GPT-4 by fzliu on Wednesday July 12, 2023 @04:34AM (#36691841)

Just skimmed the paper. Seems to me like this paper wants to optimize transformer inference e2e, i.e. from ASIC level all the way to cloud.

I'm not exactly convinced though, since all the results seem to be purely theoretical or simulated. I would've liked to see a prototype built across several FPGAs with clock speeds extrapolated for ASICs.

[ Parent | on HN ]

Re:Chiplet ASIC supercomputers for LLMs like GPT-4 by londons_explore on Wednesday July 12, 2023 @05:37AM (#36692220)

It seems fine to say "others have proved that this math makes a good LLM, we have designed an ASIC that can do this math fast, therefore we can make a good fast LLM"

[ Parent | on HN ]

Re:Chiplet ASIC supercomputers for LLMs like GPT-4 by JonChesterfield on Wednesday July 12, 2023 @06:05AM (#36692398)

Yes, but saying that shouldn't be mistaken for "we can make an asic that runs some model fast". There's a wide implementation void between the two.

[ Parent | on HN ]

Re:Chiplet ASIC supercomputers for LLMs like GPT-4 by kraken12 on Wednesday July 12, 2023 @02:45PM (#36699765)

Yep, it's a research paper in comp arch, the initial proof-of-concept study before you go and spend real money on it.

[ Parent | on HN ]

Re:Chiplet ASIC supercomputers for LLMs like GPT-4 by saturn99 on Wednesday July 12, 2023 @06:15PM (#36702402)

I think FPGAs would be an awesome prototype but maybe too constricting in terms of resources? The extrapolation might be so far out to be just as accurate as their simulated model...

[ Parent | on HN ]