Groq runs Mixtral 8x7B-32k with 500 T/s

Re:Groq runs Mixtral 8x7B-32k with 500 T/s by karthityrion on Monday February 19, 2024 @01:04PM (#39432758)

Hi. Are these ASICs only for LLMs or could they accelerate other kinds of models(vision) as well?

[ Parent | on HN ]

Re:Groq runs Mixtral 8x7B-32k with 500 T/s by tome on Monday February 19, 2024 @01:24PM (#39432996)

It's a general purpose compute engine for numerical computing and linear algebra, so it can accelerate any ML workloads. Previously we've accelerated models for stabilising fusion reactions and for COVID drug discovery

* https://alcf.anl.gov/news/researchers-accelerate-fusion-rese...

* https://wow.groq.com/groq-accelerates-covid-drug-discovery-3...

[ Parent | on HN ]

Re:Groq runs Mixtral 8x7B-32k with 500 T/s by karthityrion on Monday February 19, 2024 @01:42PM (#39433163)

So, is this specific chip only for LLMs as the name LPU suggests Language Processing Unit, right?

[ Parent | on HN ]

Re:Groq runs Mixtral 8x7B-32k with 500 T/s by tome on Monday February 19, 2024 @02:37PM (#39433758)

The chip is capable of running general numerical compute, but because we're focusing almost entirely on LLMs at the moment we've branded it the LPU.

[ Parent | on HN ]