News for hackers. Stuff that matters.

A comment on Groq runs Mixtral 8x7B-32k with 500 T/s.

Re:Groq runs Mixtral 8x7B-32k with 500 T/s by joaquincabezas on Monday February 19, 2024 @05:41PM (#39435709)
are there also experiments around image embedding generation to use in combination with the LLM?

maybe for this use-case is it better to execute the vision tower on a GPU and leave the LPU for the language part?

[ Parent | on HN ]
Re:Groq runs Mixtral 8x7B-32k with 500 T/s by matanyal on Monday February 19, 2024 @06:12PM (#39435960)
We are great for image embedding (and audio, with more to come!) There is no reason you should be forced to use graphics cards intended for gaming for any AI workload.
[ Parent | on HN ]