Re:Groq runs Mixtral 8x7B-32k with 500 T/s
by fennecbutt on Tuesday February 20, 2024 @06:42AM
(#39440143)
Thanks, that's pretty impressive. I suppose with blazing fast token generation now things like diarisation and the actual model are holding us back.
Once it flawlessly understands when it is being spoken to/if it should speak based on the topic at hand (like we do) then it'll be amazing.
I wonder if ML models can feel that feeling of wanting to say something so bad but having to wait for someone else to stop talking first ha ha.
https://www.youtube.com/watch?v=pRUddK6sxDg&t=235s