Groq runs Mixtral 8x7B-32k with 500 T/s

Re:Groq runs Mixtral 8x7B-32k with 500 T/s by michaelbuckbee on Monday February 19, 2024 @11:41AM (#39431789)

Friendly fyi - I think this might just be a web interface bug but but I submitted a prompt with the Mixtral model and got a response (great!) then switched the dropdown to Llama and submitted the same prompt and got the exact same response.

It may be caching or it didn't change the model being queried or something else.

[ Parent | on HN ]

Re:Groq runs Mixtral 8x7B-32k with 500 T/s by tome on Monday February 19, 2024 @11:52AM (#39431921)

Thanks, I think it's because the chat context is fed back to the model for the next generation even when you switch models. If you refresh the page that should erase the history and you should get results purely from the model you choose.

[ Parent | on HN ]

Re:Groq runs Mixtral 8x7B-32k with 500 T/s by michaelbuckbee on Monday February 19, 2024 @01:03PM (#39432744)

Appreciate the quick reply! That's interesting.

[ Parent | on HN ]

Re:Groq runs Mixtral 8x7B-32k with 500 T/s by tome on Monday February 19, 2024 @01:23PM (#39432977)

You're welcome. Thanks for reporting. It's pretty confusing so maybe we should change it :)

[ Parent | on HN ]

Re:Groq runs Mixtral 8x7B-32k with 500 T/s by pests on Monday February 19, 2024 @02:51PM (#39433927)

I've always liked how openrouter.ai does it

They allow you to configure chat participants (a model + params like context or temp) and then each AI answers each question independently in-line so you can compare and remix outputs.

[ Parent | on HN ]

Re:Groq runs Mixtral 8x7B-32k with 500 T/s by xanderatallah on Monday February 19, 2024 @09:56PM (#39437534)

openrouter dev here - would love to get Groq access and include it!

[ Parent | on HN ]