Re:Chatbot Arena Leaderboard: Gemini 1.5 Flash, Pro and Adva...
by minimaxir on Tuesday May 28, 2024 @03:21PM
(#40504487)
Note that the web app output is not necessarily the same as the API output, the latter is which is what LMSys uses.
The web apps tend to have extra security layers.
Re:Chatbot Arena Leaderboard: Gemini 1.5 Flash, Pro and Adva...
by jxy on Tuesday May 28, 2024 @03:33PM
(#40504656)
Does LMSys even claim to be using the public API? What API endpoint is "Gemini-Advanced-0514"?
Is LMSys "Gemini-1.5-Flash-API-0514" the same as what the public API endpoint "gemini-1.5-flash-001" provides, or "gemini-1.5-flash-preview-0514"? And is LMSys named "Gemini-1.5-Pro-API-0514" the same as "gemini-1.5-pro-preview-0514"?
Re:Chatbot Arena Leaderboard: Gemini 1.5 Flash, Pro and Adva...
by theropost on Tuesday May 28, 2024 @03:42PM
(#40504761)
Maybe that's a problem for me every time I've tried it completely denies such basic requests, I don't have to worry about how I word things I just want to get my answer based on my inquiry without having to subscribe to a proper way of speaking.
Re:Chatbot Arena Leaderboard: Gemini 1.5 Flash, Pro and Adva...
by gmerc on Tuesday May 28, 2024 @03:23PM
(#40504515)
The hallucination rate on Google models is just not competitive. It’s a disgrace.
Re:Chatbot Arena Leaderboard: Gemini 1.5 Flash, Pro and Adva...
by resource_waste on Tuesday May 28, 2024 @04:44PM
(#40505425)
You just reminded me to sell my Google stock. I got it cheap after it hallucinated in that public press conference. LLMs hallucinate, it was no big deal at the time.
I wrong, the quality IS terrible. But my stock did well. Time to take profits and move it to something like VT or SPY.
Anyone who uses/used Gemini knows its
>Not good, the quality of results are quite generic and low quality. It talks more than it answers. When it talks, it gives you the straight and narrow, nice, happy answer.
>It denies seemingly half of my requests, so I have to reword it.
>It denies genuine requests, even when reworded.
I have chatgpt4 for 99% of things, including the denied requests. I have 7B Berkley Sterling for anything chatGPT4 doesnt answer.
I continue to try Google's, Microsofts, Claude, etc... I am always looking for better, I always pay for premium, I've ran 100B param offline models. Gemini is inferior to their direct competitors, there is nothing redeeming.
I'm convinced Gemini being at the top of any ranking system is some sort of over-fitting error or dealings with Google to promote their product.