Gemini: Google's most capable AI model yet

Re:Gemini: Google's most capable AI model yet by carbocation on Wednesday December 06, 2023 @12:40PM (#38547065)

I realize that this is essentially a ridiculous question, but has anyone offered a qualitative evaluation of these benchmarks? Like, I feel that GPT-4 (pre-turbo) was an extremely powerful model for almost anything I wanted help with. Whereas I feel like Bard is not great. So does this mean that my experience aligns with "HellaSwag"?

[ Parent | on HN ]

Re:Gemini: Google's most capable AI model yet by p_j_w on Wednesday December 06, 2023 @05:05PM (#38550177)

>Like, I feel that GPT-4 (pre-turbo) was an extremely powerful model for almost anything I wanted help with. Whereas I feel like Bard is not great. So does this mean that my experience aligns with "HellaSwag"?

It doesn't mean that at all because Gemini Turbo isn't available in Bard yet.

[ Parent | on HN ]

Re:Gemini: Google's most capable AI model yet by carbocation on Thursday December 07, 2023 @12:59AM (#38553343)

I am not sure what Gemini Turbo is. Perhaps you meant Gemini Ultra? Because Gemini Pro (which is in this table) is currently accessible in Bard.

[ Parent | on HN ]

Re:Gemini: Google's most capable AI model yet by p_j_w on Thursday December 07, 2023 @12:20PM (#38559077)

Yes, that's what I meant.

[ Parent | on HN ]

Re:Gemini: Google's most capable AI model yet by tarruda on Wednesday December 06, 2023 @12:44PM (#38547126)

I get what you mean, but what would such "qualitative evaluation" look like?

[ Parent | on HN ]

Re:Gemini: Google's most capable AI model yet by carbocation on Wednesday December 06, 2023 @03:28PM (#38549045)

I think my ideal might be as simple as a few people who spend a lot of time with various models describing their experiences in separate blog posts.

[ Parent | on HN ]

Re:Gemini: Google's most capable AI model yet by tarruda on Wednesday December 06, 2023 @04:25PM (#38549681)

I see.

I can't give any anecdotal evidence on ChatGPT/Gemini/Bard, but I've been running small LLMs locally over the past few months and have amazing experience with these two models:

- https://huggingface.co/mlabonne/NeuralHermes-2.5-Mistral-7B (general usage)

- https://huggingface.co/deepseek-ai/deepseek-coder-6.7b-instr... (coding)

OpenChat 3.5 is also very good for general usage, but IMO NeuralHermes surpassed it significantly, so I switched a few days ago.

[ Parent | on HN ]

Re:Gemini: Google's most capable AI model yet by carbocation on Wednesday December 06, 2023 @04:34PM (#38549796)

Thanks! I’ve had a good experience with the deepseek-coder:33b so maybe they’re on to something.

[ Parent | on HN ]

Re:Gemini: Google's most capable AI model yet by fasttransients on Tuesday December 12, 2023 @03:00PM (#38617575)

Thank you for the suggestions – really helpful for my hobby project. Can't run anything bigger than 7B on my local setup, which is a fun constraint to play with.

[ Parent | on HN ]