News for hackers. Stuff that matters.
Re:Gemini 2.0 is now available to everyone by simonw on Wednesday February 05, 2025 @12:58PM (#42952478)
I upgraded my llm-gemini plugin to handle this, and shared the results of my "Generate an SVG of a pelican riding a bicycle" benchmark here: https://simonwillison.net/2025/Feb/5/gemini-2/

The pricing is interesting: Gemini 2.0 Flash-Lite is 7.5c/million input tokens and 30c/million output tokens - half the price of OpenAI's GPT-4o mini (15c/60c).

Gemini 2.0 Flash isn't much more: 10c/million for text/image input, 70c/million for audio input, 40c/million for output. Again, cheaper than GPT-4o mini.

Re:Gemini 2.0 is now available to everyone by iimaginary on Wednesday February 05, 2025 @01:53PM (#42953320)
The only benchmark worth paying attention to.
Re:Gemini 2.0 is now available to everyone by zamadatix on Wednesday February 05, 2025 @01:02PM (#42952546)
Is there a way to see/compare the shared results for all of the LLMs you've tested this prompt on in one place? The 2.0 pro result seems decent but I don't have a baseline if that's because it is or if the other 2 are just "extremely bad" or something.
Re:Gemini 2.0 is now available to everyone by nolist_policy on Wednesday February 05, 2025 @02:21PM (#42953688)
Re:Gemini 2.0 is now available to everyone by qingcharles on Wednesday February 05, 2025 @02:47PM (#42954077)
Not a bad pelican from 2.0 Pro! The singularity is almost upon us :)
Re:Gemini 2.0 is now available to everyone by mattlondon on Wednesday February 05, 2025 @04:24PM (#42955373)
The SVGs are starting to look actually recognisable! You'll need a new benchmark soon :)