Re:The most accurate and cheapest AI for scraping
by Shakahs on Wednesday September 25, 2024 @06:13AM
(#41645682)
Gemini Flash API costs 50% less than gpt-4o-mini. It's not clear why your benchmark recorded 13x more token usage for Flash for the same input.
Re:The most accurate and cheapest AI for scraping
by faangguyindia on Wednesday September 25, 2024 @06:18AM
(#41645713)
If you are based in US, you get 1 billion tokens per day for free from Gemini.
Re:The most accurate and cheapest AI for scraping
by marcell on Wednesday September 25, 2024 @04:36PM
(#41651594)
I checked the code and found the issue. It's a result of Gemini's larger context window.
Basically, the Foxtrot scraping library sends the page in chunks. The chunk size is capped at the max context length of each model, which for Gemini is 1,000,000 input tokens for lite. That's compared to 128,000 for GPT-4o-mini.
Typically, you won't need all the tokens in the page, and sending a million tokens when 100,000 will work is wasteful in terms of cost and runtime, and can also hurt accuracy.
I'm going to re-run the benchmarks with a cap on the prompt size for models like Gemini.
Unfortunately the bigger issue with Gemini is cost, which is too high for the scraping use case.