Re:The killer app of Gemini Pro 1.5 is video
by simonw on Wednesday February 21, 2024 @02:46PM
(#39458558)
Despite that being in their blog post, I'm skeptical. I tried uploading a single frame of the video as an image and it consumed 258 tokens. The 7s video was 1,841 tokens.
I think it's more complicated than just "split the video into frames and process those" - otherwise I would expect the token count for the video to be much higher than that.
UPDATE ... posted that before you edited your post to link to the Gemini 1.5 report.
684,000 (total tokens for the movie) / 2,674 (their frame count for that movie) = 256 tokens - which is about the same as my 258 tokens for a single image. So I think you're right - it really does just split the video into frames and process them as separate images.
I think it's more complicated than just "split the video into frames and process those" - otherwise I would expect the token count for the video to be much higher than that.
UPDATE ... posted that before you edited your post to link to the Gemini 1.5 report.
684,000 (total tokens for the movie) / 2,674 (their frame count for that movie) = 256 tokens - which is about the same as my 258 tokens for a single image. So I think you're right - it really does just split the video into frames and process them as separate images.