News for hackers. Stuff that matters.
Re:Ingesting PDFs and why Gemini 2.0 changes everything by ThinkBeat on Wednesday February 05, 2025 @04:17PM (#42955283)
Hmm I have been doing a but if this manually lately for a personal project. I am working on some old books that are far past any copyright, but they are not available anywhere on the net. (Being in Norwegian m makes a book a lot more obscure) so I have been working on creating ebooks out of them.

I have a scanner, and some OCR processes I run things through. I am close to 85% from my automatic process.

The pain of going from 85% to 99% though is considerable. (and in my case manual) (well Perl helps)

I went to try this AI on one of the short poem manufscript I have.

I told the prompt I wanted PDF to Markdown, it says sure go ahead give me the pdf. I went upload it. It spent a long time spinning. then a quick messages comes up, something like

"Failed to count tokens"

but it just flashes and goes away.

I guess the PDF is too big? Weird though, its not a lot of pages.

Re:Ingesting PDFs and why Gemini 2.0 changes everything by sumedh on Wednesday February 05, 2025 @05:10PM (#42956006)
Take a screenshot of the pdf page and give that to the LLM and see if it can be processed.

Your PDF might have some quirks inside which the LLM cannot process.

Re:Ingesting PDFs and why Gemini 2.0 changes everything by stigz on Wednesday February 05, 2025 @09:04PM (#42958103)
I experienced something similar. My use case is I need to summarize bank statements (sums, averages, etc.). Gemini wouldn't do it, it said too many pages. When I asked the max number of supported pages, it says max is 14 pages. Attempted on both 2.0 flash and 2.0 pro in VertexAI console.
Re:Ingesting PDFs and why Gemini 2.0 changes everything by shijithpk on Thursday February 06, 2025 @07:16AM (#42961647)
Try with https://aistudio.google.com Think the page limit is a vertex thing The only limit in reality is the number of input tokens taken to parse the pdf. If those tokens + tokens for the rest of your prompt are under the context window limit, you're good.