The table of costs in the blog post. At 500,000 pages per day the hardware fixed cost overcomes the software variable cost at day 240 and from then on you're paying an extra ~$100 per day to keep it running in the cloud. The machine also had to use extremely beefy GPUs to fit all the models it needed to. Compute utilization was between 5 to 10% which means that it's future proof for the next 5 years at the rate at which the data source was growing.
| Model | Pages per Dollar |
|-----------------------------+------------------|
| Gemini 2.0 Flash | ≈ 6,000 |
| Gemini 2.0 Flash Lite | ≈ 12,000* |
| Gemini 1.5 Flash | ≈ 10,000 |
| AWS Textract | ≈ 1,000 |
| Gemini 1.5 Pro | ≈ 700 |
| OpenAI 4o-mini | ≈ 450 |
| LlamaParse | ≈ 300 |
| OpenAI 4o | ≈ 200 |
| Anthropic claude-3-5-sonnet | ≈ 100 |
| Reducto | ≈ 100 |
| Chunkr | ≈ 100 |
>The longest part of the development timeline was establishing the accuracy rate and the ingestion pipeline, which itself is massively less complex than what your workflow sounds like: PDF -> Storage Bucket -> Gemini -> JSON response -> Database
Each company should build tools which match the skill level of their developers. If you're not comfortable training models locally with all that entails off the shelf solutions allow companies to punch way above their weight class in their industry.
However we do very much recommend storing the raw model responses for audit and then at least as vector embeddings to orient the expectation that the data will need to be utilized for vector search and RAG. Kind of like "while we're here why don't we do what you're going to want to do at some point, even if it's not your use-case now..."
wow, this is so bad. why do it now and introduce complexity and debt if you can do it later when you actually need it? you are just riding the hype wave and trying to get most out of it but that's fine.
The same reason I don't wait until it snows to buy snowboots. I know my environment, topography, scale, risk-profile, and costs, and can concieve of innumerable use-cases for when they will be necessary, even if it's only May, when snowboots happen to be on sale ;) What's a little closet space and the burden of locking my door when I leave the house in the interim?
Was this human-verified in some way? If not, how did you establish the facts-on-the-ground about accuracy?
The longest part of the development timeline was establishing the accuracy rate and the ingestion pipeline, which itself is massively less complex than what your workflow sounds like: PDF -> Storage Bucket -> Gemini -> JSON response -> Database
Just to get sick with it we actually added some recusion to the Gemini step to have it rate how well it extracted, and if it was below a certain rating to rewrite its own instructions on how to extract the information and then feed it back into itself. We didn't see any improvement in accuracy, but it was still fun to do.