jbarrow

This is my post! Happy to answer any follow-up questions people have.

I wrote this to try and clear up what I think is a common misconception (speculative decoding is a free lunch), when the picture is more nuanced (and imo, interesting).

I'm always glad to see more multi-page work in VLM-based OCR. Especially single-pass. One of the few other multi-page papers from recently, MinerU-Popo, treats fixing up multi-page outputs as a post-processing correction step (https://arxiv.org/abs/2605.24973). Interesting to see the drop-off in quality as you up page count, though.

I also think the attention approach (always attend to the image/prefix, with a sliding window for local context) is neat!

I do wish they updated their comparison table to include more recent work (that scores marginally better on OmniDocBench), like dots.mocr.

Very impressed with how much the Gemma ecosystem has advanced just this week.

Gemma 12B, multitoken prediction, and official quants released. Feels like Google is putting real effort into this string of releases, and I'm very excited to see that!

Very cool to see a company pushing what's possible with (relatively) tiny models! A 350M parameter trained on 28T tokens that, from the benchmarks, is competitive with Qwen3.5-0.8B.

Comparing the architecture to Qwen3.5, it seems:

- fewer, wider layers

- mixing full attention and conv's, instead of the full+linear attention of Qwen3.5

- the vocab is about 1/4 the size

Shared this because I was having fun thinking through floating point numbers the other day.

I worked through what fp6 (e3m2) would look like, doing manual additions and multiplications, showing cases where the operations are non-associative, etc. and then I wanted something more rigorous to read.

For anyone interested in floating point numbers, I highly recommend working through fp6 as an activity! Felt like I truly came away with a much deeper understanding of floats. Anything less than fp6 felt too simple/constrained, and anything more than fp6 felt like too much to write out by hand. For fp6 you can enumerate all 64 possible values on a small sheet of paper.

For anyone not (yet) interested in floating point numbers, I’d still recommend giving it a shot.

I've been noticing a _lot_ more AI-generated/edited content of late, both comments and stories. It's gotten to the point that I spend a lot less time on HN than I used to, and if it continues to get worse I expect I'll quit altogether.

At the end of the day, I'm here because of all the thoughtful commenters and people sharing interesting stories.

Watsi is incredibly inspiring!

I’ve been a monthly donor since ~the beginning when I was just an undergraduate, and I still read the stories and emails I receive. I’m glad that you opted for the steady growth path, and that you’ve made it a sustainable thing.

The whole thing feels AI written, generated from the codebase.*

*this is incorrect per the author’s response, my apologies.

For instance, it goes into (nano)vLLM internals and doesn’t mention PagedAttention once (one of the core ideas that vLLM is based on)[1].

Also mentions that Part 2 will cover dense vs MoE’s, which is weird because nanovllm hardcodes a dense Qwen3 into the source.

Here are better (imo) explainers about how vLLM works:

- https://hamzaelshafie.bearblog.dev/paged-attention-from-firs...

- https://www.aleksagordic.com/blog/vllm

- https://huggingface.co/blog/continuous_batching

Aleksa’s blog is a bit in the weeds for my taste but it’s really worth working through.

A lot of the magic of vLLM happens in the PagedAttention kernels, which are really succinctly implanted in nanovllm. And the codebase is great and readable by itself!

—

1. https://arxiv.org/abs/2309.06180

Super interesting. Would you be willing to try the Python package (https://github.com/jbarrow/commonforms) or share the PDFs?

For the non-ONNX models there are some inference tricks that generally improve performance, and potentially lowering confidence could help.

Hey, Benjamin, thanks for the attribution! Happy to field any questions HN users have.

It's really gratifying to see people building on the work, and I love that it's possible to do browser-side/on-device.

Woah, did not realize that, haha. Let me know if it works well!

Training ML models for PDF forms. You can try out what I’ve got so far with this service that automatically detects where fields should go and makes PDFs fillable: https://detect.semanticdocs.org/ Code and models are at: https://github.com/jbarrow/commonforms

That’s built on a dataset and paper I wrote called CommonForms, where I scraped CommonCrawl for hundreds of thousands of fillable form pages and used that as a training set:

https://arxiv.org/abs/2509.16506

Next step is training and releasing some DETRs, which I think will drive quality even higher. But the ultimate end goal is working on automatic form accessibility.

Existing “auto-fillable” tools are pretty lackluster in my experience. CommonForms is tooling that can automatically detect form fields in PDFs and turn those PDFs into fillable documents. The dataset is ~500k form pages pulled from Common Crawl, which I trained the object detectors on. For being vision only, the results are pretty remarkable!

Releasing the dataset, paper, models, and (imo most importantly) simple/convenient tooling to automatically prepare any PDF.

Links: Repo: https://github.com/jbarrow/commonforms - Paper: https://arxiv.org/abs/2509.16506

I’m personally a huge fan of Modal, and have been using their serverless scale-to-zero GPUs for a while. We’ve seen some nice cost reductions from using them, while also being able to scale WAY UP when needed. All with minimal development effort.

Interesting to see a big provider entering this space. Originally swapped to Modal because big providers weren’t offering this (e.g. AWS lambdas can’t run on GPU instances). Assuming all providers are going to start moving towards offering this?

If you enjoyed this essay, you should check out the author’s current project, Dynamicland[1]. It is a wonderful expression of what computing and interaction could be. Even the project website — navigating a physical shelf, and every part is hyperlinked — is joyful.

1. https://dynamicland.org/

Editing text in PDFs is _really_ hard compared to other document formats because most PDFs don't really encode the "physics" of the document. I.e. there isn't a notion of a "text block with word wrapping," it's more "glyphs inserted at location X with font Y."

If the PDF hasn't been made accessible, you have to do a lot of inferencing based on the layout about how things are grouped and how they should flow if you want to be able to make meaningful edits. Not impossible (Acrobat does it), but very challenging.

It's part of the legacy of PDF as a format for presentation and print jobs, rather than typesetting.

Wonderful! Inserted form-fields show up in Preview and Acrobat, which is not a trivial task. I run a little AI-powered tool that automatically figures out where form fields should go (https://detect.penpusher.app) and robustly adding form fields to the PDF was the hardest part.

Fwiw, I do see the issue with being unable to scroll down across both Safari and Chrome.

I'm sorry I missed this earlier, but I absolutely believe that it could do that. Do you have any pointers to PDF forms that work well or don't work well with screen readers? I'd be happy to take a look, and see if I can improve this tool based on that.

In addition, did you try the "enhanced" pipeline? It gives each field a meaningful name based on the label, which might help with accessibility.

PDF accessibility is a huge issue that _should_ be easily solved, but isn't, unfortunately.

> Unfortunately Gemini really seems to struggle on this, and no matter how we tried prompting it, it would generate wildly inaccurate bounding boxes

Qwen2.5 VL was trained on a special HTML format for doing OCR with bounding boxes. [1] The resulting boxes aren't quite as accurate as something like Textract/Surya, but I've found they're much more accurate than Gemini or any other LLM.

[1] https://qwenlm.github.io/blog/qwen2.5-vl/

Mostly because OpenAI's vision offerings aren't particularly compelling:

- 4o can't really do localization, and ime is worse than Gemini 2.0 and Qwen2.5 at document tasks

- 4o mini isn't cheaper than 4o for images because it uses a lot of tokens per image compared to 4o (~5600/tile vs 170/tile, where each tile is 512x512)

- o1 has support for vision but is wildly expensive and slow

- o3-mini doesn't yet have support for vision, and o1-mini never did

I've been very impressed by Gemini 2.0 Flash for multimodal tasks, including object detection and localization[1], plus document tasks. But the 15 requests per minute limit was a severe limiter while it was experimental. I'm really excited to be able to actually _do_ things with the model.

In my experience, I'd reach for Gemini 2.0 Flash over 4o in a lot of multimodal/document use cases. Especially given the differences in price ($0.10/million input and $0.40/million output versus $2.50/million input and $10.00/million output).

That being said, Qwen2.5 VL 72B and 7B seem even better at document image tasks and localization.

[1] https://notes.penpusher.app/Misc/Google+Gemini+101+-+Object+...

Loved the one in Kansas City! There are some great, thematically-similar museums in other countries as well, if you ever find yourself there:

- the Vasa in Stockholm, Sweden is a ship dredged from the harbor and stabilized, sank in 1628

- the Mary Rose in Portsmouth, England is a Tudor ship that sank in 1545 that was raised and stabilized

In both cases a ton of work was done to stabilize and preserve the remains of the ships that is, imo, almost more interesting than the ship itself.