It looks like he’s first using tesseract to recognize his handwriting and convert it into text. Tesseract doesn’t do a perfect job so the recognized text is full of mistakes. He treats the mistakes as spelling mistakes and “asks” GPT-3 to correct them. This is a very clever idea and will greatly improve current OCR efforts.

That's a really interesting part, and probably why the OCR works good enough for such case.

Not sure about legal implications of using it though:

https://www.technologyreview.com/2020/09/23/1008729/openai-i...

EDIT: it seems I have misunderstood the article - OP probably uses MS API to access GPT-3 anyway, so the point is moot.

They might consider it a trade secret of sorts. If I were them, I wouldn't want someone to just take the idea and undercut me.

I think from his explanation in the article it's quite straightforward to implement it yourself: get a GPT-3 subscription at OpenAI or MS Azure, use the API as described in the article, voilà.

But the idea is genius indeed.

From the screenshot I suspect the real secret is that it gives the user a chance to correct errors after scanning.