News for hackers. Stuff that matters.
Re:Writing a fuzzy receipt parser in Python by whosbein on Tuesday October 06, 2015 @09:19AM (#10338630)
This is really cool, nice work! I tinkered with the same thing a while ago. Sadly, that project has been neglected. One thing I was most proud of was developing a way to automatically rotate the receipt image to the most ideal position to better help the OCR. I would notice decent improvements even with a few degrees difference.

Question: have you thought about any better methods for getting the receipts scanned? A big hurdle for me was the time it took to scan everything. I know there are scanners designed for such a thing, and you seem to have used one, but were you satisfied with it?

And thanks for the get_close_matches() hint! I'll try and remember that if I ever start hacking again on this.

Re:Writing a fuzzy receipt parser in Python by omn1 on Tuesday October 06, 2015 @09:30AM (#10338700)
Yeah scanning the receipts takes the longest time in the whole process. So you should make sure that you have a fast and simple scanner. I can recommend to buy a simple photo scanner like I did. It's just a couple bucks and it will save you so much time. And the quality was near perfect. Even very old receipts could be parsed after scanning with it.

When it comes to rotation, you can try http://www.fmwconcepts.com/imagemagick/textcleaner/. It will rotate your scan if it's not perfectly aligned.

Re:Writing a fuzzy receipt parser in Python by whosbein on Tuesday October 06, 2015 @09:36AM (#10338743)
Ha, cool, didn't know about textcleaner either. Always fun when you write some huge routine to do something that you later find out is a simple command elsewhere. Thanks!
Re:Writing a fuzzy receipt parser in Python by zo1 on Tuesday October 06, 2015 @10:48AM (#10339229)
Just a quick FYI. I looked at "Fred's Scripts" a while back for a commercial project I was working on. Have a look at the license, if you ever plan on any commercial work using those scripts.
Re:Writing a fuzzy receipt parser in Python by aidos on Tuesday October 06, 2015 @10:55AM (#10339269)
get_close_matches is such a godsend. I've already used it this morning to deal with matching up 2 lists with about 100 things in each.

One trick I've found is to use it in both directions and then only using results that agree both ways.

Something like this (very terse version):

    closest_or_none = lambda l, options: (difflib.get_close_matches(l, options, n=1) or [None])[0]
    dir1 = [(l, closest_or_none(l, list2)) for l in list1]
    dir2 = [(closest_or_none(l, list1), l) for l in list2]
    one_to_one = set(dir1).intersection(set(dir2))
Interested to hear how you approached the rotation thing.
Re:Writing a fuzzy receipt parser in Python by whosbein on Tuesday October 06, 2015 @01:37PM (#10340720)
Rough summary: I generated best fit lines from the alignment of the letters. You know the lines on the receipt consist of straight-aligned letters, so you can calculate lines based on that. I then calculated the average slope of all those lines and then used that average to base the rotation calculation on.