Re:How We Built the ARKit Sudoku Solver
by Cthulhu_ on Tuesday October 10, 2017 @08:40AM
(#15441253)
You'd think recognising a square (containing squares and more squares if need be) would be relatively simple and not require advanced machine learning / training. Or that recognising a square doesn't take as much. The demo also indicates the sudoku needs to be fairly accurately scanned, similar to a QR code.
[ Parent
| on HN ]
Re:How We Built the ARKit Sudoku Solver
by dr_zoidberg on Tuesday October 10, 2017 @10:25AM
(#15442003)
I think he meant for character recognition -- it's explained in the post that they went for an in-house dataset (from sudoku magazines I understand) instead of MNIST. They ran into some issues, found the way to solve them, and improved their training set. This allowed them to reach 98.6% accuracy, and after a few updates to the app over 99%.
Re:How We Built the ARKit Sudoku Solver
by zimpenfish on Tuesday October 10, 2017 @08:47AM
(#15441294)
I don't think the square recognition used any machine learning.
> We use iOS11’s Vision Library to detect rectangles in the image.
Looking at https://github.com/gunapandianraj/iOS11-VisionFrameWork - this definitely doesn't touch CoreML
It's unclear whether Vision uses machine learning behind the scenes though. It's kind of implied in their docs that it uses CoreML behind the scenes (which makes sense with the other things it does like Face recognition and object tracking).
The nice thing is it detects "projected rectangular regions" so even if the puzzle isn't aligned with the camera it still works.
I do wish I had more control though; it runs into trouble sometimes and there's not much I can do other than apply heuristics afterwards to determine whether I should throw out the sample or continue.
Example of a bad read from Vision Rectangle Detection: https://imgur.com/a/RSpTG
Re:How We Built the ARKit Sudoku Solver
by zimpenfish on Friday October 13, 2017 @10:10AM
(#15465798)
> Example of a bad read from Vision Rectangle Detection: https://imgur.com/a/RSpTG
Well, it's technically correct - it did find a rectangle :)