GitHub – rvaiya/warpd: A modal keyboard-driven virtual pointer

Re:GitHub – rvaiya/warpd: A modal keyboard-driven virtual po... by kache_ on Sunday October 16, 2022 @06:02AM (#33222488)

Have you considered using ML/OCR to figure out the position of the text relative to the screen? Seems much simpler than relying on accessibility APIs

Thank you for your hard work!

[ Parent | on HN ]

Re:GitHub – rvaiya/warpd: A modal keyboard-driven virtual po... by _chendo_ on Sunday October 16, 2022 @06:16AM (#33222550)

I have plans to use ML/OCR to augment results down the road but the AX APIs and ecosystem on most apps (that I encounter, at least) are generally decent. Also, OCR means it won’t understand buttons with just icons, whereas AX APIs can grab em just fine.

Thanks! It’s easily my longest running project at a decade

[ Parent | on HN ]