I have been meaning to build something like this for myself, albeit for Linux. Does anybody know if there is any already existing efforts there?

Given that Linux doesn't have anything like an accessibility API, I think the only option is training ML models.