A thread on the deterioration of accessibility in GTK4 and Wayland

Re:A thread on the deterioration of accessibility in GTK4 an... by suby on Friday June 09, 2023 @07:59PM (#36266001)

I believe we need to bypass all of these bespoke accessibility solutions and use machine learning to visually parse a live feed of your computer screen. We should be able to take a given screenshot and determine where the buttons are, what the text is, and make something that is decently workable. Then we won't need to care about each app, each website, each operating system supporting accessibility standards because our accessibility tools will be using the computer in the same manner that humans do, visually decoding a live feed of images.

Easier said than done, but given what is happening in AI right now, it doesn't seem far fetched to think that we'll be able to accomplish this.

[ Parent | on HN ]

Re:A thread on the deterioration of accessibility in GTK4 an... by paulryanrogers on Friday June 09, 2023 @11:40PM (#36267365)

I was hoping to do something like this for adventure game hotspots after a hand injury. Even experimented with AutoIt and its OCR, before giving up.

Now there is another niche project with the goal of translation.

With more capable models it's going to be fun to see what folks come up with!

[ Parent | on HN ]

Re:A thread on the deterioration of accessibility in GTK4 an... by kaba0 on Saturday June 10, 2023 @07:24AM (#36269577)

It doesn’t sound unachievably hard, but I cry for all the “wasted” energy it would require.

[ Parent | on HN ]