News for hackers. Stuff that matters.
Re:How Chrome Accessibility Works by vhiremath4 on Tuesday August 27, 2024 @05:38PM (#41373382)
It seems like small multimodal LLMs have a killer use case to be bundled with browsers for accessibility. Eventually:

* if an image doesn’t have alt text

* you need to be read the page

* you need to be described what’s happening in a video

A model built into the OS or browser seems like a no-brainer.

Re:How Chrome Accessibility Works by dmazzoni on Tuesday August 27, 2024 @07:29PM (#41374393)
Chrome already has optional built-in support for generating alt text for images. It's been there for years, using a server-based API.

It does seem possible that this could be replaced with a local model in the near future. It's not clear the average user has the hardware specs for this to be an option today, but it will increasingly be plausible.

Keep in mind, though, that alt text is just one small part of making a web site accessible.

Re:How Chrome Accessibility Works by RandallBrown on Wednesday August 28, 2024 @12:26AM (#41375946)
> It does seem possible that this could be replaced with a local model in the near future. It's not clear the average user has the hardware specs for this to be an option today, but it will increasingly be plausible.

Siri does something like this when reading messages into your AirPods. It will give brief descriptions of photos sent in the message. I'm pretty sure it's all run locally.

Re:How Chrome Accessibility Works by freeone3000 on Wednesday August 28, 2024 @12:40AM (#41376011)
Siri has the advantage of running on either an iPhone or a Macbook. Chrome has to run on budget android phones and chromebooks.
Re:How Chrome Accessibility Works by bbb651 on Tuesday August 27, 2024 @09:03PM (#41375008)
Re:How Chrome Accessibility Works by qingcharles on Tuesday August 27, 2024 @05:54PM (#41373537)
Right now I use LLMs to generate alt text for images, and they are better than any I would have written by hand. Only in about 1% of cases do I need to correct anything.
Re:How Chrome Accessibility Works by yellow_postit on Tuesday August 27, 2024 @06:10PM (#41373704)
Re:How Chrome Accessibility Works by mananaysiempre on Wednesday August 28, 2024 @12:48PM (#41381460)
> A model built into the OS or browser seems like a no-brainer.

How about localizing it into all languages supported by a major OS or browser?

Re:How Chrome Accessibility Works by bbertucc on Wednesday August 28, 2024 @01:40PM (#41382022)
LLM-generated descriptions miss lots of context. For instance, depending on the site and content, we might mention people's races or fashion. Other times we don't.