I worked with these folks for a while. I also work with the deaf community.

Sign Language gloves are an interesting idea, but they don't work. Sign language relies heavily on facial expressions and body language beyond the hands.

This was also tried here before: https://www.huffingtonpost.com/entry/navid-azodi-and-thomas-...

But deaf people aren't actually that keen on these solutions, as I found out when I proposed this to some of them myself.

Taken to <3

I think more people should take it to heart. If you want to influence cultures for the better, you need to interact with them. It's obvious when you know it, less obvious when you don't.

For clarity, could you give an example of a sign that changes to mean a different word or phrase depending on facial expression?

> But deaf people aren't actually that keen on these solutions

People often aren't keen on solutions that are either still nascent or that they haven't given a chance yet. My mom wasn't keen on navigating with a GPS until she had one in her car and it changed her life. Not being keen is not on its own a great reason to dismiss a technology. The reason matters.

IMO a more meaningful statement might be to say that the obvious alternative is simply a small keyboard, which already exists.

I think technology might someday be able to understand ASL and other signed languages (every region/area has their own signed language, generally). But it will be long after computers can easily understand spoken/written languages, which they can only sort of do now and even then very rudimentary currently.

Languages are very complex, and signed languages are no different, but add extra complexity, because you have to also handle facial expressions and hand/arm AND body movements (the whole body moves in ASL to delineate different contexts as another example.)

I for instance just carry around an iPad with a text app (like Sorenson's Buzzcards) which make communicating with hearing people way easier, and save paper.

Thank you for the clear example. I think this context really helps people understand the conversation better.

So, cool, looks like fun to hack on, almost certainly not newsworthy.

As you can imagine, this external savior thing can be pretty frustrating to the people who actually live their lives with a particular impairment.

Is this something that is possible with arduino or raspberry pi? Or is there something else I should look into?

They decided not to pay for captioning, image enhancement or audio descriptions for their lectures for old content, even though the justice department letter said:

"UC Berkeley is not, however, required to take any action that it can demonstrate would result in a fundamental alteration in the nature of its service, program or activity or in undue financial and administrative burdens".

They could arguably have fulfilled their requirements by working to update material based on actual demand, and having a process to request updating such material. Court ordered consent degrees are negotiated and approved by a judge, and they could have attempted to negotiate something to address the complaint.

UC Berkeley is a public institution, with expenses paid for in large part by federal and state funds, and those funds come with requirements not to discriminate on the basis of disability, and instead of actually addressing that discrimination in a meaningful way for their content, they chose to simply stop providing the service. You can have your opinion on if that's the right decision, but they weren't forced to make one decision or another.

And per UC Berkeley, it wasn't just the Justice department letter, but also online "limited use" and to better protect instructor intellectual property from 'pirates'". https://news.berkeley.edu/2017/03/01/course-capture/

There have been devices like this since the 80s, because it basically does hand shapes, not signs (which have movement, grammar, spatial references, body placement references and other aspects that a hand and wrist device won't capture).

This looks suitable for fingerspelling only (Think, instead of words, spelling out everything you want to say using the letters of the alphabet, and you'll get why this isn't a full solution, though you should understand that sign languages are discrete languages with their own grammar and syntax not just english words as signs, so... I guess imagine spelling out what you want to say but it's transliterated french and using latin declensions or something).

Better then nothing, sure, but hardly translating sign language.

Can you do it with a camera and machine learning? I very much doubt it. I am not sure current gesture segmentation methods could pick out discrete signs or even hand shapes in different orientations, given that there isn't a neutral state one returns between "words".

Then you need to do gesture classification, differentiating based on handshapes (which can transform during a sign, with the nature of the transition, as well as it's speed, the size of the over all gesture, incorporated facial language etc is meaningful, and can be modified by previous or future signs, spatial designations etc), and THEN you need to do stateful real time machine language translation from the decoded signs into English (or Swahili possibly), because a nations sign language isn't english (or whatever your local language is) with the words signed, it's it's own thing, and there may not be a one to one correspondence between words.

In ASL (and I can't speak to other sign languages), you often put the subject of the sentence first (IE instead of saying, "who's the new guy in the purple tie?" the grammar is more like "GUY THERE, WEARING TIE PURPLE, NEW, WHO?") And that's something simple, where gender is included explicitly, no past or future tenses etc. Of course there's also code switching, and formalized systems for conveying English like Signing Exact English.

Oh, did I mention that no sign language has a written form? Linguists and others can create code designations based on what they see as sign word correspondences, but the same sign can be modified, such that the sign you use for a pretty face can be modified to indicate degree of beauty, so the sign for gorgeous and the sign for pretty need to be differentiated, by software.

Oh and if you got it working for one country... you got it working for one country. British sign language and american sign language are radically different, and basically every country has it's own sign language, though they may have some shared roots. Even in the US, because of segregated schools for the deaf, there's a black ASL dialect.

Anyway, we might have the technology at this point to do sign language translation, but probably not, but even if we did, the funding to get it from working on set sign language phrases in an MIT or Carnegie Melon Lab to something that can actually translate sign language isn't there.

If someone wanted to start camera classification for gestures, I'd start with recognizing flipping the bird fast enough to blur it in a live video feed, because people might actually pay for that.