Sign language is at least three-dimensional, it can embed more information. The ASL signs for "forget" and "because" differ only in how you mouth them - easy enough, but sometimes you even skip mouthing them because there is sufficient context. What's worse is the location of the hands can vary depending on how lazy the signer is feeling, so "forget" is supposed to be signed from the forehead but it can also be signed from some empty point in front of your shoulder, off a "phantom person"'s imaginary forehead. You rely on the listener to understand that you intend for there to be a phantom person there.

Parsing sign language means you have to turn a 2D recording into a 3D model - even humans can have difficulty with this, parsing signs over Zoom - and you need to solve parsing facial expressions, reading lips and more. And you need the machine to be smart enough to detect the assignment of phantom persons to various locations around the signer. That's my impression as a deaf person anyway.

cough.

Take classifiers: there are eight morphological types that allow users to express complex ideas without the use of any type of vocabulary-based utterances. You can certainly sign the equivalent of, “Did you see the car parked close to the tree,” (CAR PARK NEAR TREE SEE?) using vocabulary. Sure. It gets the point across, but it’s not efficient, and doesn’t really utilize sign language in a way that feels normal for users.

But that same question can be posed in a more condensed format if you use a classifier, (#CAR 5:CL-“tree”/3:CL-“car parked really close” SEE?). In the second utterance, one use the lexicalized (fingerspelled “car”), then holds up a five hand with bent elbow in their non-dominant hand, use the three hand representing the car placing it reeeeeeally close (with grammatical, NMC emphasis) to the tree, and finally asking the receiver if they saw it.

In that example alone, I’m using two types of classifiers: element and locative. Imagine a complex story using all eight classifier types with just a little vocabulary mixed in? I doubt AI/ML could interpret the context, characters, actions, events, descriptions, etc. accurately. Anyway, I hope this example shows there is a lot more complexity in computerized interpretation of signed languages well beyond gesture/vocabulary recognition.

Note: My glossing may not be perfect. It's been a while, and I’m a bit rusty.

What would be a good source to get the full list of the eight morphological types, and how they are used?

https://www.allenisd.org/cms/lib/TX01001197/Centricity/Domai...

The linked PowerPoint does a good job of explaining a brief overview of the different types of classifiers. (I actually referenced it to correct my own answer because I knew something felt off. lol)

> Research led by a team at Microsoft, a big computing firm, and published in 2019, estimated that a typical publicly available corpus of a spoken language consists of around a billion words from as many as 1,000 different speakers. An equivalent data-set in a sign language might have fewer than 100,000 signs from just ten people. Besides large numbers, a good corpus also needs variety. This means conversations between native signers of diverse backgrounds, dialects and levels of fluency. Because deaf people more often have physical disabilities than do those with unaffected hearing, representing those with restricted fluency of movement is important.

An example sentence of mostly polysynthetic signs:

"i was out driving, just enjoying the scenery, the nature and all the trees passing by me," (3 signs) "then suddenly" (just raise the eyebrows and look scared at some point in front of you, with your hands still on the imaginary steering wheel), "my car crashed into a wall (or other immobile object) (and the hood got all deformed)" - 1 sign.

So that takes about 5 seconds to sign, but up to a minute to represent all the embedded information in English. But in practice, you'd skip a lot of the embedded information when translating to English, depending on what was actually important. So how do you make the machine know what was important? It depends on who you're translating for!