News for hackers. Stuff that matters.
Re:Someone has to say it: Voice assistants are not doing it ... by _dain_ on Wednesday November 23, 2022 @05:31AM (#33717431)
>Voice assistants are basically just mainstream non-visual command-lines, and it's unsurprising to me that something that relies heavily on memorization and extremely specialized "skills" isn't quite taking off in the way it was imagined.

This got me thinking. Voice recognition is basically a commodity now .. there are open source AI engines that can do it offline really well. So the recognition part is solved, you can just grab it from your distro's package manager. Now there's just the language part.

Thing is, I don't want to speak to my computer using English. Aside from the enormous practical problems in natural language processing you've outlined, I just find the idea creepy[1].

What I want is to unambiguously tell it to do arbitrary things. I.e. use it as an actual computer, not a toy that can do a few tricks. I.e. actually program it. In some kind of Turing complete shell language that is optimized for being spoken aloud. You would speak words into the open source voice recognizer, it writes those to stdout, then an interpreter reads from stdin and executes the instructions.

Is there any language like this? What should it look like?

And yeah that would take effort to learn to use it right, just like any other programming language; so be it. This would be a hobbyist thing.

[1] https://i.kym-cdn.com/photos/images/original/002/054/961/748...

Re:Someone has to say it: Voice assistants are not doing it ... by viraptor on Wednesday November 23, 2022 @05:37AM (#33717478)
> So the recognition part is solved

If you're using an averaged American voice - maybe. But it's really not solved for everyone. Google assistant can't set the right timer for me 1/10 times. And that's before we get to heavy accent Scots and others.

Re:Someone has to say it: Voice assistants are not doing it ... by bambax on Wednesday November 23, 2022 @05:47AM (#33717564)
Re:Someone has to say it: Voice assistants are not doing it ... by arethuza on Wednesday November 23, 2022 @06:04AM (#33717703)
Even my "affected Edinburgh accent", as someone once described it, causes no end of trouble with voice recognition.
Re:Someone has to say it: Voice assistants are not doing it ... by draugadrotten on Wednesday November 23, 2022 @06:10AM (#33717743)
> And yeah that would take effort to learn to use it right, just like any other programming language; so be it. This would be a hobbyist thing.

There are quite a few hobbyists working on local on-prem privacy focused voice assistans with conversation support.

https://www.home-assistant.io/integrations/#voice https://www.home-assistant.io/integrations/conversation/

Have fun. It is a rabbit hole.

Re:Someone has to say it: Voice assistants are not doing it ... by Shank on Wednesday November 23, 2022 @06:15AM (#33717779)
> Voice recognition is basically a commodity now .. there are open source AI engines that can do it offline really well. So the recognition part is solved, you can just grab it from your distro's package manager.

I personally don't consider this a fully-solved problem. The best transcription system I've used is OpenAI Whisper, and it doesn't work in realtime. Maybe it's fine on small amounts but it's still not perfect. You really need error to be driven down dramatically. Zoom auto-captions are a joke in terms of how badly they work for me, and Live Text (beta) on macOS is equally dreadful. YouTube auto-captions suck. All of these use industry-leading APIs. If I'm speaking a voice command and one single word is wrong, usually the whole thing fails.

There's an entirely separate issue about things that are Proper Nouns that don't exist. For example, "Todoist" is often misunderstood by Siri. Thus, people started saying "Two doist (where doist rhymes with joist)" to fool it into understanding "Todoist". Media like anime with strange titles from other languages often flat out trolls these transcription systems. ("Hey Siri, remind me to watch Kimetsu no Yaiba tomorrow".)

Re:Someone has to say it: Voice assistants are not doing it ... by Aramgutang on Wednesday November 23, 2022 @06:23AM (#33717839)
That reminds me of the handwriting recognition approach [1] used in old Palm Pilot devices. Even though the shapes it expected you to draw resembled the corresponding letters, you would never draw them like that if you were writing on paper.

You knew that you were drawing something designed for a computer to recognise as unambiguously as possible, while being efficient to draw quickly and easy to learn for you. I feel like that's the kind of notion that voice interfaces should somehow expand upon.

[1] https://en.wikipedia.org/wiki/Graffiti_(Palm_OS)

Re:Someone has to say it: Voice assistants are not doing it ... by simiones on Wednesday November 23, 2022 @09:08AM (#33719244)
> Voice recognition is basically a commodity now .. there are open source AI engines that can do it offline really well. So the recognition part is solved, you can just grab it from your distro's package manager.

This is potentially far from true, depending on how exactly you draw the line between "voice recognition" and "language". I've looked at quite a few transcription services, and they fail a lot of the time for most people - those who either have a non-native accent (even if very slight!) or those who do any amount of stammering or other vocal tics.

Re:Someone has to say it: Voice assistants are not doing it ... by ghaff on Wednesday November 23, 2022 @10:23AM (#33720287)
I find the ML transcription services, given 2 people speaking English with high quality sound and without heavy accents/a lot of jargon, to be adequate for having a skimmable record--such as for extracting quotations. But if I'm publishing a transcript I get a human transcription. Cleaning up the ML stuff takes way too much time and I wouldn't publish a transcript without cleaning it up.
Re:Someone has to say it: Voice assistants are not doing it ... by simiones on Wednesday November 23, 2022 @10:50AM (#33720671)
I was in fact looking at some transcriptions of my recent meetings, and found one that captures how even small mistakes can make for completely not-understandable transcripts, unless they are manually cleaned up.

Manual transcription:

> So no: long story short, Slum is basically the way we can have an individual [, uhhh,] instance that carries all the licenses.

(Slum is a project name in this case)

Computer transcription (MS Teams):

> So no.

> A long story shorts. Love is basically the way we can have an individual.

> OHS instance that carries all the license.