10,000 data points, modeling an extremely complex system, and 90% "success" rate?

The word "overfitting" comes to mind.

Assuming that VocalIQ knows what they're doing - and that's a fairly safe bet, I think - that's not the case. So they've either acquired a massively larger data set, reduced the complexity of the system, or redefined "success". My bet would be on the second one.

They've been in the news before, for working on voice IO for cars. That'd make much more sense than a generic system that massively outperforms all others.

Summarize:

- Acquihired VocalIQ

- Unreleased product

- Unnamed sources

- Uses machine learning via 10,000 data points from the Mechanical Turk of people asking questions to beat Google, Microsoft, Amazon and Apple.

I want to believe.

'Remember context forever' - that's serious privacy issue - isn't it

Depends on how it's implemented. As with most things.

>> VocalIQ’s success rate was over 90%, while Google Now, Siri, and Cortana were only successful about 20% of the time, according to one source.

That could be very impressive except we have no idea what VocalIQ was successful at, 90% of the time (if indeed it was 90% of the time and not something completely irrelevant).

Well the query was "find a nearby Chinese restaurant with open parking and WiFi that’s kid-friendly.” So I suppose it's successful at doing that.

If that's all it does, then it's not very impressive. You can trivially create a program that finds nearby Chinese restaurants with open parking and WiFi that's kid friendly, by creating a table of all such establishments and their "nearby" locations.

Presumably, this was supposed to prove something else- what else? How did they measure that "90% success rate"?

Just throwing percentiles in the air like that says nothing and makes the whole thing appear not a little dodgy.

"Apple is working on an AI system that wipes the floor with Google and everyone else"

Right. Google Now and Siri aren't even in the same realm right now and Google has access to massively more data than Apple. I'm not holding my breath.

Well in the article it stated that "Siri brings in 1 billion queries per week from users" so that's nothing to sneeze at. But the real challenge is the majority of those queries are probably pretty simple queries that Siri doesn't/can't learn from - she already answers them fine. You need alternative data sets (such as the one generated by mechanical turk) to really learn anything new.

But datasets are only 1 part of the problem - innovate, sophisticated and tuned approaches to solving these problems often make the difference.

Google doesn't have massively more data than Apple.

Apple has been crawling the web for a few years now to power their suggested sites feature on Safari. And they also have access to Maps, Siri searches and all the data sources they linked to power it.

It's an advantage but not a massive nor insurmountable one.

And Google has literally all of that, plus all of Google Search plus all of Hangouts plus all of Gmail plus all of Google Voice (with voicemail).

I still think it's safe to imply that Google has a massive leg up on the data-side when compared to Apple, not to mention the today-comparison of Now vs Siri.

The reason VocalIQ's success rate was "over 90%" is probably because they were constrained to a specific domain (car control). This is very common approach among NLP community. Of course you will perform better when your algorithm is constrained to a specific domain.

All the previous speech recognition platforms like Siri, Cortana, etc. perform worse since they are for general purposes. On the other hand, the reason Amazon Echo works so well is because it has very limited purpose.

We all know just because AlphaGo beat Lee Sedol doesn't mean AlphaGo is smart enough to take over the world.

Spinning this "90%" vs. "20%" just sounds like a bullshit marketing.

Has anyone tried Hound? How does it compare with Echo/Google Now?

https://www.youtube.com/watch?v=-2RK3z6aiQc https://www.youtube.com/channel/UCXVfDSAcpnhMCYGPuFGFIMw

Google, commanding the largest voice and spoken question dataset, probably the largest knowledgegraph in the world and for all intents and purposes infinite resources is to be beat by a small Cambridge Startup at the game they have basically tried to play since they've been founded?

http://i.imgur.com/dTmqF1i.jpg

...yeah, by closing down the Freebase website and its community. Now they use it inhouse.

AI is just as much about the algorithm and tooling as it is the training data. Especially in these early days where there is still plenty of innovation yet to come.

Also stupid memes are better suited for Reddit not here.

It seems rather odd not to mention Viv at all here.

I wouldn't call Steve Kovach as credible journalist in tech. His articles in businessinsider just prove the point.

It's fine to disagree, but if you're going to post here you must do so civilly.

We detached this comment from https://news.ycombinator.com/item?id=11789834 and marked it off-topic.