News for hackers. Stuff that matters.
Re:Simple questions to help reduce AI hype by est on Sunday July 10, 2016 @06:14AM (#12065135)
> "How much does the accuracy fall as the input story gets longer?"

As a beginner of text classification, I find the contrary, if the sentence is very short, the result is more likely suboptimal. E.g. classification of short tweets vs a detailed NYT article.

Re:Simple questions to help reduce AI hype by Smerity on Sunday July 10, 2016 @07:03AM (#12065226)
I intended these questions as broad examples though some are specific to a given AI system or architecture. You're quite right in that it depends on the task and the model.

I was thinking specifically about neural machine translation (NMT) with the encoder-decoder architecture. The encoder-decoder architectures converts a sentence in language X into a fixed size vector representation then aims to "expand" that vector into the equivalent sentence in language Y. As the vector size is fixed, short sentences can be adequately represented but longer sentences end up being lossily compressed. This realization illustrated a fundamental limitation in the encoder-decoder architecture and motivated the use of attention mechanisms.

There's a great figure (Figure 1) in Nvidia's Introduction to Neural Machine Translation[1] that shows the dramatic drop in accuracy with respect to the sentence's length.

[1]: https://devblogs.nvidia.com/parallelforall/introduction-neur...