Perhaps not the hardest, but one where there's tons of room for improvement: the Story Cloze Test [1] is a test involving very simple, five-sentence stories, where you pick the ending that makes sense out of two endings.
A literate human scores 100% on this test. No computer system so far scores better than 60%. (And remember that random guessing gets 50%.)
Re:Quick, Draw
by johnlbevan2 on Thursday November 17, 2016 @03:53AM
(#12975639)
Interesting study; whilst it's possible to guess which ending is expected as correct, the alternate could be easily argued. For example, in the case of Jim's getting a new credit card, I recall during my uni days many students took that exact approach to debt...
General purpose machine translation is harder, for instance. Brute force algorithms have gotten decent, but aren't in the same ballpark as humans (though professional translation services now often work by correcting a machine translation). However, MT systems trained on a specific domain do much better (medical or legal docs, etc).
What would be the hardest task for machines that's trivial for humans? Maybe deciding if a joke is funny or not?