Measuring the Progress of AI Research

Re:Measuring the Progress of AI Research by IshKebab on Monday July 03, 2017 @05:47AM (#14687074)

There's no way we have reached human-level performance on image recognition tasks. My guess is that when there is ambiguity (e.g. 'ship' vs 'boat') the AI is better at learning the answer the labellers chose. Humans haven't looked through the training data so they use their real-life biases which may not match those of the labellers.

Just a guess, but whether that is true or not we're definitely not at human-level performance.

[ Parent | on HN ]

Re:Measuring the Progress of AI Research by IanCal on Monday July 03, 2017 @07:01AM (#14687354)

I think we're past human performance on imagenet, but one of the reasons for that might be categories like specific species of dog.

If I remember right, face recognition was passed a long time ago.

[ Parent | on HN ]

Re:Measuring the Progress of AI Research by joe_the_user on Monday July 03, 2017 @03:25PM (#14690709)

Well, the challenge is developing more fine-tuned tests than simple labeling that humans perform better at.

Things: labeling and segmenting, labeling according natural language instruction or whatever.

I mean, AI naturally have made more progress in situations where progress can be exactly quantified but the activity of creating tests is only somewhat separated from the activity of creating AIs. Most tests in different directions could spur more research in those directions.

[ Parent | on HN ]