Re:The ImageNet dataset transformed AI research
by komali2 on Wednesday July 26, 2017 @02:24PM
(#14858769)
>Matthew Zeiler built Clarifai based off his 2013 ImageNet win, and is now backed by $40 million in VC funding.
I used Clarifai at a hackathon a couple months ago. It was certainly impressive, but I didn't really see the difference between it and Google Cloud Services AI whatever offerings, other than Google seemed to have a better handle on deployment and production scaling.
I guess I'm wondering how companies like Clarifai and the couple other similar "AI API cloud service" companies intend to take on behemoths like Google?
At the time, Clarifai was the "best" one (I caveat by quoting because this was a for a small corpus, with subjective results, not a real train-test cycle). I re-ran the results about a month ago (linked to from the post), and found that Google and others have continued to invest and improve.
Re:The ImageNet dataset transformed AI research
by contingencies on Wednesday July 26, 2017 @08:11PM
(#14861563)
Great overview. Clarifai is certainly extremely impressive.
Do you play the tablas? My wife and I studied sitar but our instrument was destroyed by the movers in our latest relocation to Shenzhen :( The tabla teacher where we studied was able to play a very complex taal while chewing betel nut and rolling his eyes back in their sockets, immediately switch to a pitch, bend and time-perfect rendition of 'pink panther' melody, then switch back to a very complex taal without skipping a beat. Brilliant to see.
Re:The ImageNet dataset transformed AI research
by goberoi on Thursday July 27, 2017 @01:12AM
(#14862858)
Yeah, Clarifai did well. I am keen to learn how well their custom model feature works. Per their FAQ[0], you only need supply 20-50 images per concept. That seems remarkable to me, given that a concept like 'cow' has ~1500 images on Imagenet[1]. Perhaps they are using some sort of transfer learning to facilitate this? I.e. using a pretrained model, and then only retraining the last few fully connected layers, or retraining parts of the entire network?
I am not a deep learning practitioner, but would be curious to know from experts how their custom model feature might work; and from any of their users on how well it actually does.
Tablas: haha, great description of your teacher. I do play, with enthusiasm, but poorly. For those in Seattle, there is an amazing teacher who teaches up on Cap Hill [2].
Re:The ImageNet dataset transformed AI research
by cmarschner on Thursday July 27, 2017 @03:44AM
(#14863383)
It is not necessary to train things from scratch; you take the largest imagenet model available and fine tune it for the task. This way it reuses much of the lower layers the have seen lots of data.
Re:The ImageNet dataset transformed AI research
by shostack on Thursday July 27, 2017 @12:09AM
(#14862637)
Can you share any thoughts on which would be best for a computer vision newbie and programming novice to get started playing with? Or are none of them really great for that?
Re:The ImageNet dataset transformed AI research
by goberoi on Thursday July 27, 2017 @01:04AM
(#14862832)
These aren't really great for learning about computer vision or deep learning, but are great for building projects that require image classification.
E.g., recall the awesome project for lego sorting by jacquesm[0] ? He built his own model using Keras and Tensorflow, but you may be able to achieve similar results by using Clarifai's feature to train your own models with no understanding of deep learning. This is great if your goal is to build a thing, like a lego sorter, but not so much if you want to learn how to build a state of the art image classifier.
If you're interested in learning about computer vision or deep learning, I recommend searching this site to find threads that cover that extensively. Good luck!
Re:The ImageNet dataset transformed AI research
by sjg007 on Wednesday July 26, 2017 @03:57PM
(#14859666)
This is the age of AI infrastructure.
IBM, Apple, Amazon, Facebook, Google will all buy these types of companies until they have what they need in house. Other bigger FXXX companies may buy a few as well if customer pilots go well.
Re:The ImageNet dataset transformed AI research
by tanilama on Wednesday July 26, 2017 @06:14PM
(#14860861)
I don't think they will buy too many of them though, seems like Google/Facebook are actually leading this area, and very few companies can compete with them, even research wise, let alone startups. Big companies have: data(if not they can afford massive data annotation), machines(thousands of GPUs connected), and other infrastructures. They are the Data monopolies.
Re:The ImageNet dataset transformed AI research
by skgoa on Thursday July 27, 2017 @03:28AM
(#14863317)
The point of this kind of startup is not to sell the tech, but to assemble a team of highly capable, experienced engineers. The big corporations will then buy out the startup just to get the talent. This is called acqui-hiring.
Re:The ImageNet dataset transformed AI research
by tanilama on Thursday July 27, 2017 @04:52AM
(#14863627)
The problem is...Google/Facebook,etc, have the best people, and others are following them, why would they need to acqui-hire someone who are worse than what they already have? For less prestigious companies though, those small companies might still be attractive, however.
Re:The ImageNet dataset transformed AI research
by m_ke on Wednesday July 26, 2017 @03:17PM
(#14859253)
They launched almost two years before tensorflow and google vision APIs were released. At the time there were only a few labs doing deep learning research so an API seemed like the best way to make the technology accessible to more people.
Publishing research papers is a great way to get acquired but it's not the best use of your time if you're a small startup that's trying to make money. They still do research but it makes it into the product instead of getting published.
Re:The ImageNet dataset transformed AI research
by tanilama on Wednesday July 26, 2017 @06:23PM
(#14860921)
AI is not for startups. Not until we have sufficiently reduce the demand for data and computational resources. Or you need to sell a product, like a car or camera, not merely an API.
Re:The ImageNet dataset transformed AI research
by komali2 on Wednesday July 26, 2017 @03:17PM
(#14859246)
Google allows you to train models, and I engaged with customer support when I used their google cloud platform. I agree their other services have terrible customer service, but it seems like when you start paying out, it actually exists.
In contrast to ImageNet, only the validation and test data were manually labeled. Impressively, on such data, an accuracy of ~95% (top5) is achieved by the winning solution this year.
Re:The ImageNet dataset transformed AI research
by yonilevy on Wednesday July 26, 2017 @03:56PM
(#14859658)
(Disclaimer: I work at Nexar)
Nexar just released Nexet, a large dataset of 55K road images (5K of which are labeled). I think it's an interesting one as the images were captured by mobile phones (setup as dashcams), so you get "real world" data. Of-course there's an accompanying competition coming up: https://www.getnexar.com/challenge-2/
Re:The ImageNet dataset transformed AI research
by marcelsalathe on Wednesday July 26, 2017 @02:11PM
(#14858635)
The non-profit platform https://www.crowdAI.org currently hosts a NIPS 2017 challenge as well as a genetic challenge, and constantly adds new challenges. Btw, it's completely open source, and keen on having contributors to help build the open challenge platform the community wants the most.
Re:The ImageNet dataset transformed AI research
by contingencies on Wednesday July 26, 2017 @08:46PM
(#14861729)
Tangential anecdote: Incidentally the article mentions that ImageNet was conceived through WordNet[0]. I actually downloaded WordNet again last night with the idea to relocate qualifying adjectives from certain noun phrases in a system to a separate list of qualifiers, ie. "adj1 adj2 noun" -> {"noun",["adj1","adj2"]}
What I found was that WordNet has crap coverage, eg. large numbers of real adjectives were unlisted. In my ancient Chinese translation work (hobby), I have basically concluded that Wiktionary[1] is the best thing going, and also easiest to contribute corrections and background data to. So I thought "why not adapt English Wiktionary data[2] to produce an English adjective list?". This proved far more complicated than a quick hack, since the only way the data seemed to be supplied was as a huge XML file mixing Wiki and XML markup.[3] In the end I downloaded a Wiktionary-derived dataset known as dbnary[4] and parsed wordlists out of that, resulting in a satisfying list of 103,125 English adjectives. Total time invested, under 2 hours.
However, the prime use case of WN is as an verified inventory of machine readable senses, providing structured information about the semantic attributes of the word by providing the semantic links to other words/senses. Wictionary will tell you that a word is an adjective, but it doesn't tell a machine what does that adjective mean, how it relates to other adjectives.
E.g., the wiktionary entry on 'corgi' has a few sentences that mention that it's a type of dog, but it's not structured enough for this information to be used, and if you just extract it yourself without manual verification then it won't have high accuracy; WN, on the other hand, has an explicit structured link (hypernym) between corgi and dog. (http://wordnetweb.princeton.edu/perl/webwn?o2=&o0=1&o8=1&o1=...)
Re:The ImageNet dataset transformed AI research
by unityByFreedom on Wednesday July 26, 2017 @09:38PM
(#14861975)
Looking forward to when someone makes a significantly sized, fully labeled medical dataset. Currently, the main thing holding us back on diagnostics using machine learning isn't the software or hardware- it's the lack of labeled data, which is costly to prepare because it requires technologists to work alongside doctors. Still, the results could be huge.
I think machine learning through medicine will save more lives than self driving cars.
Re:The ImageNet dataset transformed AI research
by PeterisP on Thursday July 27, 2017 @12:08AM
(#14862631)
It's highly problematic due to privacy issues. It would be a nightmare to get a public release for the underlying unlabeled data, especially since we know how well deanonymization can work.
Re:The ImageNet dataset transformed AI research
by unityByFreedom on Thursday July 27, 2017 @01:31AM
(#14862908)
There already is some that is publicly available, like the Kaggle lung cancer competition's dataset.
I'm not saying it's easy on any front. It is possible, and it would be hugely beneficial to humans to have such a dataset. Raising awareness about the problem may be one step towards solving it.
Re:The ImageNet dataset transformed AI research
by nonbel on Thursday July 27, 2017 @03:21AM
(#14863285)
The problem with a lot of medical labeling is the labels are not very consistent/accurate to begin with. Just go to three doctors with the same symptoms and see what they say...
Re:The ImageNet dataset transformed AI research
by eanzenberg on Wednesday July 26, 2017 @02:43PM
(#14858953)
"Li said the project failed to win any of the federal grants she applied for, receiving comments on proposals that it was shameful Princeton would research this topic, and that the only strength of proposal was that Li was a woman."
Re:The ImageNet dataset transformed AI research
by scottLobster on Wednesday July 26, 2017 @03:02PM
(#14859121)
Stories like this are part of the reason I never went into academia. It's amazing how many highly educated people are judgmental and inflexible, and in academia one person like that can sink you. As an engineer in the private sector, worst case scenario I can find a better team/company to work with.
Re:The ImageNet dataset transformed AI research
by dmix on Wednesday July 26, 2017 @03:40PM
(#14859486)
Not just a dataset but hacking community and contest:
> Matthew Zeiler built Clarifai based off his 2013 ImageNet win, and is now backed by $40 million in VC funding.
On a side note, this looks like a fun company to build: hacking on ML, exposing it via APIs, developers as customers, paid for on a resource basis like a VPS style per-use payment plan.
It's interesting how many of these AI startups are raising huge rounds. The words AI seems to draw them in. At least this one has an obvious business model.
Re:The ImageNet dataset transformed AI research
by eli_gottlieb on Wednesday July 26, 2017 @04:46PM
(#14860170)
I was went to an AI talk at MIT CSAIL (the same Vicarious one I mentioned in another thread). Me and a guy got talking on the way out. He figured I seemed pretty clever, so he started asking me what sorts of companies to invest in.
Turned out he was an investor with some VC firm. I did a little mention of the place I worked at the time (which isn't an AI company, but wanted to be when it was young), but didn't have a business card to give him at he time.
I was actually kinda surprised an investor was attending a talk and saying, quite so uncritically, "hmmm, who do the smart people at talks say I should invest with?"
Re:The ImageNet dataset transformed AI research
by tanilama on Wednesday July 26, 2017 @10:46PM
(#14862296)
A truly terrible dataset that leads to AlexNet/VGG/Neural Style Transfer/GoogleNet/ResNet, and facilitate research so much, is still a good dataset. Actually no, it is a phenomenal dataset. Not to mention, it is free and open to all.
You can make assessment even if you are not an expert, but I would think you need more efforts to backup your claim a little bit to convince people and avoid downvoting.
Re:The ImageNet dataset transformed AI research
by PeterisP on Thursday July 27, 2017 @12:10AM
(#14862646)
99.9% is an extremely high bar. People aren't machines, people make errors. If you asked them to tell whether a square is black or white, you would get less than 99.9%.
Re:The ImageNet dataset transformed AI research
by mturmon on Thursday July 27, 2017 @01:24AM
(#14862884)
It sounds like you may not have experience in this domain? You're benchmarking an active, time-based task (a person "seeing a car" - implicitly allowing eye saccade, multiple images, context, depth) versus a static task ("classify this single image based on no context"). They are not comparable.
Re:The ImageNet dataset transformed AI research
by ouid on Thursday July 27, 2017 @02:51AM
(#14863173)
have you actually tried the imagenet task yourself?
The reason imagenet is popular is because it has allowed google to claim that they can beat human performance on image recognition by artifically lowering human performance on image recognition.
I used Clarifai at a hackathon a couple months ago. It was certainly impressive, but I didn't really see the difference between it and Google Cloud Services AI whatever offerings, other than Google seemed to have a better handle on deployment and production scaling.
I guess I'm wondering how companies like Clarifai and the couple other similar "AI API cloud service" companies intend to take on behemoths like Google?
At the time, Clarifai was the "best" one (I caveat by quoting because this was a for a small corpus, with subjective results, not a real train-test cycle). I re-ran the results about a month ago (linked to from the post), and found that Google and others have continued to invest and improve.
Do you play the tablas? My wife and I studied sitar but our instrument was destroyed by the movers in our latest relocation to Shenzhen :( The tabla teacher where we studied was able to play a very complex taal while chewing betel nut and rolling his eyes back in their sockets, immediately switch to a pitch, bend and time-perfect rendition of 'pink panther' melody, then switch back to a very complex taal without skipping a beat. Brilliant to see.
I am not a deep learning practitioner, but would be curious to know from experts how their custom model feature might work; and from any of their users on how well it actually does.
Tablas: haha, great description of your teacher. I do play, with enthusiasm, but poorly. For those in Seattle, there is an amazing teacher who teaches up on Cap Hill [2].
[0] http://help.clarifai.com/custom-training/custom-training-faq
[1] http://image-net.org/synset?wnid=n01887787
[2] http://www.acitseattle.org
E.g., recall the awesome project for lego sorting by jacquesm[0] ? He built his own model using Keras and Tensorflow, but you may be able to achieve similar results by using Clarifai's feature to train your own models with no understanding of deep learning. This is great if your goal is to build a thing, like a lego sorter, but not so much if you want to learn how to build a state of the art image classifier.
If you're interested in learning about computer vision or deep learning, I recommend searching this site to find threads that cover that extensively. Good luck!
[0] https://jacquesmattheij.com/sorting-lego-the-software-side
IBM, Apple, Amazon, Facebook, Google will all buy these types of companies until they have what they need in house. Other bigger FXXX companies may buy a few as well if customer pilots go well.
Publishing research papers is a great way to get acquired but it's not the best use of your time if you're a small startup that's trying to make money. They still do research but it makes it into the product instead of getting published.
Mark was the key member of the VOC project, and it would have been impossible without his selfless contributions.
https://www.computer.org/csdl/trans/tp/2012/11/ttp2012112081...
http://www.bmva.org/w/doku.php?id=obituaries:mark_everingham
Mark Everingham passed away in 2012, bringing the VOC project to an end.
In contrast to ImageNet, only the validation and test data were manually labeled. Impressively, on such data, an accuracy of ~95% (top5) is achieved by the winning solution this year.
Nexar just released Nexet, a large dataset of 55K road images (5K of which are labeled). I think it's an interesting one as the images were captured by mobile phones (setup as dashcams), so you get "real world" data. Of-course there's an accompanying competition coming up: https://www.getnexar.com/challenge-2/
What I found was that WordNet has crap coverage, eg. large numbers of real adjectives were unlisted. In my ancient Chinese translation work (hobby), I have basically concluded that Wiktionary[1] is the best thing going, and also easiest to contribute corrections and background data to. So I thought "why not adapt English Wiktionary data[2] to produce an English adjective list?". This proved far more complicated than a quick hack, since the only way the data seemed to be supplied was as a huge XML file mixing Wiki and XML markup.[3] In the end I downloaded a Wiktionary-derived dataset known as dbnary[4] and parsed wordlists out of that, resulting in a satisfying list of 103,125 English adjectives. Total time invested, under 2 hours.
[0] http://wordnet.princeton.edu/
[1] http://en.wiktionary.org/
[2] https://dumps.wikimedia.org/backup-index.html
[3] https://dumps.wikimedia.org/enwiktionary/20170720/enwiktiona...
[4] http://kaiko.getalp.org/about-dbnary/
However, the prime use case of WN is as an verified inventory of machine readable senses, providing structured information about the semantic attributes of the word by providing the semantic links to other words/senses. Wictionary will tell you that a word is an adjective, but it doesn't tell a machine what does that adjective mean, how it relates to other adjectives.
E.g., the wiktionary entry on 'corgi' has a few sentences that mention that it's a type of dog, but it's not structured enough for this information to be used, and if you just extract it yourself without manual verification then it won't have high accuracy; WN, on the other hand, has an explicit structured link (hypernym) between corgi and dog. (http://wordnetweb.princeton.edu/perl/webwn?o2=&o0=1&o8=1&o1=...)
I think machine learning through medicine will save more lives than self driving cars.
I'm not saying it's easy on any front. It is possible, and it would be hugely beneficial to humans to have such a dataset. Raising awareness about the problem may be one step towards solving it.
Any effort to create additional medical datasets would probably help enhance machine learning diagnostics.
The data that you want to be perfect is the test set, which is smaller than training sets.
Jesus..
... from my fortune clone @ http://github.com/globalcitizen/taoup
> Matthew Zeiler built Clarifai based off his 2013 ImageNet win, and is now backed by $40 million in VC funding.
On a side note, this looks like a fun company to build: hacking on ML, exposing it via APIs, developers as customers, paid for on a resource basis like a VPS style per-use payment plan.
It's interesting how many of these AI startups are raising huge rounds. The words AI seems to draw them in. At least this one has an obvious business model.
Turned out he was an investor with some VC firm. I did a little mention of the place I worked at the time (which isn't an AI company, but wanted to be when it was young), but didn't have a business card to give him at he time.
I was actually kinda surprised an investor was attending a talk and saying, quite so uncritically, "hmmm, who do the smart people at talks say I should invest with?"
You can make assessment even if you are not an expert, but I would think you need more efforts to backup your claim a little bit to convince people and avoid downvoting.
The reason imagenet is popular is because it has allowed google to claim that they can beat human performance on image recognition by artifically lowering human performance on image recognition.