hyunsoo90
hyunsoo90 on Hacker News
As far as I know, some do but most don't.
Thank you for the interest! We have a team of ~50 in-house annotators that use our AI tools and we are looking to expand out to Southeast Asia, namely Vietnam or the Philippines, to set up our annotator workforce there in the near future.
As you mentioned, we see that some data labeling companies focus on a few verticals like autonomous vehicles. These verticals are very data hungry and we actually do have clients in these sectors, but I also see a huge opportunity in the less "AI-savvy" verticals such as consumer electronics, physical security, factory automation and so on. They not only have a huge need for AI, but they also lack the AI talent to build AI themselves. So, one natural possibility is that we extend our service and actually deliver the AI built on top of the training data we make. To do that, we will need to automate that piece as well using AutoML or Meta-Learning which the co-founders already have experience with (AI building AI!). It's also possible that we stay focused on just the training data piece for a few of these verticals.
We have both AI-assisted and manual inspections in the pipeline. A good analogy would be an assembly line where humans and machines collaborate not only for building things but also for the quality control (ie. vision inspection system + manual inspection)
It depends on how accurate our AI performs on a particular task, but as a back-of-the-envelope calculation, if we had a 90% accurate AI that means human annotators only have to work on the remaining 10%, giving us 10x boost. Obviously, there is some overhead not accounted for in this calculation, but with our current technology we can boost up to 10x depending on the type of data.
Thank you so much! The human-in-the-loop piece really starts to kick in for larger volumes of data where we can iterate the fine-tuning process several times. As an example, we were able to achieve +30% speed boost for a client after a few cycles of the loop. We are a team of 13, including 9 engineers.
Thanks for pointing this out. Although using crowdsourced labor could be the most cost-effective, we don't think it can guarantee the level of quality we get from in-house team. We believe our AI will be able to automate a lot of the pieces and make our in-house team cost-effective. I'm sure you have a lot of experience in terms of scaling workforce, please let me know if you have any advice for us!
Great question! Our AI is trained specifically for data annotation and that leads to a few differences. For example, our data annotation pipeline is not "end-to-end" and would not be suitable for a real-time deployment -- it works on multiple stages and some stages are done by human workers (i.e. editing or verifying the output of the AI), while others are automated.
Thank you! Scale is doing great especially in the autonomous vehicles vertical. We are different in a few ways from Scale or other data annotation companies -- one, we really focus on building AI tools that can automate data annotation all the way from label proposals to quality control, and two, all annotations are done in-house without relying on any crowdsourced labor. We think it's difficult to guarantee quality with crowdsourced workers and we see this a lot with Amazon Mechanical Turks. We can do this because our AI tools give a huge speed boost and our in-house workers can go through the data a lot faster. And also, we also collect and provide raw data (limited to images at the moment) to companies that do not have their own.
Posted by
hyunsoo90 on Wednesday March 13, 2019 @01:04PM
from the self-post dept.
Hi HN,I’m Hyun, one of the co-founders of Superb AI (https://www.superb-ai.com) in the YC W19 batch. We use AI to semi-automatically collect and label training data for tech companies, and help them implement machine-learning based features faster.Almost all the magical AI features that we see are actually built using training data powered by humans. Companies build software tools and farm out...