Launch HN: Lightly (YC S21): Label only the data which improves your ML model

Re:Launch HN: Lightly (YC S21): Label only the data which im... by citilife on Monday August 09, 2021 @10:50AM (#28117632)

Having built a model to identify sensitive data having a solid data labeling solution would be awesome. I can attest this is a real problem. Here's the library we built:

https://github.com/capitalone/DataProfiler

In this space, Prodigy really dominates:

https://prodi.gy/

We actually built our own internal system which integrates and can export the labels (does predictive labeling, etc). Of course, we only focused on text data at the moment.

All that being said, this is going to become a crowded space and highly competitive. Plus, once the data is labeled companies often drop their labelers. I would recommend ensuring some consistent use. Potentially, hosting their models off-prem or something to lock companies in.

[ Parent | on HN ]

Re:Launch HN: Lightly (YC S21): Label only the data which im... by isusmelj on Monday August 09, 2021 @12:19PM (#28118764)

The library looks great!

Prodi.gy is great but focuses heavily on NLP and speeding up the labeling process. Our goal is really to help you reduce what you label before you use any labeling tool.

We are working with labeling tools as well as providers to streamline the workflow.

[ Parent | on HN ]

Re:Launch HN: Lightly (YC S21): Label only the data which im... by tracyhenry on Monday August 09, 2021 @03:08PM (#28120955)

Doesn't Prodi.gy also claim to do active learning, which essentially reduce the instances to label too?

Haven't used Prodi.gy so don't know how its active learning algo works. May you share the difference?

[ Parent | on HN ]

Re:Launch HN: Lightly (YC S21): Label only the data which im... by isusmelj on Monday August 09, 2021 @05:46PM (#28122706)

Most active learning frameworks just use model predictions to find the images where the model has the lowest confidence and ignore the image diversity aspect. E.g. the model struggles with bicycles at night. The problem with this approach is that you might end up adding many new images to your labeling pipeline that are very similar to each other.

However, with Lightly you can additionally make sure you only select images that are visually different from each other. And you always get visual feedback on the selected data in our web platform. The additional control and feedback mechanism allow you to have a more focused workflow.

[ Parent | on HN ]