A better way to build ML: Why you should be using Active Learning

Re:A better way to build ML: Why you should be using Active ... by rocauc on Thursday February 04, 2021 @02:16PM (#26028735)

Nice read.

Can you shed some light on what you think are the most valuable methods for identifying high entropy examples for the model to learn faster? I'm familiar with Pool-Based Sampling, Stream-Based Selective Sampling, Membership Query Synthesis[1], but less certain which techniques are most useful in NLP.

[1] https://blog.roboflow.com/what-is-active-learning/

[ Parent | on HN ]

Re:A better way to build ML: Why you should be using Active ... by razcle on Thursday February 04, 2021 @02:35PM (#26028913)

So entropy based active learning methods are an example of pool based sampling. Even within pool based sampling there a few different techniques.

Entropy selection for pool based methods looks at the output probability for each prediction of the model in the unlabelled data-set. Then it calculates the entropy of the distributions. (in classification this is a bit like looking for the most uniform predictive distributions) and prioritises those.

Entropy based active learning works ok but doesnt distinguish uncertainty that comes from a lack of knowledge (epistemic uncertainty) from noise. Techniques like Bayesian Active Learning by disagreement can do better. :)

[ Parent | on HN ]

Re:A better way to build ML: Why you should be using Active ... by rocauc on Thursday February 04, 2021 @04:22PM (#26029959)

Makes a lot of sense, thanks. I'll need to dig deeper into Bayesian active learning techniques.

[ Parent | on HN ]

Re:A better way to build ML: Why you should be using Active ... by pgao on Thursday February 04, 2021 @06:39PM (#26031123)

Here's a great paper to get started with! https://arxiv.org/abs/1703.04977

[ Parent | on HN ]