If you’re trying to classify images as whether or not they contain something, you need a bunch of images that you already know don’t contain the thing (got it), and a bunch of images that you already know /do/ contain the thing. We lack the latter, so there’s nothing for the algorithm to learn based on.

But this is not the only way you can deploy machine learning techniques. You could deploy unsupervised learning to first learn what visual features are common in such images, and then successively refine until you end up with a set of images containing very rare phenomena. That could give human evaluators then a considerable head start.

In fact, this idea seems so obvious that it must have been tried already…

Has it? I'm very interested in using data science for astronomy as it was my childhood passion and I'd love to read anything you can provide.

I'm sorry, this was pure speculation on my part, I don't have actual knowledge of this happening.

Oh I see I assumed the GP was talking about after such a dataset I agree you likely couldn't bootstrap without a lot of data but couldn't you use more classic approaches to extract data and then train a model (obviously once you know the extraction process is fool proof?)