- As you imply, a lot has to do with the available sources of input signal- whether labeling functions, or 'transformation functions' for data augmentation, or other ops we've worked on... the input is obviously key.
- For data modalities like image, video, etc: Often the most successful approach is to (A) rely on some pre-processed features or "primitives" and write labeling functions over these- as my co-founder Paroma in particular has published about over the years- and/or (B) use metadata
- External models are definitely expressable as labeling functions, and we've worked on exactly that problem of modeling (local) biases and correlations!
For example, lets assume you want to identify something like a lung tumor. So you have many MRI images and they're all largely the same template of image. Using traditional image processing software like open CV, it's suprisingly easy to do more coarse grained tasks programmatically, like say, search this image for any circle that's brighter than the surrounding tissue and has a radius greater than say x.yz mm. If you find one, that function returns True if not False. That x.yz mm number is what you get from your radiologists that you work with to help you develop the labeling functions and this is just _one_ of the labeling functions. But basically it turns out if you construct a few of these functions with the help of domain experts and then use those functions all together with the information theory research the Snorkel folks do, you get pretty damn good performance!
That's btw why a lot of examples of ML today are ones where data is (i) simple for non-experts to label, (ii) non-private and therefore easy to outsource for labeling, and (iii) low rate of change (e.g. images for self-driving, basic NLP stuff for chat bots, etc)- this kind of data can be labeled cheaply and once, so hand-labeled training sets are (barely) economically feasible to build manually. However, most data is not that easy or cheap to label, needs to be relabeled constantly to adapt to change, and thus the investment in a programmatic approach is often far better even if certainly not push-button!
That said, I don't see anything here that would prevent you from using a pre-trained conv net as a labeling function, but I expect that multiple conv nets trained on a small corpus of data would be biased and make correlated errors, which violate their assumptions.
This looks super powerful in some cases, but I'm just not seeing how it can possibly generalize to every ML problem.