Some of the process on gathering the data to create the labelling dataset is described here: https://useguard.com/experiment

But most probably I'll be publishing a paper later this year detailing all the details and process :)

Very cool idea! One question I couldn’t seem to find the answer to on your site was are the policies featured on your a/b training exercise distinct from the polices that the ai grades? For example will a user going through your a/b trainer ever see a snippet from the Instagram privacy policy?

Yes, initially it will all draw from the same dataset, so a user in theory could definitely see all services' snippets. But, to increase statistical significance in the data it gathers, I've restricted the initial amount of items in the test so right now this will not be the case (otherwise, I'd be dealing with circa 3,500,000,000 different pairwise comparisons hehe)

Where can we follow you to know abt the paper when you do eventually publish?

I'm most active on Twitter :) https://twitter.com/rameerez