News for hackers. Stuff that matters.
Re:Apple machine learning in 2020: What’s new? by gwern on Tuesday June 30, 2020 @04:06PM (#23694217)
The data augmentations we are interested in are in fact 'midstream', as they augment the examples before passing into the D or the classification loss but you must backprop from that back through the augmentation into the original model, because you don't want the augmentations to 'leak': the G is not supposed to generate augmented samples, the augmentation is there to regularize the D and reduce its ability to memorize real datapoints. It would probably be better to consider them as a kind of consistency or metric loss along the lines of SimCLR (which has helped inspire these very new GAN data augmentation techniques). It's a bit weird, which is perhaps why despite its simplicity (indicated by no less than 4 simultaneous inventions of it in the past few months), it hasn't been done before. You really should read the linked Github thread if you are interested.
Re:Apple machine learning in 2020: What’s new? by spott on Tuesday June 30, 2020 @04:36PM (#23694495)
Ah! I can see that in a GAN architecture. That makes much more sense.

It wasn't clear from your original post that you were augmenting generated images, not real data.

Re:Apple machine learning in 2020: What’s new? by gwern on Wednesday July 01, 2020 @10:20AM (#23701605)
You're augmenting the real data too.