Re:“Less than one”-shot learning
by vikramkr on Sunday October 18, 2020 @04:03PM
(#24820669)
So this is more like data storage/some sort of extreme compression than anything? Would it be accurate to interpret this as basically saving the trained neural network into 10 images and reading it back to retrain a model? Or, what really does this do/accomplish?
[ Parent
| on HN ]
mostly it seems that it tells you something sort of weird about how neural networks train. it’s not obvious that this should work, and that it can be made to work is interesting.
Agreed, the idea is interesting and worthy of further exploration. For starters, does it scales beyond MNIST or 'tiny synthetic datasets'?
Re:“Less than one”-shot learning
by freeone3000 on Sunday October 18, 2020 @04:24PM
(#24820844)
The approach appears similar to model distillation, performed on the input data instead of on the weights. Model distillation scales well in classification but less well in generation.