Re:“Less than one”-shot learning
by ampdepolymerase on Sunday October 18, 2020 @04:05PM
(#24820676)
It may mean we might finally have a method for reliably updating big neural networks instead of having to do continuous on the fly retraining. (Imagine future neural networks on your smart car having "upgrade packs" or country specific data that can be used to fine tune the main network in a matter of minutes) A high-level form of patch-and-diff for networks. There is probably a ML Ops startup opportunity somewhere in this.
[ Parent
| on HN ]
"Upgrade packs" might be already possible using transformer adapters, i.e. tiny networks trained on customized data plugged into a large fixed pretrained transformer, providing whatever custom functionality you require.
Re:“Less than one”-shot learning
by heyitsguay on Sunday October 18, 2020 @09:49PM
(#24822707)
I don't think this changes anything. Deployed networks typically use only inference from pretrained weights, and those weights are what get transferred for model "updates". You can have all your devices using weight array W0 for a neural net architecture, spend a million compute hours training that net on cutting edge systems to produce a much better weight array W1, then upgrade all the deployed devices by sending them W1 which will be the same size as W0.
Re:“Less than one”-shot learning
by ampdepolymerase on Sunday October 18, 2020 @11:57PM
(#24823213)
For aerospace applications where bandwidth is severely limited, a hundred megabyte NN is a lot.