Re:A group of ex-NSA and Amazon engineers are building a ‘Gi...
by darawk on Thursday February 20, 2020 @01:34PM
(#22377346)
Except that if their models have already inferred sufficient structure from your data to do this in a way useful for training, then their models have already solved the problem your models are trying to learn.
Re:A group of ex-NSA and Amazon engineers are building a ‘Gi...
by AndrewKemendo on Thursday February 20, 2020 @02:12PM
(#22377680)
As a point of clarification it's not the model that is being abstracted, it's the raw data.
The use cases that this addresses are ones where you cannot train models in the same system/space/network as where the data exists.
It's a technique we've used in the DoD a long time and it works ok when everything is perfect. There are a lot of boundary problems like being able to troubleshoot bad data if you are doing your initial analysis with the transformed data, having DS's actually grok the problem-set since it's abstracted etc...
Edit: It's worth noting that this is a technical solution to a policy/legal roadblock. As organizations mature into better data governance, they are pushing more fundamental changes to governance that gets at these problems where solutions like this will no longer be necessary. For example, hiring data scientists into the groups that have access to the raw data (in our case hiring data scientists and giving them security clearances).