> So there was a 5 petabyte learning set we trained a 20 layer deep neural net and a value came out the other end. Here's all the math we did to figure that out (several gigs of float computations). We don't even know what it's doing exactly. What does GDPR say about that?

That depends completely on what you are using the value for. Recommending five funny articles - GDPR does not apply. Denying an insurance claim - GDPR most certainly applies, and you have to be able to explain what factors went into the decision. You can't have unaccountable oracle boxes.

Note that a perfectly valid answer could be something like "We've analyzed your posts on social media and we've categorized you as having severe anger issues, which is why we're denying this auto collision insurance claim, because you are most likely at fault given situations like this". The GDPR says you have to be able to explain your decision like that. It doesn't forbid you from making these decisions.

> If you took someone's picture and ran it through neural style, would that be illegal because you couldn't tell them exactly why it painted their nose blue

This is not a business decision, don't be ridiculous.