Pretty much all AI algorithms seek to classify their data (us) into a smaller more manageable number of categories. AI methods work best when each of us 'consumers' fits neatly into one of these classes (whether predefined or emergent; each class's origin doesn't matter). Upon implementation, you shall become a label, so that future interaction with the AI can assume you to be a "Class 137" and lock you into a simplified model (with all your features having low value eigenvectors conveniently excluded). Deviation from this norm will not be tolerated, simply because there's less profit in it.
Blah. If anything, better AI models of the world need to capture more subtlety leading to better user-driven customization, not less. For Predictive Design not to become yet another Big Brother, it needs to reflect each individual's unique constellation of attributes, not the single dumbed-down class they were labeled into.
Seven billion classes. That's the ticket.
We see generic and bland work from marketers and designers quite often, to be honest. With predictive designing, we want to step up everyone's game.
If you create unique and engaging content, then for sure no A.I. can judge you. In fact, if you create extremely unique content you will be an Outlier in the prediction probably and will not score high enough. But think of banners on websites.
You have to gather the attention of the user, that will last not even 2 secs, and you have to make your creative memorable in order to increase the brand recall in your company. This is a case study were A.I. could rank your design in terms of memorability.
Which I suppose makes it easier for someone who's actually honed their craft into something distinctive to stand out from the crowd, but...