That could give you an starting point. But I believe the main issue is that fashion products have, by definition, short shelf life, so you can't run the algos on SKU data. Then you can use deep learning on product images + user categorical data to try to predict preferences, maybe simple binary classification?
I guess using images as input should give better features than textual description.
You would have to tag the hell out of these photos, right? Disambiguating preferences is the challenge -- the user may have liked images #3 and #7, but why? Specific items, or the color palette, or the silhouette, or just the model? A post by Chicisimo on the front page addresses these hurdles [1].
I've also seen some of these image recognition apps in action -- they pick up on patterns and color very well, but struggled with silhouette.
[1] https://hackernoon.com/how-we-grew-from-0-to-4-million-women...
That's part of what makes this industry hard.
I've thought about this as well. How would you personally try to build this profile for users, then market to them based on it?