It's trained on fanart, mostly from pixiv via another site, which is tagged safe/sort-of-nsfw/nsfw but many things are tagged safer than they actually are.
No one knows why but it’s often said(with anecdatas) that >50% of creators for these content are biological female.
So girls draw sexualized girls, some men do as well. Some says the high composition of female artists comes from the fact that moe arts are extension of Shoujo Manga market where predominantly female characters are portrayed by predominantly female artists for predominantly younger female audiences.
Whatever the actual reason might be, it’s probably more complicated than “person with polar opposite sexuality of the subject consume porn”.
Most Japanese animation is entertainment targeting young teenagers and younger. Japan does however have a considerable adult animation market. The a.i. is not trained with any “anime”, however, but with still promotional art which is of far higher quality and clearly trained with the adult market where sexualization is, as one might expect, more common.
The paradox is that the unrealistically cute “moe” designs are typically targeted at adults and adolescents and most of the children's antertainment does not look like that.
Often in Japan, one speaks of four principal demographics: young males, young females, older males, and older females. — the a.i. is quite clearly entirely trained with the “older males” demographic, which should be undertood to target around 18-30 years of age, and even within that demographic it is trained almost exclusively with moe art.
Most “anime” would not even be instantly recognized as such, which is why I think the term is fairly useless. — it seems when people speak of “anime” more often than not what they mean is “moe art” whether it be animated or not.
This is what the average “anime” looks like: https://assets.moomin.com/uploads/2016/02/Moomin-animation-s...