> It seems to me that anything trained on internet data is kinda doomed to poison itself on the high ratio of garbage floating around here?

Ah one only needs to think back to Tay to know how these sort of things will end.

https://en.wikipedia.org/wiki/Tay_(bot)

(Imagine 4chan got a wind of this bot and retrained it, which is probably what happened...)