> the semantics it learns sometimes have significant artifacts resulting from associations that are common in news coverage

I think this is largely the point. The game is significantly easier (for me at least) when I limit myself to thinking about what words would have high co-occurrence in English print journalism.