It's a good idea to think through ideas before attempting them.

This experiment was like asking people to flag "news", and getting surprised that people were flagging tech news, when they really wanted to avoid crime news.