I honestly believe most MTurkers just clicked random BS in order to complete the tasks as soon as possible.
What I ended up doing was make some Python CLI-based took that made it extremely fast for us to classify messages; after seeding it with about 1000 classifications, we would then focus on the messages based on certain dimensions: eg “contradictions” (“positive” and “negative” being closest as possible, or “angry” and “happy”), “least” (it was surprisingly difficult to find positive and uplifting tweets, and you don’t want a dataset with 90% negative messages!), etc.
That way we worked our way through the dataset and were able to get a pretty decent dataset in about a week time.
No idea how others approach this type of problem, but it’s what I came up with.
This is correct based on my (limited) mechanical turk experience. Most tasks pay peanuts (minimum payout can be as low as $0.01) so the only reasonable way to make an income is to complete as many tasks as humanly possible, and doing anything but clicking random buttons would slow them down. I doubt paying more could overcome that because so many people engage with the platform in bad faith.