Mechanical Turk: Now with 40.92% Spam.

Re:Mechanical Turk: Now with 40.92% Spam. by andrewljohnson on Thursday December 16, 2010 @11:49PM (#2015213)

At first, I thought MTurk was good for getting translation work done, and checked by others. It worked OK, but I had to be clever to avoid the cheaters who would ignore instructions not to use machine translation. After refining my tactics, talking to a lot of workers, and fending off the crooks, I got a few decent translations (for my iPhone app).

Then I tried a new tactic. I hired some pros from various translation websites. All were more expensive (money-wise) than MT, but the results were better, and I spent a lot less time. Some were even close to MTurk prices.

So, I don't use MTurk anymore. Maybe if I had a more mindless task I'd try again, but it is a lot of hassle.

I know this isn't exactly what the author means by "spam" - but even the workers on that site are often dishonest. It's a total black market.

[ Parent | on HN ]

Re:Mechanical Turk: Now with 40.92% Spam. by j2d2j2d2 on Friday December 17, 2010 @12:36AM (#2015337)

I had good success stopping cheating by using images of text instead of actual text. Typing in a foreign language is not trivial if it's not something you're used to doing.

I used a python wrapper for pango-view to generate the text. I have a python module for this here: https://github.com/j2labs/pyango_view

Nothing complicated about what I'm doing but the github page shows how nice the text can really look.

[ Parent | on HN ]

Re:Mechanical Turk: Now with 40.92% Spam. by j2d2j2d2 on Friday December 17, 2010 @12:37AM (#2015346)

It might also be of interest that I recently posted a gist that shows how to post some basic mechanical turk tasks using Python's boto library.

https://gist.github.com/740267

[ Parent | on HN ]

Re:Mechanical Turk: Now with 40.92% Spam. by trotsky on Friday December 17, 2010 @12:16AM (#2015285)

It seems to be a failed social experiment to me, there is definitely a negative feedback loop there. Bad HITS, low wages, withheld payments, bad workers, sweatshops. Hard to imagine it will get any better.

I wonder if you reversed the process and had projects bid on workers if it could counter the race to the bottom effect. Include a system where work could be divided into common types and include worker metrics and feedback that captured any specialization. Perhaps allow some workers to fill a managerial role, where they acquire large blocks and have a trusted group they farm out work to. Presumably wages would go up but bad work would go down, perhaps offsetting it somewhat.

[ Parent | on HN ]

Re:Mechanical Turk: Now with 40.92% Spam. by byoung2 on Friday December 17, 2010 @11:16AM (#2016613)

It sounds like you have a great idea for a startup to bring some competition to this area.

[ Parent | on HN ]

Re:Mechanical Turk: Now with 40.92% Spam. by eru on Friday December 17, 2010 @06:57AM (#2015968)

Mturk does have `qualification' (or so) that do capture specialization. Your ideas seem interesting, though.

[ Parent | on HN ]

Re:Mechanical Turk: Now with 40.92% Spam. by d_r on Friday December 17, 2010 @01:34AM (#2015460)

Not 100% relevant to this thread, but if it helps, I've had success with http://www.icanlocalize.com for localizing apps. They assign a human translator for your strings and are accommodating with custom directions you might have. They'll even take your strings file from Xcode.

(Disclaimer: I'm not affiliated with this site, your mileage may wary.)

[ Parent | on HN ]

Re:Mechanical Turk: Now with 40.92% Spam. by DenisM on Friday December 17, 2010 @02:36PM (#2017411)

what's more important - they will return you files fit for XCode! icanlocalize is awesome.

[ Parent | on HN ]

Re:Mechanical Turk: Now with 40.92% Spam. by rgrieselhuber on Friday December 17, 2010 @12:55PM (#2017012)

I'm starting to use MyGengo quite a bit for translations. It's more expensive than MT but the quality is great and service is super-fast.

[ Parent | on HN ]