News for hackers. Stuff that matters.
Re:I hired 50 people today. This is what they look like by ashishk on Tuesday December 22, 2009 @01:19PM (#1010395)
Cool experiment. Thinking of using Turk for photo approvals on my site, but the accuracy issue leaves me slightly hesitant. Has anyone had experience doing this?
Re:I hired 50 people today. This is what they look like by jfno67 on Tuesday December 22, 2009 @02:25PM (#1010539)
What I did was to have up to five person work on a hit. Many simple hits are done at 0.05$ so it's still not too expansive.

For text submissions, you can enforce validation rules (simple and regex). So you don't have to manually reject non-conforming work. You can also have the Turker fill in their work on a webpage that you control, so validation can be much more advanced. I never went all the way but sometimes I think about it.

Re:I hired 50 people today. This is what they look like by eru on Tuesday December 22, 2009 @01:23PM (#1010408)
From what I read the standard technique seems to be let two people review everything. When there's a discrepancy you can add more manpower (and take the majority view or so) or look at it yourself.
Re:I hired 50 people today. This is what they look like by jules on Tuesday December 22, 2009 @03:03PM (#1010656)
A problem is that there are rooms of people turking together. They'd probably all submit the same translation, for example.
Re:I hired 50 people today. This is what they look like by eru on Wednesday December 23, 2009 @06:02AM (#1011907)
I guess you should flag exactly similar translations. Though I do not see how you can do this without relying on security by obscurity.
Re:I hired 50 people today. This is what they look like by eli on Tuesday December 22, 2009 @01:36PM (#1010446)
Yup. There are a couple of language translation services that use techniques like that.
Re:I hired 50 people today. This is what they look like by hyung on Wednesday December 23, 2009 @11:49PM (#1013423)
Hey ashishk, sorry for using HN to contact you, but I can't seem to find your contact info anywhere.

Our company provides very affordable and accurate photo approval services. Photos are manually approved and we can handle anywhere from 10-1,000,000 photos per day. If you're interested, feel free to email me (email address is in my profile).