Re:I hired 50 people today. This is what they look like
by mindstab on Tuesday December 22, 2009 @01:26PM
(#1010421)
It could have something to do with the time the job was created. As the author mentioned in a comment, after 3 hours the job kinda died. So If he posted it in peak hours for India, than it's not surprising that most of the respondents were Indian, while the fellow from your link may have posted it in peek hours for a more "white" timezone.
What we really need I supposed is one of these every hour for 24 hours. Then we'll get a better idea of the demographics of MTurk.
Re:I hired 50 people today. This is what they look like
by paulgerhardt on Tuesday December 22, 2009 @01:28PM
(#1010427)
Andy Baio's project took place over the course of 3 days; it's not explicitly stated how long this task took to complete though given how high the pay was (relatively) I could see this job filling up in an hour.
(Edit: or 3 hours - pay seems to be a large factor, I wonder if by paying less one gets more people who do the task out of boredom rather than necessity and how that correlates to quality of results.)
Re:I hired 50 people today. This is what they look like
by benhoyt on Tuesday December 22, 2009 @03:00PM
(#1010643)
This is similar to a short survey I ran the other day: I wanted some stats about how many people used wedding gift registries, so I put an ad on Facebook (targeted at married females in NZ) that said "Answer 4 questions about your wedding and be in to win $50".
70% of people who clicked did the survey, which is a really high "conversion" rate -- I suspect the $50 prize draw really helps. I spent about $0.25 per click, so all up 200 survey results cost me about $120.
Re:I hired 50 people today. This is what they look like
by philfreo on Tuesday December 22, 2009 @09:55PM
(#1011392)
Facebook used to have a really useful (and inexpensive) Polls feature that allowed you to get something like 1000 or 5000 responses from your demographic to a multiple choice question. They pulled it for some reason though.
http://www.washingtonpost.com/wp-dyn/content/article/2009/01...
Re:I hired 50 people today. This is what they look like
by benhoyt on Tuesday December 22, 2009 @03:44PM
(#1010751)
Yes -- though not yet, because they (weirdly) haven't replied to my "you've won the $50 survey prize" email. Either they thought it was spam, or they typed in their email address wrong. I'm guessing the later, and will move onto the next randomly-selected participant in a couple of days.
Re:I hired 50 people today. This is what they look like
by markerdmann on Tuesday December 22, 2009 @02:53PM
(#1010617)
I'm surprised that this made it to the top of HN. I did something similar for my friend's birthday last year, but I never considered it particularly newsworthy. Another fun project was proving to our product manager that he was completely wrong (about listing overdue timesheet dates in reverse chronological order) by submitting the question to Mechanical Turk and getting instant user feedback.
Does the popularity of this link mean that there is enough interest in Mechanical Turk to justify the creation of web apps around it?
Re:I hired 50 people today. This is what they look like
by steveeq1 on Tuesday December 22, 2009 @01:17PM
(#1010388)
Yeah, it is. There were a lot of bad submissions. I'm just experimenting with Mechanical Turk right now. Initial impressions. . . 1) a lot of fraud (about half sent in pictures of their cat or something) 2) most people are from india 3) Most of your replies come in the first 3 hours or so. After that it wains. You have to resubmit after this 4) There were repeat submissions. Three of them looked like they are from the same room, same camera. But people claimed different locations.
Re:I hired 50 people today. This is what they look like
by jfno67 on Tuesday December 22, 2009 @02:30PM
(#1010557)
Mechanical Turk is a fascinating tool, maybe this is not the most interesting use, but it still make hacker drool. When you can't have AI, you use AAI (Artificial AI).
You can use it to train pattern recognition. It sure beats doing the work myself.
Re:I hired 50 people today. This is what they look like
by eli on Tuesday December 22, 2009 @01:33PM
(#1010437)
Mechnical Turk is kind of interesting. It's one of those things that I'd really like try out, but have no immediate use for. I'm sure I'm not the only one. (In fact, most of the cool projects I've seen using MTurk are of the Gee Whiz variety)
Re:I hired 50 people today. This is what they look like
by markbnine on Tuesday December 22, 2009 @01:52PM
(#1010477)
This makes me curious about Amazon's vetting process for requests.... taken one step further, I can imagine all sorts of ridiculous things to ask people to do.
Re:I hired 50 people today. This is what they look like
by noonespecial on Tuesday December 22, 2009 @02:48PM
(#1010607)
The one fascinating angle I could think of is crowdsourcing something that is harmless and legal if done once in isolation but becomes harmful and dangerous when done on a huge scale. My brain is totally failing me here on a coming up with a good example but it seems conceivable (and diabolical!)
One papercut is a nuisance 100 million is another thing altogether.
Re:I hired 50 people today. This is what they look like
by mattiss on Tuesday December 22, 2009 @02:53PM
(#1010619)
Something with stock manipulation could work. Buy a ton of options in some company, like Palm for instance. Then on the release of some hot new gadget, like the Pre, pay a ton of people to wait outside the doors of the vendors. Initial buzz will be that Palm is going to strike it big. Cash in.
Re:I hired 50 people today. This is what they look like
by jacquesm on Wednesday December 23, 2009 @06:46AM
(#1011936)
If it were in a movie you'd say 'no way', but to see it in real life... I'd hate to be that guard, being pepper-sprayed is not nice, but otherwise this 'heist' has me with ambiguous feelings towards the robber.
Re:I hired 50 people today. This is what they look like
by shpxnvz on Tuesday December 22, 2009 @03:03PM
(#1010652)
However, there would be a record of the entire thing in Amazon's system, not to mention that a huge audience of potential Mechanical Turk workers would see the offer - it seems likely that at least one would be suspicious and try to notify an authority.
Re:I hired 50 people today. This is what they look like
by nostrademons on Tuesday December 22, 2009 @05:17PM
(#1010949)
What if they told people to stand in line at the store next to Palm? Since there's a finite amount of space, they'll naturally overflow. To an outside observer, it looks like all these folks are standing in line for Palm - particularly if you do the same thing at 1000 locations nationwide, and let journalists connect the dots.
It's also far less traceable. Instead of paying 50,000 people to stand in line outside of Palm, you're paying 50 people each to stand in line outside of 1000 different local businesses, whose only connection is that they happen to be next to a Palm store. A just-as-plausible explanation is that you're organizing a "buy local!" protest that day.
You could also pay people on Amazon Mechanical Turk to pay turkers to stand in line, making things even harder to track. Now instead of one guy paying 50 people each to stand in front of 1000 different locations, you have 50 guys, each one of which is paying people to stand outside Subway, or McDonalds, or Verizon, or the corner bookstore, or whatever happens to be next to that Palm store.
Re:I hired 50 people today. This is what they look like
by justinhj on Wednesday December 23, 2009 @02:09AM
(#1011729)
That naively assumes that you jump up then fall. They could simply climb up steps and drop down, thus resulting in a net movement. Over time we would move into moon collision orbit.
Re:I hired 50 people today. This is what they look like
by markbnine on Tuesday December 22, 2009 @02:36PM
(#1010570)
What if somebody asked people to write something profane? Or political? Or perhaps deface the image of an icon or religious leader? Once you move past written messages, you open up all sorts of avenues. Crowd-sourced politics? Crowd-sourced porn? Crowd-sourced violence? Etc. At what point does Amazon need to filter these requests?
Re:I hired 50 people today. This is what they look like
by thirdusername on Tuesday December 22, 2009 @02:40PM
(#1010584)
When it's illegal? There's a huge difference between crowd sourcing violence and writing something political (which a lot of people already get payed to do), ones organized crime the other is not.
Re:I hired 50 people today. This is what they look like
by brk on Tuesday December 22, 2009 @04:08PM
(#1010810)
While I don't doubt that everyone has their price... I think you would find it cost-prohibitive to pay enough people to do something like that to have any sort of real-world impact.
People are not going to go out of their way, or risk jail time, for $1.00/task. You'd need to pony up a lot more money (like, I'm guessing, 100's of thousands of dollars) to get enough people to violate basic principles on a large enough scale for this to create some sort of shift.
Re:I hired 50 people today. This is what they look like
by imgabe on Tuesday December 22, 2009 @02:41PM
(#1010585)
I don't see any problem as long as they're not asking them to break any laws. Ultimately it's up to the parties involved to obey the law though. If someone puts an ad for a hitman in the help wanted section of the newspaper, should the newspaper be held responsible?
Re:I hired 50 people today. This is what they look like
by SwellJoe on Tuesday December 22, 2009 @06:42PM
(#1011103)
If they were to do so, beyond merely making people agree to terms of service that prohibit such things, they would very likely open themselves up to a lot more legal liability. If they did claim to review and filter projects, and they missed one, they could then be liable for whatever horrible things happened as a result of someone performing the task. I'm pretty sure being a neutral exchange, where both buyer and seller beware (though each has processes for remedying lack of performance or lack of payment), is a safer model for Amazon.
On top of the seeming additional liability Amazon would be inflicting on themselves, I can't see any upside to Amazon going to extra lengths. It would anger project creators to have to wait for human approval for every project, it would likely reduce the number of jobs underway at any given time (making it less profitable for them and less lucrative a market for service providers), and it would increase the expense of running the market at human scale rather than computer scale (even if they somehow figured out how to use Mechanical Turk itself to provide the vetting). In short, vetting jobs would be a recipe for a fiery death for Mechanical Turk.
Re:I hired 50 people today. This is what they look like
by yaacovtp on Tuesday December 22, 2009 @02:56PM
(#1010627)
There is a request up now looking for someone to take a picture of themselves dressed as a terrorist. Many others ask people to fill out a quick survey or enter in an email for lead generation/reward websites.
Re:I hired 50 people today. This is what they look like
by khandelwal on Tuesday December 22, 2009 @01:25PM
(#1010417)
That makes sense. $1 is 40 Indian Rupees, and $50 (the prize money) is 2339 Indian Rupees at current rates. Neither amount is an insignificant amount of money in India.
Re:I hired 50 people today. This is what they look like
by nishantmodak on Tuesday December 22, 2009 @02:50PM
(#1010613)
If $1 = Rs. 40
Then $50 = Rs. 2000
Yes. Neither amount is insignificant. But I still wonder, why only India?
People from Pakistan, China and even Sri Lanka , Singapore are in +/- 3 hrs. timezones.
The timing of the post would be crucial to understand this.
Re:I hired 50 people today. This is what they look like
by agravier on Tuesday December 22, 2009 @07:20PM
(#1011155)
FYI, $1 is insignificant in Singapore. Average household monthly income probably around 4000 USD (personal estimate, I live there, didn't look for official data)
Re:I hired 50 people today. This is what they look like
by jacquesm on Wednesday December 23, 2009 @07:02AM
(#1011949)
It shouldn't just irk you as an Indian, it irks me as a dutch person as well.
If a service finds people from a certain country good enough to be 'minions' how come it finds them not good enough to be 'bosses' ?
This is actually a really nice opportunity to give amazon a run for their money and to set up a competing service that operates world-wide on both the give and the take side.
Re:I hired 50 people today. This is what they look like
by paulsilver on Wednesday December 23, 2009 @09:18AM
(#1012087)
There are employment laws in the UK which mean a Mechanical Turk-like system wouldn't be able to have jobs listed for such low amounts - I think it comes under the minimum wage laws.
Because of this and similar laws in other countries, Amazon have kept job-posting MT accounts limited to the USA at the moment, which doesn't have the restriction.
I don't know enough about Dutch or Indian law to know whether a similar system could be set up within them, but there are problems with rolling something like this out globally.
Re:I hired 50 people today. This is what they look like
by jacquesm on Wednesday December 23, 2009 @10:25AM
(#1012181)
Minimum wage laws only apply to a situation where there is an employer-employee relationship between the two parties, this is much more like a free-lance situation.
Re:I hired 50 people today. This is what they look like
by dylanz on Tuesday December 22, 2009 @01:21PM
(#1010402)
I think he paid them all $1 for the task, with the added bonus of possibly getting $50 if they win. Why is that not nice? They got paid, and entered a contest. $1 for a HIT on Mechanical Turk is pretty good, especially considering the amount of work they needed to do here (which is not much).
Re:I hired 50 people today. This is what they look like
by malvim on Tuesday December 22, 2009 @02:23PM
(#1010536)
$1 seems to be pretty great for this task.
I just read some HIT descriptions, and there were A WHOLE LOT of "write 200 words for $0.03" jobs. Which really sucks if you ask me.
Re:I hired 50 people today. This is what they look like
by Poiesis on Tuesday December 22, 2009 @03:51PM
(#1010762)
The funny thing is, every single one of those participants thought it was worth their time. They looked for something to earn money, and found this. They clearly thought the time/money tradeoff was sufficient.
If you can earn a buck doing what a lot of (generally wealthier, admittedly) people do for free, why not?
Re:I hired 50 people today. This is what they look like
by ashishk on Tuesday December 22, 2009 @01:19PM
(#1010395)
Cool experiment. Thinking of using Turk for photo approvals on my site, but the accuracy issue leaves me slightly hesitant. Has anyone had experience doing this?
Re:I hired 50 people today. This is what they look like
by jfno67 on Tuesday December 22, 2009 @02:25PM
(#1010539)
What I did was to have up to five person work on a hit. Many simple hits are done at 0.05$ so it's still not too expansive.
For text submissions, you can enforce validation rules (simple and regex). So you don't have to manually reject non-conforming work. You can also have the Turker fill in their work on a webpage that you control, so validation can be much more advanced. I never went all the way but sometimes I think about it.
Re:I hired 50 people today. This is what they look like
by eru on Tuesday December 22, 2009 @01:23PM
(#1010408)
From what I read the standard technique seems to be let two people review everything. When there's a discrepancy you can add more manpower (and take the majority view or so) or look at it yourself.
Re:I hired 50 people today. This is what they look like
by hyung on Wednesday December 23, 2009 @11:49PM
(#1013423)
Hey ashishk, sorry for using HN to contact you, but I can't seem to find your contact info anywhere.
Our company provides very affordable and accurate photo approval services. Photos are manually approved and we can handle anywhere from 10-1,000,000 photos per day. If you're interested, feel free to email me (email address is in my profile).
Re:I hired 50 people today. This is what they look like
by cookiecaper on Tuesday December 22, 2009 @01:53PM
(#1010480)
Would you do that for a dollar? I might if I knew where I had a spare piece of paper and a Sharpie, but I don't.
The payment for most of the tasks on Mechanical Turk and other things like it is just not worth the time for almost any task I've seen. Which is really sad, because it'd be awesome if it were worth it.
Re:I hired 50 people today. This is what they look like
by nostrademons on Tuesday December 22, 2009 @02:04PM
(#1010504)
I would if I just happened to come across the posting and just happened to have paper and a sharpie and a digital camera next to me.
It's interesting to read all the comments that say "$1?!?! You're ripping off these people! Why would anyone ever do anything on Mechanical Turk?" How many hours do y'all spend on Hacker News? How long did it take you to write those comments? How much were you paid for it?
In strictly economic terms, Turkers are getting ripped off a lot less than news.YCers are. After all, $1 is infinitely more than $0. We labor for free so that Paul Graham can have a viable online community and pick of the top posters for his seed capital firm.
Re:I hired 50 people today. This is what they look like
by cookiecaper on Tuesday December 22, 2009 @02:07PM
(#1010509)
This isn't really labor, first of all. For most of us it's both a leisure and educational activity that we participate in not for PG's benefit, but for our own benefit by learning and socialization. Making a sign that says "Hi Lili" and taking a picture of it is much less fruitful, even if it nets me $1 in money. The things I learn on HN net me much more money in the long run.
Re:I hired 50 people today. This is what they look like
by pbhjpbhj on Tuesday December 22, 2009 @03:03PM
(#1010653)
Except perhaps the ad-hoc (non-rigorous) studies that have been done before asking people why they turk. Someone mentioned last years submission of such a study further up the thread.
Of course you may be being philosophical about it. In which case, no we have no real way of knowing if they even think.
Re:I hired 50 people today. This is what they look like
by robryan on Tuesday December 22, 2009 @03:39PM
(#1010741)
The taking a picture task to could be though of as fun and something different compared to a lot of the jobs on there. So in this case someones motivation might not having anything to do with the dollar on offer.
Re:I hired 50 people today. This is what they look like
by socratees on Tuesday December 22, 2009 @02:21PM
(#1010532)
I hope I'm not sounding offensive. But what is the author trying to prove here? It's a well known fact that majority of the people on mechanical turk come from developing / under developed countries.
Re:I hired 50 people today. This is what they look like
by SwellJoe on Tuesday December 22, 2009 @04:25PM
(#1010848)
No one has accused you of being on the offensive side, I don't think, and I wasn't arguing with you. I just didn't see any indication that this project had much to do with the location of participants. So, I just wasn't sure what you were getting at with your "But what is the author trying to prove here?" query, since it seemed to me that the guy was just trying out Mechanical Turk and documenting the results, without any social or political commentary that I could perceive. It doesn't seem like he was trying to prove anything.
http://waxy.org/2008/11/the_faces_of_mechanical_turk/
Interesting to see how the demographic has shifted in that time period.
Regarding the bit about crowd-sourcing ideas; Andrew Hyde used Turk to solicit startup-ideas: http://andrewhy.de/using-mechanical-turk-for-ideas/
What we really need I supposed is one of these every hour for 24 hours. Then we'll get a better idea of the demographics of MTurk.
(Edit: or 3 hours - pay seems to be a large factor, I wonder if by paying less one gets more people who do the task out of boredom rather than necessity and how that correlates to quality of results.)
70% of people who clicked did the survey, which is a really high "conversion" rate -- I suspect the $50 prize draw really helps. I spent about $0.25 per click, so all up 200 survey results cost me about $120.
Does the popularity of this link mean that there is enough interest in Mechanical Turk to justify the creation of web apps around it?
That's making other people pay with their time for your own lack of specificness.
wanes
Can someone please tell me why this submission is interesting to HN? (Obviously it is, I just can't figure out why.)
You can use it to train pattern recognition. It sure beats doing the work myself.
Also, people like to look at pictures.
Or, try soliciting feedback from 100 people on the clarity/ease of your company's sign-up process, introductory video, etc.
Many of the comments will be worthless, but you'd be surprised at the quality you get from some people, even at rates as low as $0.05-0.10.
http://www.thestever.net/viewer.php?num=27
One papercut is a nuisance 100 million is another thing altogether.
http://news.cnet.com/bank-robber-hires-decoys-on-craigslist-...
It's also far less traceable. Instead of paying 50,000 people to stand in line outside of Palm, you're paying 50 people each to stand in line outside of 1000 different local businesses, whose only connection is that they happen to be next to a Palm store. A just-as-plausible explanation is that you're organizing a "buy local!" protest that day.
You could also pay people on Amazon Mechanical Turk to pay turkers to stand in line, making things even harder to track. Now instead of one guy paying 50 people each to stand in front of 1000 different locations, you have 50 guys, each one of which is paying people to stand outside Subway, or McDonalds, or Verizon, or the corner bookstore, or whatever happens to be next to that Palm store.
So?
People are not going to go out of their way, or risk jail time, for $1.00/task. You'd need to pony up a lot more money (like, I'm guessing, 100's of thousands of dollars) to get enough people to violate basic principles on a large enough scale for this to create some sort of shift.
After all, for someone to get anywhere doing ill with MTurk the request must be seen by people.
I do think it is very interesting to see the faces behind Mechanical Turk.
On top of the seeming additional liability Amazon would be inflicting on themselves, I can't see any upside to Amazon going to extra lengths. It would anger project creators to have to wait for human approval for every project, it would likely reduce the number of jobs underway at any given time (making it less profitable for them and less lucrative a market for service providers), and it would increase the expense of running the market at human scale rather than computer scale (even if they somehow figured out how to use Mechanical Turk itself to provide the vetting). In short, vetting jobs would be a recipe for a fiery death for Mechanical Turk.
Perhaps considerations of aiding and abetting prostitution, kidnapping and drug-dealing would enter if the job requests were entirely unvetted?
Uh, and I think India has labor laws too.
I am writing her now.
http://www.thestever.net/viewer.php?num=33
http://www.thestever.net/viewer.php?num=34
http://www.thestever.net/viewer.php?num=35
http://www.thestever.net/viewer.php?num=10 http://www.thestever.net/viewer.php?num=14 http://www.thestever.net/viewer.php?num=45 http://www.thestever.net/viewer.php?num=42 http://www.thestever.net/viewer.php?num=41 http://www.thestever.net/viewer.php?num=32 http://www.thestever.net/viewer.php?num=31 http://www.thestever.net/viewer.php?num=30 http://www.thestever.net/viewer.php?num=29
If you're getting $1 for each picture of a person holding a sign you submit, you might as well have everyone else in the room take a turn.
Yes. Neither amount is insignificant. But I still wonder, why only India? People from Pakistan, China and even Sri Lanka , Singapore are in +/- 3 hrs. timezones.
The timing of the post would be crucial to understand this.
I wanted to highlight - the time as a factor for other countries. Timezone.
If a service finds people from a certain country good enough to be 'minions' how come it finds them not good enough to be 'bosses' ?
This is actually a really nice opportunity to give amazon a run for their money and to set up a competing service that operates world-wide on both the give and the take side.
Because of this and similar laws in other countries, Amazon have kept job-posting MT accounts limited to the USA at the moment, which doesn't have the restriction.
I don't know enough about Dutch or Indian law to know whether a similar system could be set up within them, but there are problems with rolling something like this out globally.
Blows my mind how 4 out of 50 (8%) did not even complete the stated requirements.
Also crazy how all many of these people are Indian.
Don't be surprised when people don't take your requirement that seriously.
If you can earn a buck doing what a lot of (generally wealthier, admittedly) people do for free, why not?
For text submissions, you can enforce validation rules (simple and regex). So you don't have to manually reject non-conforming work. You can also have the Turker fill in their work on a webpage that you control, so validation can be much more advanced. I never went all the way but sometimes I think about it.
Our company provides very affordable and accurate photo approval services. Photos are manually approved and we can handle anywhere from 10-1,000,000 photos per day. If you're interested, feel free to email me (email address is in my profile).
The payment for most of the tasks on Mechanical Turk and other things like it is just not worth the time for almost any task I've seen. Which is really sad, because it'd be awesome if it were worth it.
It's interesting to read all the comments that say "$1?!?! You're ripping off these people! Why would anyone ever do anything on Mechanical Turk?" How many hours do y'all spend on Hacker News? How long did it take you to write those comments? How much were you paid for it?
In strictly economic terms, Turkers are getting ripped off a lot less than news.YCers are. After all, $1 is infinitely more than $0. We labor for free so that Paul Graham can have a viable online community and pick of the top posters for his seed capital firm.
Of course you may be being philosophical about it. In which case, no we have no real way of knowing if they even think.
I'm pretty sure that might be above the US minimum wage.
While that may be true for you, there are plenty of places across the globe where $1 (with a chance of $50) is not an insignificant amount.
Oh, maybe not...
As a bonus, he got some nice smiles out of the process ^_^.
http://www.thestever.net/viewer.php?num=33 http://www.thestever.net/viewer.php?num=34 http://www.thestever.net/viewer.php?num=35