I doubt the experiment is set up that way, but that would be an ethical way to do it.
I don't think we need to have real human risk to get results from the experiment.
“John and Jill are not at risk. This is a controlled experiment and everyone working at Andon Market is formally employed by Andon Labs, with guaranteed pay, fair wages, and full legal protections. No one’s livelihood depends on an AI’s judgment alone.”
which was refreshing to read.
Personally I find the entire tone of the article to be creepy and disturbing.
There was a scifi story about a guy who gradually falls through the cracks of a dystopian future society in which McDonalds managers are replaced by AI that talks to workers through their headsets.
At first it's quite benign, like: "Hello, John. In 5 minutes it will be time to inspect the washrooms and perform any necessary cleaning."
Before long it's firing people who don't smile enough and don't have the correct attitude.
(Of course, to keep readers from becoming despondent and killing themselves, the story takes a hard left turn towards a post-scarcity economy and everyone lives happily ever after. But when one reflects on it at the end, 90% of humanity doesn't have that post-scarcity life. And those who get left behind are far from content with their futures....)
That doesn't mean the AI couldn't be the decision maker for the legal entity that's hiring these people.
But the thing is that if this startup is telling these people they are employees of this company, not "Luna", it would give these people the impression that all their interactions with the AI are kind of a sham, a game, not to be taken seriously and they are basically being paid to role-play as "Luna's employees".
And this kind of where such experiments are likely to go. Another user mentioned that it would be useful to discover the kind of inputs and output the machine. A human boss could manage a store with just phone calls and a camera but I overall get the vague impression Luna doesn't have anything like that sort of ability, though really we just aren't given the information for any accurate determination.
I'm not sure what sort of labor regulations exist in San Francisco, but presumably they can be fired as easily by an AI as a real person, right? If Luna decides to fire them, and it can do so, then their livelihood does rather depend on an AI's judgement alone.
Unless of course all of its decisions are vetted by humans - as they should be - which makes this experiment a lot weaker than they're saying it is.