Although you could argue some of the non-answers to questions concerning African American diaspora, porn, eugenics etc suggests that the overriding of the AIs responses by the humans running OpenAI has lent it a left-leaning bias.
I'd argue it doesn't work for humans neither. It makes you barely scratch the surface of politics and then you double down on the results without understanding any of the arguments made in the past 200 or so years that lead to the current situation.
It also doesn't help that every quiz I took is just bad. Oh, you're not opposed to same-sex marriage? Boom, lib left. About as anarchist as Emma Goldman who was against prosecuting gay people in the beginning of 20th century (unheard of at the time).
Editors of newspapers are supposed to know how AI models work?
Sorry I don't see how you could possible challenge this article without a deeper understanding of the technology. That is the fundamental critique here is it not?
Expecting every newspaper to have those domain experts on hand for each topic is naive and highly optimistic. The journalism world doesn't work that way for better or worse.
If it did we would have 50%+ articles never written. Probably more.
Doesn't sound like a very bad outcome honestly.
- from the movie Tar
Editors of newsmagazines are supposed to know how journalism works, and to probe journalists to assure that they’ve done the research to properly understand and represent the subjects of their stories. So, yes, when publishing a story about AI models, the editor should know enough about how AI models work to understand that the story is a proper representation, not in advance of seeing the story, but as a result of the dialogue with the reporter on the research that the reporter has done before the editor signs off on the story.
All people are shaped by unique experiences and productions of those experiences can be "consistent" and can hold firm in the face of even contradictory evidence.
But in the absence of deep personal experience, people, like ChatGPT, are argument-producing machines, where the shape of the produced arguments depend more on prompt, social context like the status of other parties in the argument, and personal context like whether someone is hungry or tired, etc.
Thank goodness the for-centuries-incorrect models of people as primarily intellectual creatures are subsiding and the reality of people being storytelling creatures are coming to the fore.
Relatedly, agree completely that Reason lacks informed editing on many topics. Cheers.
Why does the idea of a "political quiz" work for humans? Because humans are at least somewhat consistent - their answer to a simple question like "would you rather have a smaller government or a bigger government?" is pretty informative about their larger system of beliefs. You can extrapolate from the quiz answers to an ideology, because humans have an underlying worldview that informs their answers to the quiz. ChatGPT doesn't have that. If it says "bigger government", that doesn't mean that its free-form outputs to other prompts will display that same preference.
Trying to ascribe a larger meaning to ChatGPT's answers to a political quiz tells me that the author is very confused about what ChatGPT is and how it works. It's too bad Reason doesn't have an editor or something who can filter this sort of nonsense.