It feels like the example was cherry-picked, but the combination of "less accurate" and "more confident" made me immediately think of this disaster. I truly worry that it's going to take civilian deaths / liability before the proper use / place of these new technologies is adopted, but we could probably start with 1) not in safety critical systems, and 2) don't replace your specialists / experts with algorithms.
[1] https://www.engr.psu.edu/ae/thesis/failures/MKP/failures/fai...
But for the people who do actual work with AI, who are the people who pay a lot of money for it, they prefer to be called wrong when they are, because it is better to be called wrong by an AI than being wrong in front of a customer.
It is not an easy problem to fix, because as much as we want an AI to give us the right answer rather than just being agreeable, we still need them to follow orders, and AI that doesn't would be quite useless. So there is a balance to be found. Smarter models tend to do better because they know what is right, compared to smaller models that don't and then assume you are right and hallucinate from there.
I do like the idea of AI personalities ( I actually use them a fair bit for varying opposing perspectives ), but I am not entirely certain people want that.
As someone who rarely uses LLMs because I dont need to, nor does it benefit me - I work on stuff that is original for which LLMs are useless at - Im glad.
I want everyone around me to get dumb as hell. It makes my path in life much easier and more successful.
Closer to you, you don’t want your customers, your coworkers, your friends, and your neighbors to be dumb as hell. Whatever you may gain by being the smartest one on your block is more than offset by the myriad ways in which dumb people make life worse.
Thats the problem with this place - y'all think you are so smart.
Haha. No.
We’ve invented a technology that hijacks that part of humanity more effectively than anything ever. Asking people to “not be so stupid” is not going to help.
And especially their design. The con artist politeness and confidence is not an accident of output, but a design choice by the developers.
I tell non-technical people: It's a software program; there is always a person behind it, and they have consciously chosen this output, to serve their own interests.
> Humans are wired to anthropomorphise; we do it with sticks and rocks.
I do not see people anthropomorphize their TVs, books, paintings, cars, houses, rocks, sticks, ... I'm not sure what you're talking about here. I haven't seen someone call a rock or stick by a pronoun, for example.
I've seen people fancifully do it, but it's exaggeration for a cute effect, and works because everyone knows they're exaggerating.
> We’ve invented a technology that hijacks that part of humanity more effectively than anything ever. Asking people to “not be so stupid” is not going to help.
People learn to look at things in different ways. Humans learn, society changes. It's changed quite a bit in recent years; it's much different than a century ago, a century before that, etc.
In a way, it's more worship of technology to say humans are hopelessly, powerlessly dumb, and technology inevitably rules them. The inevitability of technology is of course a self-serving trope of the industry: you are powerless, we are inevitable. How convenient, and short-sighted.
> In a way, it's more worship of technology to say humans are hopelessly, powerlessly dumb, and technology inevitably rules them.
You as an individual have some autonomy against the machines. Some. (Scroll facebook an hour a day and show me your resilience to engagement engineering). But the point of this is what happens at population scale.
To ignore our biology is as stupid as suggesting we must be ruled by it.
It's not just fanboys doing that, but "official" educational channels. Both an internal training (more like an all-hands knowledge share) and some dumb training session by a guy from Anthropic explicitly, repeatedly recommended users anthropomorphize their chatbots on no uncertain terms. "Treat it as your super-smart coworkers", "I named mine Becky and I go to her with all my work challenges", etc.
Even if AI gets smarter it will still be agreeable, and people will use it to more confidently reinforce thier stupider ideas, especially in areas where they lack the knowledge to know if they are right.
Even if this can be solved, technically, people won't want to use the model that says they are wrong, so they will choose the glib lies that reinforce their beliefs.
Freedown of speech forces people to think they may be wrong, but freedom of association lets people avoid this. AI is going to be internet hug box echo chambers at an unbelievable scale.