>> They will say anything confidently without knowing anything.

Understanding how these algorithms work, I don't know why we would expect anything different from this?

Most ppl don't understand shit about these algorithms and will expect it be truthful and knowledgable. That is how it's marketed, despite the disclaimers..

It should be marketed more like “uncle Jim” than a trustworthy, all-domain competent oracle. I think if you just gave it a twangy accent and changed some output nouns to thingy, whatchamacallit and doohickey, people would actually be able to use it more competently.

Because it's marketed as AI.

For me, my expectations are adjusted. I kinda know what to expect it to do, and it does quite well for what I use it for, essentially smart intellisense for coding.

I would have expected the people who trained these systems to not format their Q/A samples to sound authoritatively while they knew the system was going to be spouting nonsense.

The sheer amount of companies, media, influencers, heck even government agencies, acting otherwise? With billions of dollars on the line, there are many powerful forces pushing for people to have unrealistic expectations about LLMs (or "AI" as they call it).

A lot of money is being spent on trying to improve it too. But right now the emperor has very little clothes. It remains to be seen whether they have materialized by the time people open their eyes.

Perfectly humanlike

Indeed, the more I use LLMs and see the confabulated output, I've started seeing similar in how actual people talk.

We gave up the pursuit of pure crystalline Intelligence and instead just remade ourselves. Hopefully the AIs will be able to do better.

Yeah. We don't need Artificial Intelligence. We need to discover Actual Intelligence :)

My favorite is “mansplaining as a service”

LLMs are really disrupting the market for Mckinsey junior analysts meeting with clients.

Can an LLM provide the sort of psychological buffer that McKinsey Junior Analysts (some random person picked up off the street in a college town and handed a clipboard, a frequent flier card, and a six figure salary) provide to senior managers against the scrutiny of their board and chief executive?

If I want to close the paper division, and my boss wants to close the paper division, does the LLM whisper sweet nothings in our ear and show soothing Powerpoint decks about how in the eyes of an objective observer, the paper division is unprofitable and there would be no career risk in closing it?

It's not a sexual thing I swear, it's a sort of comforting intimacy that needs to be injected into the business process in order to cope with the pressures executives feel. I don't know that anybody has trained an LLM to do that yet.

I think this is the next sequel to the movie "Her".

You bring up a good point though. Very dystopian idea of a new type of enterprise tooling that emerges to soothe and calibrate human knowledge workers. Very Severance Wellness room.