Aren't we also machines? If we make theories that discount our own intelligence that's just throwing our hands up and giving up.

Maybe I would agree with you if LLMs weren't already embedded in a lot of useful products. I myself measurably save a lot of time using ChatGPT in such "unintelligent conversations". It's intelligent in the ways that matter for a tool.

IMO you're using the wrong term or have a low bar for 'intelligence'.

It's reasonably good at reproducing text and mixing it. Like a lazy high school student that has to write an essay but won't just download one. Instead they'll mix and match stuff off several online sources so it seems original although it isn't.

That may be intelligence but it doesn't justify the religious like tone some people use when talking about LLMs.

I don't know, it's more then this. I ask ChatGPT to teach me all about Hinduism, the siva purana, ancient indian customs, etc., and it is an incredible tutor. I can ask it to analyze things from a Jungian perspective. I can ask it to consider ideas from a non-typical perspective, etc. It is pretty incredible, it is more then a word gargler.

"teach you all".

So you don't know enough to evaluate the answers. Or where to find the original sources it quoted or mixed.

Original sources aren’t necessarily infallible either, but that’s beside the point, asking ChatGPT needs to be viewed as asking a person to recount something from memory. It hasn’t been reviewed or edited so it’s like a first draft rather than a published document. That’s the error that people tend to make since they treat it like a search engine instead of a conversation.

> they treat it like a search engine instead of a conversation.

That's because web searches are so polluted that for a lot of purposes even ChatGPT 3.5 is better.

Um yes I do? Lol why do you just assume this?

It is, unquestionably, from the math, a "word gargler" as you put it.

https://xkcd.com/1838/

No it's a digital neural network expressing it's inputs and outputs in language.

If you trust the words that machine strings together, you're making dangerous assumptions - look at the lawyers getting busted for submitting fake cases. These things are literally designed to sound plausible as the primary concern - facts aren't really a thing.

> People are talking about ChatGPT hallucinating

It's really unfortunate that some much of the terminology around LLMs is so anthropomorphic; it's extremely misleading.

(People sometimes suggest replacing this one with confabulating, but I'm not really sure that's much better)

Absolutely. I feel like they're essentially Markov chain generators on steroids. Generally entertaining, sometimes useful, ultimately shallow.

I'm really surprised when people, especially technical people, say they "asked" something to ChatGPT, or that they had a "conversation".

It doesn't know anything. It doesn't understand anything. The illusion is very convincing, but it's just an illusion.

I know I'm part of a minority of people who think this way :( The last few months have felt like I'm taking crazy pills :(

Sales people, and company owners, have a vested interest in pushing the narrative that the illusion is actually magic.

I believe that for a large number of applications the distinction between "knowing something" and "containing knowledge" doesn't really matter.

In 2010s Google I could type in a query, and it would redirect me to a human-written answer on something like StackOverflow with hopefully the right answer; in 2024 ChatGPT you can type in a query and it'll mash together some relevant human-written snippets into something which is hopefully coherent, relevant, and correct. Sure, it's "just" a Markov chain generator on steroids, but it's still giving you useful output.

I do agree that it is a bit worrying to what extent people are anthropomorphizing it, and treating it like something with an actual opinion or an unquestionable source-of-truth. The "I asked ChatGPT" comments should be treated the same as "I put it into Google and this was the first result": it's just line noise and doesn't add anything to the conversation. If you don't have anything to add yourself, why comment at all?

You're not alone

I feel the same way. It's frustrating feeling like I'm one of the few people that can see that the emperor has no clothes, and nice to see the occasional person like you that pushes back on this cargo cult fanaticism.

>It doesn't know anything. It doesn't understand anything. The illusion is very convincing, but it's just an illusion.

What would know and understand anything with this vague undefinable metric ? You ? How do i know you know and understand anything ? Can you prove that to me better than GPT can ?

I've been ranting about this for months. The underlying shape of these things is inherently an approximation. We're just guessing but some are acting like we've built the godhead.