Your arguments are all over the place and frankly I don't understand what is the problem you are trying to say. Something can be both undesirable and normal. In fact, I would even say that most of the time, the most normal outcome is a failure and we have to put in extra effort to make it a success.

I have seen plenty of bright and intelligence students who most of the time are just parrots of what were told in class or from some other sources. It is actually highly unusual for a high school student or even a freshman or sophomore in college to have an absolutely original and correct idea. That doesn't mean I look at my students as unintelligent, even the below-average ones.

And here comes the crux of the issue. You automatically assign the ability to "reason" to a human even when we make mistakes aplenty. But when a machine does it, you said that is a proof of unintelligence. This is what the OP was criticizing in their post. Most deniers are using arbitrary requirements applied only on the AI and not humans.

After teaching quite a few college level kids, I dare to say that the AI models right now exhibit reasoning ability close to many human. Doesn't mean they are sentient or human-level yet, but they are definitely much much better than what I would ever expect out of dumb silicon and steel.

> You automatically assign the ability to "reason" to a human even when we make mistakes aplenty. But when a machine does it, you said that is a proof of unintelligence ... requirements applied only on the AI and not humans

Whereas I apply them requirements also to humans? :)

I must correct you, to help understanding: I do assign in general (I hope, to all individuals but for an n-th standard deviation) «the ability to "reason" to a human»; that «we make mistakes aplenty» is overriding (or under-using) a faculty which was present but, in those cases, was not exercised.

You write «when a machine does it»: but can the machine do it? Can it criticize its ontological contents, its built concepts? Humans can, and pretty much must - deontically. The machine should.

> we have to put in extra effort

Exactly. We can, many don't, and the machine should - just like people in general should.

> the problem you are trying to say

While people do not always exercise critical thinking, they should. Verifying, trying your ideas and concepts, making them solid, should be a constant effort. Humans have those modules, which makes them apt to possibly be employed as "predicting machines" (which is a definition of intelligence), or "things you can ask things".

When you build a machine that produces outputs, they may be "for fun", so every generator goes that satisfies the "fun" based "performance indicator", but more normally you will want the output to be reliable. So, the "verification" features are crucial.

> Something can be both undesirable and normal

In engineering, I hope not :)

If I understood you correctly, you are asserting that the LLMs entirely lack the "self-awareness" or "critical thinking" part while human simply doesn't use it enough but we are all born with it and always have that ability as a built-in feature, inherent to our beings?

If so, then you are right on the machine's part. The current LLMs can't change its own weights thus in a sense, incapable of questioning its own existence. It can't perform a highly philosophical debate with its own "self" and reform its mind as a result of that action.

What I am still arguing is many human can't do that either. Such thing, in my experience, is actually not that common in the general populace. And while you can say it is there, just "unexercised", that would be functionally identical to it not existing and you can't absolutely prove that "it is still there" in all human.

>In engineering, I hope not

Perhaps in your particular type of engineering. In civil engineering, material degradation is undesirable and normal. In chemical engineering, side reactions are undesirable and normal. In biomedical engineering, well there are too many to list, from undesirable epigenetic changes to patients' unhealthy lifestyles. Things can definitely be undesirable but normal.

> From a calculator you do not expect "«pretty»" reliable, but just reliable. From a human you expect faulty computation because of the economization of resources, but you know it can reason.

Just have it use Wolfram Alpha plugin. Not an issue.

> Again (and again, and again), this argument is the mysteriously recurrent "many people fail", implying that failing would be normal, instead of undesirable.

That doesn't really answer the question. Why is the immediate expectation, when AI is in its infancy, that it must outperform humans in the ability to be original and always factually accurate? That's not a reasonable expectation when there are not only no examples that it's even possible, but we ourselves aren't original. There's a reason why both computers and human beings have supervisors. What about AI suggests an intelligence that can or should be free of supervision?

> It is also the mark of unintelligence. As such, it is a warning alarm.

That depends on a matter of degree. You and I parrot things all the time. We neither have the time or capability to not take information for granted and pass it along without fully understanding it. What we do is best effort, which is far more achievable than "perfect" computation.

For instance, I could ask you to paint us a painting of literally anything, and chances are your painting will be based on your experience of what a "painting" is and what you paint will probably be fairly unremarkable on a grand scale, no matter how much you try. I don't mean this as an insult, because it applies to virtually everyone and it is what it is. Of course there is the possibility that you will prove me wrong and paint something that no one has painted before using a style or method that is completely new, but I won't bet on it.

In such a case, you'd be "parroting" what artists have been doing since time immemorial. Does that actually devalue your art? Or the art of practically every other artist who mostly painted things based on previous ideas? I don't think so, but that's just me.

> From a calculator you do not expect "«pretty»" reliable, but just reliable. From a human you expect faulty computation because of the economization of resources, but you know it can reason.

Yeah... because calculators don't reason anything.

> The expectation is that of a reliable process bringing to the output.

What is your best example of how the current generation of LLMs aren't reliable enough for a wide variety of tasks?

> when AI is in its infancy

Well, having started in 1955, we really must frame that "infancy" in a scale like dog-years, human-years, AI-years...

> that it must outperform humans in the ability to be ... accurate

No, the expectation of some of us was not about «outperform[ing]», but about not losing the focus of «accura[cy]».

> a matter of degree

It does not seem a «matter of degree» but its opposite, a "matter of quality": does the machine question its statements?

> You and I

Actually, the process is that of building on already vetted ideas, and continuously vetting. True, there is an economy of effort, but there is also a capitalization on past effort. // It is unfair to say that you "«parrot»", just because of early stopping, assuming you are trained to vet your output.

> new ... what artists have been doing ... previous ideas

You should revise your perspective there: we learn from the (bigger and smaller) "giants" in the past, and learning is very different from plain imitation. What is learnt is internally elaborated, understood, integrated. There is a whole world of difference from "I learnt from A" and "Z copied A".

> calculators don't reason

But calculators bring a reliable result, and that is what is expected from them. You expect the same from intelligent entities.

> What is your best example

Decision Support Systems. (Edit: I meant as a support Analyst; as an information retriever it could instead be a dramatic asset, if employed as a hound to retrieve sources.)

> You should revise your perspective there: we learn from the (bigger and smaller) "giants" in the past, and learning is very different from plain imitation

Ah, I see. So when an AI imitates the things it was trained off of it's simply parroting the material. But when we imitate things we've trained off of, we're synthesizing new information that we've learned from. /s

This ambiguity is exactly what OP sounds like they're talking about in regards to intelligence. Who's to say what parroting is and what synthesizing new information from old ideas is? Where is the line? Can we score how "parroty" an idea is? Can we quantify how much intelligence is imbued in a written statement? There is no firm science (that I know of) that quantifies intelligence, but it sounds like we'll be needing to determine hard metrics to find out if AI is truly intelligent or not. Otherwise, we'll have these circular arguments forever that basically dismiss an AI as being intelligent simply because it's an AI.

> when we imitate

Also in humans imitation as copying and imitation as intelligent inspiration are discriminated.

Or to put it another way: new styles of art were historically invented every century or so, and a bit faster these days, maybe every decade. All of the other art made by thousands of artists were derivative in some sense. It’s a rare master that can come up with something entirely new.

You see similar things in areas like science. When was the last time you’ve seen a paradigm shift akin to relativity or quantum mechanics? Not for over a century!

These are just some subjective, odd philosophical arguments.

Question is - does this tool provide massive value and productivity boost? Yes.

> does this tool provide massive value and productivity boost

And the «philosophical arguments» there highlight the other side, about the limitations. Are they relevant? Surely, if they are not evident to the user.