beanbean01

Here's the problem with LLMs: https://arxiv.org/abs/2301.06627 Basically, they're missing a lot of the brain machinery required to function. For example, if you ask them to solve a math problem, they do just fine ... until you ask them to apply an inference rule on top of it that takes them outside of their training set. The result is something that LOOKS like AGI until you realize it's read the entire Internet.

I suspect that a great many of the people who say AGI is just around the corner and are giving <2030 timelines are saying that because they want it to be around the corner.

Yes. Thank you. This is one of the things that really worries me about aligning AI.

If you're even worrying about how to align human level intelligence, you don't have the capacity to ALIGN THE HUMANS towards the goal of creating safe AI.

In a situation in which all of them are either A) willing to take massive risks on unaligned AI or B) being dragged along by those which are.

And then you are, as they say, "screwed".

THANK YOU. You've stated the difference perfectly. Humans have a concept of truth or grounding, LLMs do not.

Yes, but there's a critical difference. When humans hallucinate, it's normally because they misremember something: did you know, for instance, that our brains have to piece every memory together from scratch every time we recall something? It would be a miracle if humans did NOT hallucinate.

In other words, the human has a concept of truth or facts and simply had a memory lapse. This thing has no concept of truth at all.

If we solved the problem of unlimited greed, we wouldn't need unlimited power in the first place.

Here's my theory: if you look at surveys, it does say a 10% chance or so of an extremely bad outcome. BUT it says a ~20% chance of an extremely good outcome, and an 80% chance of at least a neutral one. Simple cost benefit analysis.

To properly explain this would take longer than the length of the comment limit (is there a length limit? I don't know, but even if there isn't I don't feel like explaining this for the 70th time), but here's why: chrome-extension://efaidnbmnnnibpcajpcglclefindmkaj/https://arxiv.org/pdf/2301.06627.pdf To sum up: a human can think outside of their training distribution, an LLM cannot. A larger training distribution simply means you have to go farther outside the norm. In order to solve this problem would require multiple other processing architectures besides an LLM, and a human-like AGI cannot be reached by simply predicting upcoming words. Functional language processing (formal reasoning, social cognition, etc) require other modules in the brain.

This explains the various pejorative names given to LLMs - stochastic parrots, Chinese Rooms - etc, etc, etc.

Depends on the researchers. Mathematicians might mind, but I am going to guess (assuming there was a plan in place to make sure they didn't wind up on the street) climate researchers wouldn't mind being made obsolete tomorrow.

Actually ... that's a reasonable goalpost, in my opinion. Yes, humans make careless mistakes. However, humans mostly make careless mistakes because A) their brains have to reconstruct information every time they use it, or B) they are tired. LLMs, as piles of linear algebra, have neither excuse. Their training data is literally baked into their weights, and linear algebra does not get tired.

Thank you for this.

This is one of the things that bugs me about the AGI debate when people argue over when/if computers will match human intelligence.

Just one small problem: we don't have a working definition of human level intelligence!

"OpenAI" does not believe that it will be AGI. There is an interview farther down in the thread in which Altman states that A) GPT-5 will not be AGI and B) the GPT paradigm has some fundamental problems which will not get it to AGI.

Later down, Siqi clarifies several things:

1. That he believes AGI is a continuum, and encompasses most stuff that isn't a narrow system. So what he is ACTUALLY saying is "GPT-5 is not a narrow system, and is near the bottom of the AGI continuum".

2. That this is NOT a consensus belief within OpenAI, but rather one only shared by a "non-zero number". This should be a BIG RED WARNING SIGN, because it likely means that the number is very low and he is puffing it up/twisting words.

3. Seeing as he went on to actively deny that it was from someone making it up, that means that it's likely this is just his opinion and maybe a few of his friends.

4. He went on to answer the "agency" question by stating that one could attach a goal loop to it. So a prototype paperclip maximizer. Greeeeeeat.

And regarding ASI ... seeing the destruction and stupidity that an overgrown autocomplete can cause, maybe, just maybe, it's a good thing if we aren't on track for that right now.

Yeah, I think he's just making this up and this is his opinion. Later down he says "someone who didn't; (source, I made it up reaction image)". And I'm thinking: if you have to pointedly say that it's not made up, it's probably made up. For what it's worth, there's an interview slightly farther down in which Altman says that he does not think GPT-5 will achieve AGI, and that no GPT will, likely, because he thinks an AGI needs some degree of autonomy and "want".

Because they are being deployed wrong. We should be lining up our transition to post-work so we can spend as much time on art and science as we damn well please, and wholly ignore commercialized drudgery. Instead we are using it to commercialize art and science and turn it into drudgery which we already said we would automate.

Stupidity singularity achieved.