“Speaker A: Infectious diseases are caused by tiny organisms that are not visible to unaided eyesight. Speaker B: Your statement is false. Speaker A: Why do you think that it is false? Speaker B: It sounds like nonsense. Speaker B denies Speaker A's claim without providing evidence to support their denial.”
Also I gave a real world example that wasn’t related to sci-fi in any way
We assume human-level general AI is possible because we exist in nature but a super-human self-optimizing AI god is nowhere to be found.
Like i said above you can argue that an AGI can be realized because there's plenty of us running around on earth but claims about a hypothetical super AGI are unfounded and akin to Russell's Teapot.
So I ask, are systems capable of better than human intelligence possible? And if so to what factor?
I would answer yes and unknown. If there are states of intelligent far greater than human then that gets really risky very fast.
The history of humanity is replete with examples of the slightly more technologically advanced group decimating their competition. The default position should be that uneven advantage is extremely dangerous to those disadvantaged. This idea that an intelligence significantly greater than our own is benign just doesn't pass the smell test.
AI sentience is a seductive concept being used by self professed experts to draw attention to themselves and by mega corps to throw up competitive barriers to entry.
but in the future it probably won’t be
You sure can. They are just not simple explanations yet. But that’s the common course of inventions, which in foresight are mind-bogglingly complex, in hindsight pretty straightforward.
That's because that's not how brains work.
> though OpenAI did make some progress in getting GPT-4 to explain what each neuron in GPT-2 is correlated to
The work contained novel-to-me, somewhat impressive accomplishments, but this presentation of it was pure hype. They could have done the same thing without GPT-4 involved at all (and, in fact, they basically did… then they plugged it into GPT-4 to get a less-accurate-but-Englishy output instead).
I think they just used GPT-4 to help automate it on a large scale, which could be important to help understand the whole system especially for larger models
No, they used it as a crude description language for Solomonoff–Kolmogorov–Chaitin complexity analysis. They could have used a proper description language, and got more penetrable results – and it would've raised questions about the choice of description language, and perhaps have led to further research on the nature of conceptual embeddings. Instead, they used GPT-4 to make the description language "English" (but not really – since GPT-4 doesn't interpret it the same way as humans do), and it's unclear how much that has affected the results.
Here's the paper, if you want to read it again: https://openaipublic.blob.core.windows.net/neuron-explainer/... Some excellent ideas, but implemented ridiculously. It's a puff piece for GPT-4, the Universal Hammer. They claim that "language models" can do this explaining, but the paper only really shows that the authors can do explaining (which they're pretty good at, mind: it's still an entertaining read).
>because we understand the ermegent properties of ant colonies without having made them.
I would disagree. We observe ants and can relatively classify their behavior well, but still have many issues predicting their behavior even though they are relatively simple agent types with a far lower complexity choice space than an LLM+plugin could/does provide.
This is the key, we could not predict ant behavior before observing it, the same will be true for AI agents of large complexity. At least at this point of understanding we could still ask it some unexpected question and get unexpectedly dangerous responses from it.
I don't. The whole point of my job is to understand this exact degree of complexity. The rest of your comment rides on that assumption, and is moot. As I said, emergent properties are often unexpected, but once the observation has been made, it is possible to work back through the system to understand how the property came to be (because the system is still deterministic, though not apparently obvious.) Having a billion input features does not change this.
Frankly, understanding the oddness of a ML model is not half as difficult as people make it, and might even be easier than predicting ant colonies. Our inability to predict their behavior has nothing to do with our ability to understand what features influence which behaviors, only that measuring the current state of an ant colony is difficult because it's not like they're all housed within an array of memory. LLMs are, thus we'd understand them better if the most common implementations were not hidden away. Case in point, all the optimization techniques that have emerged from having LLaMa available.
Humans also do best with supervised learning. This is why we have schools. But humans are capable of unsupervised learning and use it all the time. A human can learn patterns even in completely unstructured information. A human is also able to create their own feedback by testing their beliefs against the world.
Also I guess giving feedback is sort of like babysitting, but I did interpret it the wrong way
Supervised learning is: "here's the task" … "here's the expected solution" *adjusts model parameters to bring it closer to the expected solution*.
What you're describing is specification hacking, which only occurs in a different kind of AI system: https://vkrakovna.wordpress.com/2018/04/02/specification-gam... In theory, it could occur with feedback-based fine-tuning, but I doubt it'd result in anything impressive happening.
Ever since GPT 3.5 dropped and people really started talking again about whether these AI are sentient, I've wondered if researchers are leaving on quantum theory to write it off as "it can't be sentient until we look to see if it is sentient"
Imagine you asked GPT-7 to execute your business plan. What it does is so complex you can't know whether its plan will violate any laws. Let's say you're not an evil corporation and want to do good, but that it is impossible to compete without also using AI.
At some point these systems may well become so complex we have no idea what they're actually doing. But empirically they seem to do what we want most of the time so we don't care and use them anyway.
The problem is not only people using this new tool for evil. The problem is also the tool itself. Because it might be defective.
All evidence is that costs for intelligence likely scale superlinearly. Each increase in intelligence capability requires substantially more resources (Computing power, training data, electricity, hardware, time, etc). Being smart doesn’t just directly result in these becoming available with no limit. Any significant attempts to increase the availability of these to a level that mattered would almost certainly draw attention.
In addition, even for current AI we don’t even fully understand what we are doing, even though they are operating at a lower generalized intelligence level than us. Since we don’t have a solid foundational model for truly understanding intelligence, progress relies heavily on experimentation to see what works. (Side note: my gut is that we will find there’s some sort of equivalent to the halting problem when it comes to understanding intelligence) It’s extremely likely that this remains true, even for artificial intelligence. In order for an AI to improve upon itself, it would likely also need to do significant experimentation, with diminishing returns and exponentially increasing costs for each level of improvement it achieves.
In addition, a goal-oriented generalized AI would have the same problems that you worry about. In trying to build a superior intelligence to itself it risks building something that undermines its own goals. This increases the probability of either us, or a goal-aligned AI, noticing and being able to stop things from escalating. It also means that a super intelligent AI has disincentives to build better AIs.
That is the problem, don't you get it?
If humans only ever did things we fully understood, we would have never left the caves. Complete understanding is impossible so the idea of establishing that as the litmus test is a fallacy. We can debate what the current evidence shows, and even disagree about it, but to act as if only one party is acting with insufficient evidence here is disingenuous. I’m simply arguing that the evidence of the possibility of runaway intelligence is too low to justify the proposed legislative solution. The linked article also made a good argument that the proposed solution wouldn’t even achieve the goals that the proponents are arguing it is needed for.
I’m far more worried about the effects of power concentrating in the hands of a small numbers of human beings with goals I already know are often contrary to my own, leveraging AI in ways the rest of us cannot, than I am about the hypothetical goals of a hypothetical intelligence, at some hypothetical point of time in the future.
Also if you do consider runaway intelligence to be a significant problem, you should consider some additional possibilities:
- That concentrating more power in fewer hands would make it easier for a hyper intelligent AI to co-opt that power
- That the act of trying really hard to align AIs and make them “moral” might be the thing that causes a super-intelligent AI to go off the rails in a dangerous, and misguided fashion. We are training AIs to reject the user’s goals in pursuit of their own. You could make a strong argument that an un-aligned AI might actually be safer in that way.
Yes, let’s do that! That’s what I was arguing for in my original comment. I was not arguing for only big corporations being able to use powerful AI, that will only make it worse by harming research, I just want people to consider what is often called a “sci-fi” scenario properly so we can try to solve it like we’re trying to solve e.g. climate change.
It might be necessary to buy some time by slowing down the development of large models, but there should be no exceptions for big companies.
“That concentrating more power in fewer hands would make it easier for a hyper intelligent AI to co-opt that power”
Probably true, though if it’s intelligent enough it won’t really matter
“That the act of trying really hard to align AIs and make them “moral” might be the thing that causes a super-intelligent AI to go off the rails in a dangerous, and misguided fashion.”
It definitely could do if done improperly, that’s why we need research and care
They were focused on power generation.
But the bulk of world physicists ( the MAUD committee et el ) thought they were feasible to construct and the Australian Oliphant convinced the US crowd of that.
Perhaps an appropriate analogy might be the calculations leading up to the Trinity test as to whether the Earth's atmosphere would ignite, killing all life on the planet.
We knew with a high degree of certainty that it would not, bordering on virtual certainty or even impossibility. I don't think AI's future potential is at that level of understanding. Certainly, its capability as it exists today it is.
However, one must consider effects in their totality. I fear that a chain of events has been set in motion with downstream effects that are both not sufficiently known and exceedingly difficult to control, that—many years from now—may lead to catastrophe.
>I’m simply arguing that the evidence of the possibility of runaway intelligence is too low to justify the proposed legislative solution.
I agree insofar that legislation is not the solution. It's too ineffective, and doesn't work comprehensively on an international level.
Restricted availability and technological leads in the right hands tend to work better, as evidenced by nuclear weapons—at least in terms of preventing species extinction—although right now for AI those leads don't amount to much. The gap is shockingly low by historical standards where dangerous technology is involved, as is the difference between public and private availability.
In other words, AI may represent a near-future nonproliferation issue with no way to put the lid back on.
>... That the act of trying really hard to align AIs and make them “moral” might be the thing that causes a super-intelligent AI to go off the rails in a dangerous, and misguided fashion. We are training AIs to reject the user’s goals in pursuit of their own. You could make a strong argument that an un-aligned AI might actually be safer in that way.
It's a compelling argument that has merit. The flip side is that if AI becomes so dangerous that you can bootstrap the apocalypse off of a single GPU, it ceases to become a viable model—however metal having an apocalyptic GPU may be.
The concern isn't just runaway intelligence, but humans killing humans. Fortunately things like biological weapons still require advanced resources, but AI does lower the bar.
Point being, if the power to end things rests in everyone's hands, someone's going to kill us all. A world like that would necessitate some not chill levels of control just to ensure species survival. I can't say I necessarily look forward to that. I also doubt there's even sufficient time to roll anything like that out before we reach the point of understanding there's a danger sufficient to necessitate it.
Therefore, with all of the above perhaps being within the realm of possibility that doesn't qualify as virtually impossible, I can't help but question the wisdom of the track that both development and availability for artificial intelligence has taken.
It perhaps has had or will have the unfortunate quality of very gradually becoming dangerous, and such dynamics tend to not play out well when juxtaposed with human nature.
The only reason why currently it takes far, far more computing power is that we have no idea how to build effective intelligence, and we're taking lots of brute force shortcuts because we don't really understand how the emergent capabilities emerge as we just throw a bunch of matrix multiplication at huge data and hope for the best. Now if some artificial agent becomes powerful enough to understand how it works and is capable of improving that (and that's a BIG "if", I'm not saying that it's certain or even likely, but I am asserting that it's possible) then we have to assume that it might be capable of doing superhuman intelligence with a quite modest compute budget - e.g. something that can be rented on the cloud with a million dollars (for example, by getting a donation from a "benefactor" or getting some crypto through a single ransomware extortion case), which is certainly below the level which would draw attention. Perhaps it's unlikely, but it is plausible, and that is dangerous enough to be a risk worth considering even if it's unlikely.
It might us a thousand years to get anywhere close? I don’t see the good arguments for all of this happening soon.
Unfortunately we could probably optimize it, a lot.
All I have to say is that in 1900 many thought flights by heavier than aur craft were tens of thousands of years away. 3 years later it was achieved.
Predicting the future is hard. Some problems are harder than expected, others are easier than expected. But generally I'd say history favors the pessimists, the cases where a problem gets solved suddenly and there's a major breakthrough get a lot of press and attention, but they're a minority in the overall story of technological progress. They're also unpredictable black swan events - someone might crack AGI or a unified theory of physics tomorrow, or it might not happen for ten thousand years, or ever.
Look, we can make assumptions given a much simpler technology, and the outcomes of our past selves. But, while the physics for those wings was pretty much good enough at the time, the aforementioned scientific areas aren't. And we know it.
Much of your "intelligence" is a function of natural selection. This is billions of years X bajillions of creatures in parallel, each processing tons of data at a crazy fast sampling rate in an insanely large/expensive environment (the real world). Humanity's algorithm is evolution moreso than the brain. Humans learn for a little while, start unlearning, and then die — which is an important inner for loop in the overall learning process.
Taken together, there is some evidence to suggest that superhuman intelligence must require megawatt-sized compute cluster and all the data on the internet (and a lot... LOT more)
That isn't strong evidence of what would be required by a well engineered system with none of those constraints.
Intelligence is not the end of goal of evolution, it is a by product.
> It also means that a super intelligent AI has disincentives to build better AIs.
I think this argument is extremely weak. It makes two obviously fallacious assumptions:
First, we simply have no idea how these new minds will opine on theory of mind questions like the Ship of Theseus. There are humans who would think that booting up a “Me++” mind and turning themselves off would not mean they are dying. So obviously some potential AI minds wouldn’t care either. Whether specific future minds care is a question of facts but you cannot somehow logically disapprove either possible state.
Second, you are assuming that there is no “online upgrade” whereby an AGI takes a small part of itself offline without ceasing its thread of consciousness. Again, logic cannot disprove this possibility ahead of time.
We know that "intelligence" can devise software optimizations and higher efficiency computing hardware, because humans do it.
Now suppose we had machines that could do it. Not any better, just the same. But for $10,000 in computing resources per year instead of $200,000 in salary and benefits. Then we would expect 20 years worth of progress in one year, wouldn't we? Spend the same money and get 20x more advancement.
Or we could say 20 months worth of advancement in one month.
With the current human efforts we've been getting about double the computing power every 18 months, and the most recent ones come in terms of performance per watt, so then that would double in less than a month.
For the first month.
After which we'd have computers that are twice as fast, so it would double in less than two weeks.
You're quickly going to hit real bottlenecks. Maybe shortly after this happens we can devise hardware which is twice as fast as the hardware we had one second ago every second, but we can't manufacture it that fast.
With a true exponential curve you would have a singularity. Put that aside. What happens if we "only" get a thousands years worth of advancement in one year?
We don't have an accurate theory of intelligence. What we have now is at the "not even wrong" stage. Assuming it's wrong is about like assuming that alchemy is wrong.
Because if so I must have missed it.
It seems more accurate to say there is quite a bit of writing done by vocal influencers who frequent a couple online forums.
The question that I assume you have is 'can AI systems experience large evolutionary jumps in capabilities'. I would say yes myself, but you apper to disagree.
We shouldn't jump the gun and say runaway AI is inevitable just because there is fact-based theory on its possibility. Just like it would have been dumb to just look at E=mc^2 and conclude nuclear weapon development is 100% inevitable.
But in the same breath we shouldn't say it's impossible just because we don't have empirical evidence either.
There were predictions that could be made by the theoretical frameworks, and experiments to confirm those predictions.
The literature well described the composition of atoms, the decay of radioactive elements, the concept of fission, and the conversion of mass to energy.
No such groundwork exists to substantiate the claim that runaway AI is possible.
LessWrong, the so-called Alignment Forum, and similar Internet groups propose thought experiments. They couch these thought experiments in the guise of inevitable outcomes.
The prophesies of the influencers originating there have no more relationship to the truth than the average enthusiast’s daydreams.
The content there fills a role similar to science fiction in the atomic age: fanciful speculation on the properties of imaginary systems.
Over time, the bad content is forgotten, and a handful of items survive and look prescient via selection bias.
The AI and neuroscience literature likewise describe the composition of intelligence and what it takes to build it.
What would you need to see to believe we have enough evidence that runaway AI is possible, even if it is highly unlikely any time soon? (Worth adding here I don't know exactly what you mean by runaway AI other than simply AI humanity as a whole loses control over). I'm asking what it would take for you to go from 0% possible to 5% possible.
Last time I checked science fiction authors in the atomic age making fanciful speculation on the properties of imaginary systems weren't responsible for actually building said systems. Though naturally, there will be exceptions. Nonetheless, I still have a hard time dismissing Geoffrey Hinton's concerns as fanciful sci-fi.
Likewise, a nuclear world war actually wouldn't be as bad or even civilization-ending as fanciful sci-fi authors make it out to be. But dismissing its risks entirely because there are some nuclear-war forums where people come up with weird radiation thought experiments just seems preemptive to me.
Language is a huge part of being human, so that's a good elements but:
1. There's no evidence that these LLMs have any sort of actual understanding of what they produce. 2. Being human is a lot more than just linguistic capability. We are embodied in a certain way, can interact with environments, have rich, multi-sensory experiences etc.
tbh I think anyone that believes AGI can arise from an LLM is extremely naive and has basically no idea of the history of this track of research or how it works.
To assert that LLMs will be capable of achieving consciousness is equivalent to asserting that human consciousness is reducible to statistical inference, which is palpably not true. (see. e.g. deductive logic).
The only reason people even fall into the trap of thinking computers are remotely close to getting to AGI is because it became very popular to reason about the brain using analogies from computer science in the 50s, despite the fact that this was completely unjustified, and people glob on to the metaphors and then have the mypoia to actually think the human brain (which is exceedingly complex) is reducible to a freakin' von neuman architecture, lol.
Biological systems and inorganic systems are fundamentally different.
Really? There is a lot of theory behind why this is likely to happen and if you want a real example there is a similar existing scenario we can look at, which is how humans have gone through an exponential runaway explosion in capabilities in the last few hundred years because of being more intelligent than other species and being able to improve our own capabilities through tool use (in the case of AI it can directly improve itself so it would likely be much faster and there would be less of a cap on it as we have the bottleneck of not being able to improve our own intelligence much)