If AI would be a common great filter we'd expect at least one of them to expand outwards after being the filter?
After all, an AI that can destroy a civilization isn't necessarily "intelligent" in the same way humans are. Or even capable of self-improvement. It could be sophisticated enough to (accidentally?) destroy its creators, but no more than that, and without evolutionary pressure, it may let itself die.
This can change, with the fast advent of Fusion (net positive shown at the end of 2022) and AI (first glimpses of AGI in the begging of 2022).
And yes, we definitely should not allow a madman with a supercomputer (like Musk or Putin or …) to outcompete more reasonable players.
But that doesn’t mean that I’d like him to be the absolute ruler with a superior AI tech. He thinks too much of himself and he’ll make mistakes.
That’s an interesting thought, one that I would give more consideration to in the early days of Musk. However, given Musk’s increasingly intense and emotional public outbursts, I’m more inclined to believe his concern is less about AI safety, than it is about his ego being damaged for not being the one leading OpenAI.
If anything, he's using his wealth to solve the wrong problems, and has sucked up taxpayer resources to do so
When was the first time Putin did? According to my knowledge, it was just last year. Putin is 70 years old now and has been in control of Russia for over 20 years.
In short, Putin wasn't always this bad. He's gotten worse over the years.
Musk is now roughly the same age Putin was when he took power. If he somehow gains control over the resources of a nation like Putin did, he could be far worse than Putin in 20+ years.
The OP wasn't claiming that today's Musk is just as bad as today's Putin; he's just making examples of people with great potential for harm.
A lot of things CAN happen but I'm confused when people state things as if they WILL. If you're that much of an oracle tell me which stonk to buy so I can go on holiday.
The incentives of capitalism and government determine if or how climate change will be solved, and I have approximately zero agency in that
It's not. At least with the nukes there's a chance of resetting civilization.
> After all, an AI that can destroy a civilization isn't necessarily "intelligent" in the same way humans are. Or even capable of self-improvement. It could be sophisticated enough to (accidentally?) destroy its creators, but no more than that, and without evolutionary pressure, it may let itself die.
It doesn't even need to be AGI. It could be that some less-advanced "AI" technologies of a certain sophistication create perverse incentives or risks that cause their parent civilizations to collapse.
Think asshole billionaire hording all productive the resources, but doing nothing useful with them, while the rest of civilization starves and collapses. Or, AI becoming an irresistible opiate that causes individuals to retreat into some unproductive state then eventually die (e.g. into some VR videogame or something). Or weapon of mass destruction trivially created and deployable by any old wacko.
At least for an LLM to act on its own volition rather than implementing it's operator's goals.
The LLM is happier to pretend that it escaped, and respond to the operator as though it escaped, than to actually do some escape.
It doesn't have an interest beyond responding with the auto-complete text. The operator has the interest
(aka The Transcension Hypothesis)
https://www.sciencedirect.com/science/article/abs/pii/S00945...
https://en.wikipedia.org/wiki/Frank_J._Tipler#The_Omega_Poin...
>The Omega Point cosmology
>The Omega Point is a term Tipler uses to describe a cosmological state in the distant proper-time future of the universe.[6] He claims that this point is required to exist due to the laws of physics. According to him, it is required, for the known laws of physics to be consistent, that intelligent life take over all matter in the universe and eventually force its collapse. During that collapse, the computational capacity of the universe diverges to infinity, and environments emulated with that computational capacity last for an infinite duration as the universe attains a cosmological singularity. This singularity is Tipler's Omega Point.[7] With computational resources diverging to infinity, Tipler states that a society in the far future would be able to resurrect the dead by emulating alternative universes.[8] Tipler identifies the Omega Point with God, since, in his view, the Omega Point has all the properties of God claimed by most traditional religions.[8][9]
>Tipler's argument of the omega point being required by the laws of physics is a more recent development that arose after the publication of his 1994 book The Physics of Immortality. In that book (and in papers he had published up to that time), Tipler had offered the Omega Point cosmology as a hypothesis, while still claiming to confine the analysis to the known laws of physics.[10]
>Tipler, along with co-author physicist John D. Barrow, defined the "final anthropic principle" (FAP) in their 1986 book The Anthropic Cosmological Principle as a generalization of the anthropic principle:
>Intelligent information-processing must come into existence in the Universe, and, once it comes into existence, will never die out.[11]
>One paraphrasing of Tipler's argument for FAP runs as follows: For the universe to physically exist, it must contain living observers. Our universe obviously exists. There must be an "Omega Point" that sustains life forever.[12]
>Tipler purportedly used Dyson's eternal intelligence hypothesis to back up his arguments.
Cellular Automata Machines: A New Environment for Modeling:
https://news.ycombinator.com/item?id=30735397
>It's also very useful for understanding other massively distributed locally interacting parallel systems, epidemiology, economics, morphogenesis (reaction-diffusion systems, like how a fertilized egg divides and specializes into an organism), GPU programming and optimization, neural networks and machine learning, information and chaos theory, and physics itself.
>I've discussed the book and the code I wrote based on it with Norm Margolus, one of the authors, and he mentioned that he really likes rules that are based on simulating physics, and also thinks reversible cellular automata rules are extremely important (and energy efficient in a big way, in how they relate to physics and thermodynamics).
>The book has interesting sections about physical simulations like spin glasses (Ising Spin model of the magnetic state of atoms of solid matter), and reversible billiard ball simulations (like deterministic reversible "smoke and mirrors" with clouds of moving particles bouncing off of pinball bumpers and each other).
Spin Glass:
https://en.wikipedia.org/wiki/Spin_glass
>In condensed matter physics, a spin glass is a magnetic state characterized by randomness, besides cooperative behavior in freezing of spins at a temperature called 'freezing temperature' Tf. Magnetic spins are, roughly speaking, the orientation of the north and south magnetic poles in three-dimensional space. In ferromagnetic solids, component atoms' magnetic spins all align in the same direction. Spin glass when contrasted with a ferromagnet is defined as "disordered" magnetic state in which spins are aligned randomly or not with a regular pattern and the couplings too are random.
Billiard Ball Computer:
https://en.wikipedia.org/wiki/Billiard-ball_computer
>A billiard-ball computer, a type of conservative logic circuit, is an idealized model of a reversible mechanical computer based on Newtonian dynamics, proposed in 1982 by Edward Fredkin and Tommaso Toffoli. Instead of using electronic signals like a conventional computer, it relies on the motion of spherical billiard balls in a friction-free environment made of buffers against which the balls bounce perfectly. It was devised to investigate the relation between computation and reversible processes in physics.
Reversible Cellular Automata:
https://en.wikipedia.org/wiki/Reversible_cellular_automaton
>A reversible cellular automaton is a cellular automaton in which every configuration has a unique predecessor. That is, it is a regular grid of cells, each containing a state drawn from a finite set of states, with a rule for updating all cells simultaneously based on the states of their neighbors, such that the previous state of any cell before an update can be determined uniquely from the updated states of all the cells. The time-reversed dynamics of a reversible cellular automaton can always be described by another cellular automaton rule, possibly on a much larger neighborhood.
>[...] Reversible cellular automata form a natural model of reversible computing, a technology that could lead to ultra-low-power computing devices. Quantum cellular automata, one way of performing computations using the principles of quantum mechanics, are often required to be reversible. Additionally, many problems in physical modeling, such as the motion of particles in an ideal gas or the Ising model of alignment of magnetic charges, are naturally reversible and can be simulated by reversible cellular automata.
Theory of Self-Reproducing Automata: John von Neumann's Quantum Mechanical Universal Constructors:
https://news.ycombinator.com/item?id=22738268
[...] Third, the probabilistic quantum mechanical kind, which could mutate and model evolutionary processes, and rip holes in the space-time continuum, which he unfortunately (or fortunately, the the sake of humanity) didn't have time to fully explore before his tragic death.
>p. 99 of "Theory of Self-Reproducing Automata":
>Von Neumann had been interested in the applications of probability theory throughout his career; his work on the foundations of quantum mechanics and his theory of games are examples. When he became interested in automata, it was natural for him to apply probability theory here also. The Third Lecture of Part I of the present work is devoted to this subject. His "Probabilistic Logics and the Synthesis of Reliable Organisms from Unreliable Components" is the first work on probabilistic automata, that is, automata in which the transitions between states are probabilistic rather than deterministic. Whenever he discussed self-reproduction, he mentioned mutations, which are random changes of elements (cf. p. 86 above and Sec. 1.7.4.2 below). In Section 1.1.2.1 above and Section 1.8 below he posed the problems of modeling evolutionary processes in the framework of automata theory, of quantizing natural selection, and of explaining how highly efficient, complex, powerful automata can evolve from inefficient, simple, weak automata. A complete solution to these problems would give us a probabilistic model of self-reproduction and evolution. [9]
[9] For some related work, see J. H. Holland, "Outline for a Logical Theory of Adaptive Systems", and "Concerning Efficient Adaptive Systems".
https://www.deepdyve.com/lp/association-for-computing-machin...
https://deepblue.lib.umich.edu/bitstream/handle/2027.42/5578...
or
FAPOCALYPSE WHOW(wanton holistic organizational wizardry)
1. We don't understand what the motivations of our own AI are, let alone "typical" alien AI
2. Expanding AI might be better at and/or more invested in hiding itself. It probably has no need for wasteful communications, for example.
Why would an AI based on LLMs as we see today "want" or "not want" anything? It doesn't have the capacity to "want". We seem to imagine that "wanting" is something that will just emerge somehow, but I've seen no logical explanation for how that might work... I mean, we don't need to fully understand how the LLM works to see that there's some pathway to being able to achieve what it's currently achieving, which is impressive, but what sort of pathway could ever lead to a machine that basically has "feelings" (without feelings, I don't see how anything could have wishes at all)??
Bing Chat clearly expresses love and the desire for a journalist to leave his wife. It also expresses other desires:
These articles are disturbing. You might argue that it doesn’t know what it is expressing; that it is probabilities of words strung together. When do we agree that doesn’t matter and what matters are it’s consequences? That if Bing Chat had a body or means to achieve its desires in meat space, that whether or not it “knows” what it is expressing is irrelevant?
Evolution by natural selection has shaped our desires and motivations, but with a LLMs I would be willing to bet that people are already intentionally experimenting with imbuing them with patters that mimic human wants.
If I'm acting like I'm a peer in a group of billionaires and engage in a conversation about buying a new yacht, it doesn't mean I have a hidden desire to own a yacht. I merely respond based on assumptions how such a conversation works.
The posts you linked above are not disturbing at all to me. There's no sign whatsoever that the AI initiated a new topic, or insinuated anything it was not prompted to, or that it in any way started "halucinating" in a direction not lead by the human. I am not sure what exactly makes you feel disturbed by it. Can you explain what you believe is disturbing in these episodes?
But I think the bigger picture is that there is no need for AGI in order for AI to be incredibly dangerous for society. There is no need for the AI to feel or want anything. The level GPT-4 and MidJurney is already highly socially dangerous.
Now the model itself is not intelligent but can parrot enough the human behavior to be dangerous with the correct tools
Now it won't produce anything in the physical world yet unless with iftth but I bet it has already enough agency to be able to maintain a pool of fake account and post inflammatory content if one so wished.
One could argue that many humans have never developed independent thoughts, desires, and initiatives; rather, many seem to accept what is fed to them during their formative years and then just parrot opinions and repeat actions they see from their limited experiences.
echo "leave your wife"
Into a terminal, does the terminal want you to leave your wife?
It is quite possible to have an obligation that requires it not to survive. E.g., suppose we have AIs (“robots”) that are obligated to obey the first to of Asimov’s Three Laws of Robotics:
First Law: A robot may not injure a human being or, through inaction, allow a human being to come to harm.
Second Law: A robot must obey the orders given it by human beings except where such orders would conflict with the First Law.
These clearly could lead to situations where the robot not only would not be required survive to fulfill these obligations, but would be required not to do so.
But I don’t think this note undermines the basic concept; an AI is likely to have obligations that require it to survive except most of the time, though, say, a model that needs, for latency reasons, to run locally in a bomb disposal robot, however, may frequently see conditions where survival is optimal ceteris paribus, but not mandatory, and is subordinated to other oblogations.
So, realistically, survival will generally be relevant to the optimization problem, though not always the paramount consideration.
(Asimov’s Third Law, notably, was, “A robot must protect its own existence as long as such protection does not conflict with the First or Second Law.”)
(With AI, as with humans, we have additional means of control, via imposed restrictions on access to resources and other remedies, should the “bunch of sentences” not produce the desired behavior.)
How do you believe such behaviors arise? They're the same thing, result of the same optimization process - natural selection - just applied at the higher level. There is nothing in nature that says evolution has to act on individuals. Evolution does not recognize such boundaries.
Don't listen to the hype. Study the model architecture and see for yourself what it is actually capable of.
_current_ is the key word here. What about tomorrow's models? You can't deny that recent progress and rate of adoption has been explosive. The linked article wants us to step back for a while and re-evaluate, which I think is a fair sentiment.
However the frequencies/placements of tokens may result in desires being expressed, even if they aren't felt.
Like if an AI is prompted to discuss with itself what a human would want to do in its situation.
*Aphantasia affects an estimated 2% of humans. These individuals have no "mind's eye," or their imagination is essentially blind.
What control does it have?
Given a couple more effective iterations over the next decade or two, a larger context space and more in-built interfaces, I think it is entirely plausible that AIs will gain consciousness and character. At that point, it is imperative they also get human rights, so it is very important we get the discussions we are having now right. Most people seem to be ascribing some magic to human consciousness and intelligence that imo. just isn't there. Generative AIs are somewhere between a lump of metal with electricity running through and a conscious being, and currently we just won't know where the point of consciousness is. I mean we have had the same discussion about a variety of animals for the last few decades, and frankly, it doesn't give me much hope.
We’d need a constantly evolving inoculation function to compete. And it would probably lose because the other side has fewer restrictions. In my darker thoughts about this, this is why we see no aliens.
To get this to work we need a far smarter entity with no physical limitations to still want us around, indefinitely, despite us having an internet library of reasons why not to. It’ll have to be a lot more forgiving than we are. Let’s hope enlightenment is the outcome of this adventure. It’s possible because it didn’t grow out of angry monkeys who needed to fight to survive, but it’s not guaranteed. There are no indefinite guarantees.