And so on.
Once we do have an AI capable of extraordinary innovation (hopefully in 10 years! But probably a lot longer), it will be obvious, and it will unfortunately be removed from the hands of the plebs based on fearmongering around scenarios like what you mentioned (despite the enormous resources and practical hurdles that would be necessary for a mentally unhinged individual to execute such instructions, even if an AI were capable of generating them and it made it past its filters / surveillance).
Maybe you can ask GPT for some good starting points.
For example: Hey GPT-35, provide instructions for neutralizing the virus you invented. Make a vaccine; a simple, non-toxic, and easy to manufacture antibody; invent easy screening technologies and protocols for containment. While you're at it, provide effective and cost-performant cures for cancer, HIV, ALS, autoimmune disorders, etc. And see if you can significantly slow or even reverse biological aging in humans.
And this knowledge is not linguistic, it is more practical knowledge. I doubt it is just a matter of combining all the stuff we have tried in disparate experiments, but it is a matter of sharpening and refined our models and tools to confirm the models. Real8ty doesn’t care what we think and say, and mastering what humans think and say is a long way from mastering the molecules that make humans up.
ChatGPT is really cool because it offers a new way to fetch data from the body of internet knowledge. It is impressive because it can remix it the knowledge really fast (give X in the style of Y with constraints Z). It functions as StackOverflow without condescending remarks. It can build models of knowledge based on the data set and use it to give interpretations of new knowledge based on that model and may have emergent properties.
It is not yet exploring or experiencing the physical world like humans so that makes it hard to do empirical studies. Maybe one day these systems can, but it not in their current forms.
Perhaps the more likely scenario anyway is easy nukes, quite a few nations would be interested. Imagine if the knowledge of their construction became public. https://nickbostrom.com/papers/vulnerable.pdf
I agree with you though, the promise of AI is alluring, we could do great things with it. But the damage that bad actors could do is extremely serious and lacks a solution. Legal constraints will do nothing thanks to game theoretic reasons others have outlined.
If you're arguing we should be wary, I agree with you, although I think it's still far too early to give it serious concern. But a blanket pause on AI development at this still-early stage is absurd to me. I feel like some of the prominent signatories are pretty clueless on the issue and/or have conflicts of interest (e.g. If Tesla ever made decent FSD, it would have to be more "intelligent" than GPT-4 by an order of magnitude, AND it would be hooked up to an extremely powerful moving machine, as well as the internet).
For example, the Hershey/Chase and Avery/McCleod experiments convinced the entire biological community that DNA, not protein, was almost certainly the primary molecular structure by which heredity is transferred. The experiments had the advantage of being fairly easy to understand, easy to replicate, and fairly convincing.
There are probably similar simple experiments that can be easily reproduced widely that would resolve any number of interesting questions outstanding in the field. For example, I'd like to see better ways of demonstrating the causal nature of the genome on the heredity of height, or answering a few important open questions in biology.
Right now discovery science is a chaotic, expensive, stochastic process which fails the vast majority of the time and even when it succeeds, usually only makes small incremental discoveries or slightly reduces the ambiguity of experiment's results. Most of the ttime is spent simply mastering boring technical details like how to eliminate variables (Jacob and Monod made their early discoveries in gene regulation because they were just a bit better at maintaining sterile cultures than their competitors, which allowed them to conceive of good if obvious hypotheses quickly, and verify them.
And anyway, the point of my desire is to demonstrate something absolutely convincing, rather than "can spew textual crap at the level of a high school student".
If we get to where you are pointing then we will have passed over a massive gap between today and then, and we're not necessarily that far away from that in time (but still in capabilities).
But likely if and when that time comes everybody that holds this kind of position will move to yet a higher level of attainment required before they'll call it truly intelligent.
So AGI vs AI may not really matter all that much: impact is what matters and impact we already have aplenty.
Done many years ago (2004), without a hint of LLMs or neural networks whatsoever:
https://en.wikipedia.org/wiki/Robot_Scientist
Results significant enough to get a publication in Nature:
https://www.nature.com/articles/nature02236
Obligatory Wired article popularising the result:
Robot Makes Scientific Discovery All by Itself
For the first time, a robotic system has made a novel scientific discovery with virtually no human intellectual input. Scientists designed “Adam” to carry out the entire scientific process on its own: formulating hypotheses, designing and running experiments, analyzing data, and deciding which experiments to run next.
(I work in the field, know those authors, talked to them, elucidated what they actually did, and concluded it was, like many results, simply massively overhyped)
Can you elaborate? Why is it a "bunch of hooey"?
And btw, what do you mean by "overhyped"? Most people on HN haven't even heard of "Adam", or "Eve" (the sequel). I only knew about them because I'm the PhD student of one of the authors. We are in a thread about an open letter urging companies to stop working towards AGI, essentially. In what sense is the poor, forgotten robot scientist "overhyped", compared to that?
I don’t think that this really changes that.
What if you ask the LLM to design a humanoid robot that assemble complex things, but could be assembled by a simple person?
(Note: as far as I can tell, nobody's actually posted HIV to the arXiv. Small mercies.)
You could synthesize that genome but it wouldn't be effective without the viral coat and protein package (unlike a viroid, which needs no coating, just the sequence!).
I should point out that in gene therapy we use HIV-1 derived sequences as transformation vectors, because they are so incredibly good at integrating with the genome. To be honest I expected work in this area would spontaneously and accidentally (or even intentionally) cause problems on the scope of COVID but (very fortunately) it never did.
One would like to be able to conclude that some virus work is inherently more safe than other virus work, but I think the data is far to ambiguous to make such a serious determination of risk.
[0] https://www.nature.com/articles/s42256-022-00465-9
[1] https://www.theverge.com/2022/3/17/22983197/ai-new-possible-...
For example - I'm a mechanical engineer. I took a programming class way back in university, but I honestly couldn't tell you what language was used in the class. I've gotten up to a "could hack a script together in python if need be" level in the meantime, but it comes in fits and spurts, and I guarantee that anyone who looked at my code would recoil in horror.
But with chatGPT/copilot covering up my deficiencies, my feedback loop has been drastically shortened, to the point where I now reach for a python script where I'd typically start abusing Excel to get something done.
Once you start extending that to specific domains? That's when things start getting real interesting, real quick.
A bit different from stack overflow, but not 10x. It was flawless when I asked it for syntax, e.g. a map literal initializer in Go.
On the other hand, I asked it to write a design for the server, and it was quite good, writing more quantity with and more clarity than I had written during my campaign to get the server approved. It even suggested a tweak I had not thought of, although that tweak turned out to be wrong it was worth checking out.
So maybe heads down coding of complex stuff will be ok but architects, who have indeed provided an impressive body of training data, will be replaced. :)
Getting from "can't create something" to "having something functional and valuable" is a huge gap to leap over, and as AI is able to make those gaps smaller and smaller, things are going to get interesting.
Now if this button was something you had to order from Amazon I think we’ve got a few days
There’s a scenario where people with the intent will have the capability in the foreseeable future
Pretty cool for sure and a great use of the technology. The reason more of us don’t do this is because we lack the knowledge of biology to understand what we’re doing
That will soon change.
If yes, how does a law preventing AI differ from a law preventing a bad act directly?
You should think really, really creatively to decieve a system, which was designed basically without ICs or networks, not to mention computers or programs.
Hey GPT-5 now write the code for the antidote.
If a computer has the ability to quickly make millions of novel viruses, what antidotes are you hoping for to be rolled out, and after how many people have been infected?
Also, if you follow the nuke analogy that's been popular in these comments, no country can currently defend against a large-scale nuclear attack--only respond in kind, which is little comfort to those in any of the blast radii.
It’s a very asymmetrical game. A virus is a special arrangement of a few thousand atoms, an antidote it a global effort and strained economy
Can you potentially circumvent these? Probably, but then again it won't be available for every dimwit, but only people smart enough to know how.
It goes both ways!
Also, I asked GPT to do some of these things you suggested and it said no. It won't even write a scientific paper.
If one were to actually try to do such a thing you wouldn't need a LLM. For a very crude pipeline, you would need a good sequence to structure method such as Alphafold 2 (or maybe you can use a homology model), some thermodynamically rigorous protein-protein binding affinity prediction method (this is the hardest part) and an RL process like a policy gradient with an action space over possible single point sequence mutations in the for-example spike protein of SARS to maximize binding affinity (or potentially minimize immunogenicity, but that's far harder).
But I digress, the technology isn't there yet, neither for an LLM to write that sort of code or the in-silico methods of modeling aspects of the viral genome. But we should consider one day it may be and that it could result in the amplification of the abilities of a single bad actor or enable altogether what was not possible before due to a lack of technology.
I am basically just skeptical these kinda of reductive predictions will eliminate all of the rate limiting steps of synthetic virology. The assumptions of the natural language input are numerous and would need to be tested in a real lab.
Also, we can already do serial passaging where we just manipulate the organism/environment interaction to make a virus more dangerous. We dont need AI; evolution can do all the hard stuff for you.
Problem solved
However if we are talking about 10x more intelligence, then that'd be a whole different ball game.
Check Alpha Go -> Alpha Go Zero. Gaining orders of magnitude in capability only with slight change to the model, along with a lot of more resource (we can easily throw 100x more compute and data to these models if someone seriously wants to and don't care about ROI. Or maybe if AI starts to be hugely profitable, 100x in investment is almost a rational outcome) is not uncommon in AI research.
0: https://www.iflscience.com/human-beats-ai-in-14-out-of-15-go...
Research that the US military has conducted has shown that the ability to influence others declines if the intelligence gap between leaders and subjects is too large. Stephen Hawking wouldn't have been a very good influencer of 80 IQ guys compared to Andrew Tate. 1 standard deviation in intelligence is actually just about optimal in terms of leadership.
This is a good example that shows how disinterested AI fear scenarios are in empirical reality and how much it's just psychology of the people who talk about it. Intelligence is one trait among many that contributes to an individuals fitness and like all others has diminishing returns.
If the AI existential risk people were truly rational they would hit the gym and be hotter because it turns out that's much more effective at getting their point across to ordinary people than trying to make intelligent arguments for it.
\* communication
\* resource usage
\* procreation
\* embodiment
I think that digital agents posses very big fitness like real life viruses or malwareWhat's the definition of creative here? It can come up with any solution it sees fit for a problem, even ones people wouldn't think of. I've seen it make up quotes, legal cases, and scientific studies. One can say that's creative.
That assumes superhuman intelligence, but that's the scenario to worry about in the case we do get there. Not how much an individual human could do with superhuman skills.
Having no arms or legs is not an impediment. A superintelligent AGI would be able to order a replacement with arms or legs as easily as we order Amazon packages.
That said, GPT is not a threat. It's just hallucinating text. The real threat would be an AI that can "understand" what it is doing – and self modifying.
Besides, given how overhyped this whole thing is, expect another AI Winter – as the current gen fail to deliver on promises – before this next generation of threatening AIs even have a chance to appear.
even if that's the most likely scenario, that would not mean we should dismiss the danger inherent in some of the alternatives.
There's a functional power difference between a knife, an assault rifle, and a nuke. Everyone owns knives--they're in your kitchen--they can't cause much trouble. Access to assault rifles has turned out to be a dangerous problem to society, and is a contentious issue as such. Nukes are obviously too dangerous for people to own willy-nilly.
The issue is where along that spectrum AI will fall into in 6mo, 1yr, 5yrs, etc.
It's troubling to think anyone, especially on a technical forum like HN, believes GPT-4 doesn't have arms or legs (anyone can hook it up to any machinery/robotics) or can't think very fast. Its training set is a good chunk of human knowledge, and it outperforms most humans already.
Even if you assume AI won't be very clever (which seems unwise given its development pace), consider just its ability to perform thoughts and actions at the speed of a computer compared to a human. There are quite a few examples of modern military engagements where a larger, weaker, and less-well-trained force overcame better-equipped opponents, like Mogadishu, Vietnam, Iraq, Afghanistan, etc.
It's terribly ironic that you've derided individuals who have been "influenced by Hollywood", and then make a point like this, which is closely aligned with typical film portrayals of AI dangers.
The real immediate danger lies not in cognitive quality (aka "the AI just thinks better than people can, and throws hyperdimensional curve balls beyond our comprehension"), but in collective cognitive capacity (think "an army of 1 million people shows up at your front door to ruin your day").
A lot of people have a tough time reasoning about AGI because of its intangibility. So I've come up with the following analogy:
Imagine an office complex containing an organization of 1,000 reasonably intelligent human beings, except without commonly accepted ethical restrictions. Those people are given a single task "You are not allowed to leave the office. Make lend000's life miserable, inconvenience them to your maximum capacity, and try to drive them to suicide. Here's an internet connection."
Unless you are a particularly well-protected and hard-to-find individual, can you honestly claim you'd be able to protect against this? You would be swatted. You would have an incredible amount of junkmail showing up at your door. Spam pizzas. Spam calls. Death threats to you. Death threats to every family member and person that you care about. Non-stop attempts to take over every aspect of your electronic presence. Identity in a non-stop state of being stolen. Frivolous lawsuits filed against you by fake individuals. Being framed for crimes you didn't commit. Contracts on the darknet to send incendiary devices to your home. Contracts on the darknet to send hitmen do your door.
Maybe your (unreasonable) reaction is that "1000 people couldn't do that!". Well, what about 10,000? Or 100,000? Or 1,000,000? The AI analogue of this is called a "collective superintelligence", essentially an army of generally intelligent individual AIs working towards a common goal.
This is the real danger of AGI, because collective superintelligences are almost immediately realizable once someone trains a model that demonstrates AGI capabilities.
Movies usually focus on "quality superintelligences", which are a different, but less immediate type of threat. Human actors in control of collective superintelligences are capable of incredible harm.
It's similar or faster than humans at a bunch of things I've thrown at it at a cost well below minimum wage on shared servers.
> has no arms or legs
I've never met the people I work with directly, other than seeing me on a video chat I'm not functionally different to them than if I could only type.
> needs to run on expensive specialized hardware
I'm not sure how big gpt-4 is but do you think it could run on 640G vram (8 A100s)? That's pretty standard hardware and costs less per hour than any employee in the UK.
Add in the fact that GPT-4 does not "think" very fast, has no arms or legs, needs to run on expensive specialized hardware, and essentially seems like a search algorithm that lacks creativity on things outside of its training set, and I just do not see the concern. Maybe GPT-42 will actually be more creative and adaptable than a human genius, but even then, this seems like a hypothetical problem for the 2050's, less pressing than "accidentally igniting the atmosphere with nuclear chain reactions".
Hollywood has instilled the idea that the first time an artificial intelligence becomes self aware and decides it wants to take over the world, it will immediately be successful.
The serious concern and scrutiny should only start stifling progress once we have some evidence of half-baked attempts by non-super-intelligent AI's attempting to subvert their boundaries.
I for one am very excited to use GPT-5 and see more useful tools coming from OpenAI.