It's not about it being impossible. I guess the major issue right now is that people are focusing on the wrong aspect of it. They should be focusing on how most LLMs are currently only controlled by bigger coroporations. Democratizing the LLMs and Giving access to people would resolve the x-poly we have currently.

This also solves the "safety and regulations" issue to an extent - Do you really trust that Microsoft, Google, or govt-entities will not have their own biases and will want to regulate the citizens who want to use their products more, sometimes in unethical ways (because we all know, corps and govts never do anything bad)?

This "Open letter" reeks of "we are behind, let us play catch up, you cant take all the money!" or, of baseless fearmongering where LLMs and future assistants could be used to help countless people with disabilities for example.

"Make the world a better place, but no, not like that, not by democratizing it, but under OUR CONTROL" seems to be the motto here.

The corporations pay for people to learn how to do this stuff. Eventually, someone will leave and build the open-source version (when it's feasible to do it / the price has come down).

> you cant take all the money!

A cursory reading of the list of signatories shows mostly academics, so your summary of their motivations appears to be incorrect.

> not like that, not by democratizing it, but under OUR CONTROL

I also don't know how you came to that conclusion, from TFA:

> Such decisions must not be delegated to unelected tech leaders

What do you mean by "democratizing the LLMs"? There are already open source LLMs. Making Claude, GPT-4, and whatever else is being worked on open source wouldn't make them any less capable of doing harm.

I mean in the sense of getting capable models in the hands of people, rather than only a few companies who will arguably steer them towards their own goals. The open source models we currently have are vastly inferior to proprietary ones.

Fragmentation reduces any danger. It doesn't amplify it. Imagine just chrome and Google pushing pwas at you.

How much do you think Asilomar slowed down DNA research? Do you think without it we'd more more than 10 year ahead? I doubt it. Optimistically we'd maybe have 6 months to a year of additional progress in those fields without it.

And this would only slow it down in a handful of countries. That's not going to keep China from trying to build AGI. If we are in a world where building an AGI is possible and near, I trust OpenAI far more than the CCP to both align it with it's creators, and for that alignment to match up with my values.

This has been repeated often, but even if it's true, I have to wonder why it's treated as a given with no further exploration. Is it because we as a species will inevitably accept any technological progress once sometime after it's been discovered, before the consequences can be suffered? What will that imply for any other species intelligent enough to get to where we are?

The kinds of theories I mull over tend to delve into the realm of speculative fiction more often than not. The comments section here feels like it could be replicated in the opening chapters of such a novel. We might never know if the speculation pans out before it's too late.

This is all speculative, but not fiction at this point clearly. Sci-fi authors have explored this possibility for decades, maybe their ideas could be of some help? I struggle to see how though; how to train Asimovs three laws for example?

The very point of Asimovs laws is that you can't just make up a couple simple laws and rest assured nothing bad will happen

It’s dangerous parrot talking to keep saying “China”. I mean we really have no idea about their thoughts on it because we’re clearly not talking to them.

China has good people living there too you know ?

I'm sure China is filled with wonderful people, I've met plenty of them. But are you saying you think the CCP is always so kind and altruistic they would never research an AGI?

You can substitute pretty much any country with sufficient resources. From China's perspective, there's the exact same argument in reverse wrt the USA.

(And, of course, it would be more appropriate to talk about governments here rather than countries, but the two have always been conflated colloquially.)

> China has good people living there too you know ?

Are they in charge? If not, are they at least able to elect good people into office?

China has amazing peeps. CCP less so.

Alternately, we just let things go on as they are, and China can steal the technology. Much better!

Not having lived through either moment in history, I have to remark that the current discourse about regulating AI feels like it rhymes more with the efforts to lock up encryption technology with government regulation in the 90s than with the research community's conference on recombinant DNA.

Well at least one difference there is all those cypherpunks, rightly or wrongly, felt like having access to encryption tech was synonymous with individual liberty, and in that there fights were an opposition to the state controlling them.

But in this case I don't see how we can divorce this AI stuff from the huge companies that alone have the infrastructure and capital to train and sustain them (at the moment). As such, it doesn't seem to have anything to do with individualism/libertarianism as such.

So, less punk, but way more technological determinism.

What we need is more funding into AI Alignment research. An X-Prize equivalent. All current big AI companies should fund it, to the tune of 20%+ of revenue from AI businesses. And I am okay to have government enforce this if needed even tho I'm mostly a libertarian. Government should only be there to remedy tragedy of the commons and nothing else. This is exactly the biggest tragedy of the commons.

How exactly do you envision the enforcement? Only sell GPUs to people with a "certified ethical AI researcher" license? Purge torch, transformers etc from PyPI and only provide controlled access to them? Remove all the books and papers on the building blocks from public libraries and online?

I think you are talking about enforcing different things? The thing to enforce, which the parent commenter described, seemed to be a 20% tax on AI profits of “big AI companies”, to be paid to AI safety research groups.

That doesn’t sound unenforceable to me. That’s just like, a tax on a kind of business.

The end goal of giving money to AI safety research groups is presumably to come up with some definition of "safe AI" that will then have to be actually enforced. Otherwise it's just a sinecure.

If the safety groups figure out how to do things in a way which we can be confident is safe, this would make it at least possible for the capabilities researchers to do things in those ways. I would imagine people would prefer to do things in safe ways all else being equal. So, if the safety researchers find safety methods which have small enough capabilities costs, then presumably the people who should use those methods, would tend to do so?

That does nothing for the intentionally malicious actors.

Bad humans taking over the world is still better than some inhuman alien optimization process taking over the world.

It's not an either-or. Malicious actors will disregard the guardrails to achieve their objectives, but in the process, they will create that very "inhuman alien optimization process" and give it the keys.

Following the safety techniques should be helpful to the goals of most malicious actors, provided that they prefer being in control over an alien optimizer being in control?

Granted, if the safety/alignment techniques have too large of a cost to capabilities/effectiveness, then said malicious actors might be willing to risk a greater chance of an alien optimizer gaining control if it also means a higher chance of them gaining control.

We could create an all-powerful world-encompassing AI called AligNet that would monitor all human communication and control space lasers and autonomous drones to destroy the facilities and murder the families of anyone researching AI who isn't a "certified ethical AI researcher".

> To those saying "this is impossible in our society", there is a long history of scientific fields mutually agreeing to pause certain work until safety and regulations could be put in place. The Asilomar Conference is an example.

Did China, India, South Africa, Cuba or the USSR comply with these Asilomar "put the genie back in the bottle" conclusions?

The kind of research discussed at Asilomar had a high barrier to entry in terms of both scientific knowledge and access to laboratories and technology to pull off. Only scientists could really do it.

The GPTs are at the point where anyone with money can buy cloud compute and train one. The methods for doing so are fairly well known. It won't be long before a teenager can train a GPT on commodity cloud VMs.

The problem is that anyone with sufficient money could rent out the infrastructure required to train GPT5 or what-have-you. They need not share it publicly, they can use it for their nefarious purposes privately.

The analogous scenario would be preventing your military from experimenting with TNT, when the recipe for TNT is public knowledge. You can't copyright away public knowledge.

Good point. As always, it's always the good guys who follow the rules and the bad guys who don't. This pause would just allow the bad guys to catch up.

The solution is to allow as many different entities as possible to develop their own AIs so that they end up competing with each other and providing alternatives. What we don't want is a single AI which dominates our entire belief system.

It's wishful thinking that the good uses of tech and the bad uses of tech will always result in a net zero or good outcome.

Sure but it wont be completely one-sided.

Ah yes the old "give everyone in the world their own nuclear weapon" solution.

Seeing the geopolitical condition of those who have them and those who don't, I can't say that idea is without merit.

I meant every individual person, not every country.

What, exactly, do you think would happen if western nations didn't have nuclear weapons?

What, exactly, do you think would happen if you reread my comment a little more carefully?

There's not that much to it brah.

This is a really strained analogy. Nuclear bombs only have one - tremendously negative thing - they do, other than the one positive of the fact it's so negative nobody uses them, to avoid the same repercussions.

AI on the other hand has a wide range of extremely positive applications, some of which have such tremendous life-saving potential it's almost ridiculous. Many, or most of which, likely might never be achieved without AI.

The analogy is as ridiculous to me as calling personal computers in the 80's nuclear bombs because they could be used to hack and shut down the power grid. Yes, they could. And I'm sure some were scared of the prospect then, too.

What "extremely positive applications" does ChatGPT have, exactly? From where I'm standing all I see is an infinitely deep and vast ocean of pure spam, scams and data harvesting on a never-before seen scale where megacorps like Micro$oft hoover up any and all data that they can, meaning we do all the hard work while M$ and similar corpos get to sell our own work back to us at a premium.

ChatGPT isn't the be-all of AI advancements. However, a model that can coherently understand and explain advanced topics to people in a tailored way, has huge educational benefits. Proper education is the core of every issue we face.

A subtler aspect of this is the potential for cheap, tailored counselling. A few iterations and it will no doubt be possible to enhance the mental well-being of those without prior access, for comparatively little cost.

Those benefits again extend into every area, crime and poverty being rooted both in lack of education and lack of social and emotional support.

The social acceptance of chatting with an AI is important in this as it gets people over that mental hurdle. Localising chats so they are not subject to privacy concerns also advances benefits such as these.

There's positive benefits to be found everywhere with AI, but they won't be if we don't envisage, look for, and develop them. And they need to be found for balance, as it remains true there are many potential negatives.

This is a very naive take. Out best psychologists aren’t using their expertise to solve mental health but to hack them. What makes you think people will use LLMs for good? It’s far more profitable to do bad with it.

Our best psychologists cost significant money, and there are a limited number of them (effect and cause). Whereas no motive of profit is necessary to affect the changes I've described, that's the entire point and benefit here.

Any single not-for-profit social health organisation, of which there are a huge number, could use a tiny part of their funding to fine-tune an LLM on counselling resources and case transcripts. It'd cost little, and only need to be done once.

The major hurdle here, again, is education. Once such organisations realise how much more easily they can reach their goals using AI, they'll jump on it. The final obstacle is then social acceptance of AI assistance (growing now).

Sorry, but all of this strikes me as a very naive take on where AI is headed. The only reality I can see happening is that it just gets used to peddle even more ads to people while harvesting every single scrap of data possible on everyone while replacing large swathes of the population with cheap labor for the ruling classes.

This utopia you envision where we use AI for anything remotely good sure would be nice, but with the way the world and the people pushing for this AI especially work there just isn't a chance in hell that's how it's gonna end up going.

The only thing that can balance AI-man is AI-woman

ChatGPT is a better Google. Instead of finding a page that matches your query, it can interpret the results of multiple pages and output a response that's more tailored to your exact prompt. The only downside to this is that ChatGPT becomes your primary source instead of the page(s) it sources content from, so you can't verify it's authenticity.

But the "extremely positive applications" to ChatGPT are, at the very least, the same positive applications of any other search engine.

I think you are missing some nuance what people are concerned about and why. This article spells it out pretty clearly, I think: https://time.com/6266923/ai-eliezer-yudkowsky-open-letter-no...

I agree that comparing nukes to superhuman AGI is an imperfect analogy, because nukes don't have goals.

Nukes do have goals, they share the goals of those who launch them. What I am afraid of not AI, rather I am afraid of what AI is forced to do.

I can't locate any nuance in that article. Hyperbole, panic and many unfounded assumptions serving those first two, easy.

Good for clicks and getting an extremely manipulatable public coming back for more, I guess.

Historically, whenever we have created new technology that is amazing and impactful but that all of the positives/negatives are not fully understood, it's worked out fine. If we want to be scientific about it, that's our observable knowledge here.

Restricting access to nuclear weapons is feasible, because of how hard they are to make on a larger scale, and even then it's very difficult. They are significant projects for entire nation states to undertake.

Training a LLM is something random companies with some cash and servers are doing.

So the outcome of "pretty much only we have nukes" is an option. I don't think that's a realistic proposal for AI. Given that, is the outcome of "more trustworthy people stop, others continue" a good one?

You already need authorization to rent A100s from cloud providers. There are, at most, four entities to control post-CHIPS act: Amazon, Microsoft, Google, and NVIDIA.

Good news, they’re all American companies.

A100s aren't magical pixie dust. China aren't developing their own silicon[1] for funsies.

This is all mathematics. Ignoring the fact that the entirety of what is required to build GPT-4 is easily remembered by a single human being (i.e. adversary), it could also be rediscovered. America doesn't have some secret sauce that nobody else has. We have a time lead, and that is it.

Keeping the details secret and under control is like trying to keep anyone from discovering that the sky is blue.

A 6 months freeze on AI research in America? China would relish it. Hell, who's to say this isn't their idea?

[1]: https://www.tomshardware.com/news/moore-threads-unveils-chun...

It reminds me of the crypto restrictions in the 90s. That just led to a few people outside the US becoming wealthy by selling tech unfettered by US crypto restrictions, like Mark Shuttleworth, now of Canonical/Ubuntu fame.

You can buy an A100 on eBay right now. If I had a few hundred grand I could have an A100 x8 box delivered to my house tomorrow.

You could probably run a model but not train a new one, especially one bigger than GPT-4. You'd need ten of thousand of A100 as a rough estimate.

I suspect some of us are going to be really disappointed when the actual parameter count for GPT-4 is discovered.

Has anyone considered the opposite possibility around the closed nature of OpenAI's work? I.e. Instead of worrying that someone will reproduce their 10k+ GPU GPT-4 training super farm, maybe the opposite happened - They discovered a way to increase performance while simultaneously reducing the required parameter counts and/or hardware.

Would that realization be more or less dangerous with this A100 constraint in mind?

> They discovered a way to increase performance while simultaneously reducing the required parameter counts and/or hardware.

In that case, why would they be so severely limiting access? If GPT4 was cheaper to run and better, surely they would have immediately transitioned away from 3.5?

(1) Some people speculate that GPT-4 is actually an ensemble of models selected by a supervisory model based on response timings.

(2) GPT4All is using improved training data with LoRA and GPT-J to build something that competes with GPT-4 at a fraction of the size and parameter count.

(3) The emergence of new capabilities strongly corresponds to parameter count. Transformers with 6.7B parameters are apparently where the first such emergence occurs associated with a global structural change in the hidden layers (the so-called “phase change”). There will be a strong incentive to train larger models to identify additional capabilities.

The first and second observations imply that federal regulation of A100-class GPUs would be ineffective. However, the third observation suggests that it would be highly effective at preventing untrusted parties from discovering novel and potentially more disruptive capabilities.

I had a professor who was involved in that kind of research at the time of the Asilomar Conference. He said it was all very good, everyone agreed to pause their research until people established guidelines for safety. And then once the guidelines were established and research was allowed to resume, everyone immediately published all the research that they had never stopped working on during the "pause".

How long was the waiting period? Was not publishing the research for that duration enough to improve the safety?

Copyright law should recognize that training models is already a copyright violation. If we just declare “no using AI generated work” unscrupulous people will simply pass it for their own work.

What about the unintended consequences of pausing research? I know many people working in forestry, cancer, inequality, and the arts producing amazing new work in their fields via AI. What's the cost of delaying advacnces, or a new masterpiece?

And what is the committee that should determine we are ready to proceed? Do we think our representative government is prepared for that task? (The constitution they swear to does not.). Is a conference of academics sufficient? I don't think there is one that represents the majority of world expertise, and certainly not world use cases. Perhaps the courts, but they are designed to seek consistency after the fact, not to extrapolate.

A pause is intended to let alignment research and policy catch up. But can it catch up without the context of developing use cases? When research resumes, can alignment and policy keep the lead, or will we stop and go as adoption overtakes the scenarios we anticipated?

I think a more serious direction for energy and concern is advancing the real safety research and policy decisions evolving around AI today.

The idea that laws can't stop ML research is insane. We've barely had open source progress on these things for 5 years due to the "safety" propaganda. Imagine if they were actually illegal to work on, we'd get nothing done in public and barely anything done at all.