News for hackers. Stuff that matters.
Re:Microsoft lays off an ethical AI team as it doubles down ... by ssalka on Monday March 13, 2023 @11:39PM (#35147148)
AI alignment people are the group who are capable of bringing AGI to life, but want to do it responsibly.

AI ethics people are the group who are not capable of bringing AGI to life, but want to make sure that those who can do it responsibly or not at all.

Re:Microsoft lays off an ethical AI team as it doubles down ... by nradov on Tuesday March 14, 2023 @12:19AM (#35147408)
There is no evidence that any group is capable of bringing AGI to life.
Re:Microsoft lays off an ethical AI team as it doubles down ... by swyx on Tuesday March 14, 2023 @03:21AM (#35148315)
no evidence? none? Google Lambda and Bing/Sydney are not at least incrementally closer to evidence?
Re:Microsoft lays off an ethical AI team as it doubles down ... by bonzini on Tuesday March 14, 2023 @04:31AM (#35148716)
That's not AGI. An AGI would be able to tell you they don't know things, ask for clarification as to why you think they're wrong, etc.

ChatGPT just spouts out a different wild guess when you tell it it's wrong, but doesn't learn from its mistake—not even within a single chat. Sydney just goes full psychopath.

Re:Microsoft lays off an ethical AI team as it doubles down ... by EVa5I7bHFq9mnYK on Tuesday March 14, 2023 @08:28AM (#35150388)
That's not AGI. AGI would be able to write its own code, and improve it exponentially.
Re:Microsoft lays off an ethical AI team as it doubles down ... by nradov on Tuesday March 14, 2023 @08:41AM (#35150525)
That's a big assumption. No one has built an AGI yet so we don't know whether an AGI would be capable of improving it's own code exponentially. Any exponential growth function quickly runs into hard physical limits.
Re:Microsoft lays off an ethical AI team as it doubles down ... by AstralStorm on Tuesday March 14, 2023 @08:43AM (#35150555)
Not necessarily, much like every human does not know pharmacology, genetics, robotics or other specialties that could be used for self improvement. Including social engineering and programming.

What you're describing is a particular kind of superintelligence, recursively improving kind. Between that and general intelligence is another gap.

Re:Microsoft lays off an ethical AI team as it doubles down ... by EVa5I7bHFq9mnYK on Tuesday March 14, 2023 @01:41PM (#35155354)
Humanity as a whole can improve itself indefinitely, there is little doubt about that. For robots, there is no distinction between a single robot and robotuity, as software can be replicated cheaply and quickly.
Re:Microsoft lays off an ethical AI team as it doubles down ... by CuriouslyC on Tuesday March 14, 2023 @08:52AM (#35150671)
We're generally intelligent but we can't consciously rewire our brains.
Re:Microsoft lays off an ethical AI team as it doubles down ... by pnut on Tuesday March 14, 2023 @09:40AM (#35151325)
We sort of can, through meditation, habit, study, and chemical alteration - obviously within constraints, barring future technological enhancements.
Re:Microsoft lays off an ethical AI team as it doubles down ... by EVa5I7bHFq9mnYK on Tuesday March 14, 2023 @01:26PM (#35155072)
We can, this process is called "learning".
Re:Microsoft lays off an ethical AI team as it doubles down ... by CuriouslyC on Tuesday March 14, 2023 @02:52PM (#35156644)
Practice and study are conscious behaviors. Learning is not.
Re:Microsoft lays off an ethical AI team as it doubles down ... by hgsgm on Tuesday March 14, 2023 @09:15AM (#35150918)
That's Super Intelligent AGI, not merely AGI.
Re:Microsoft lays off an ethical AI team as it doubles down ... by mcguire on Tuesday March 14, 2023 @03:09PM (#35156869)
None of the existing models of what we call 'general intelligence' are capable of improving themselves exponentially.
Re:Microsoft lays off an ethical AI team as it doubles down ... by Arkhaine_kupo on Tuesday March 14, 2023 @06:06AM (#35149327)
> Google Lambda and Bing/Sydney are not at least incrementally closer to evidence?

No, in the same way that boiling water in a kettle is not incrementally closer to inventing the fuel for a rocket ship.

You can, in hidsight do a linear connection of water boiling -> steam engine -> fuel engine -> aeronautics -> space travel

But the first time someone boiled water, if you said that got you closer to being on the moon I think you would be mad. There is a non trivial chance that all our current work in AI is functionally useless in terms of creating AGI, it might take a complete breakthrough in ML, tech not invented yet, or it might not be possible at all.

Re:Microsoft lays off an ethical AI team as it doubles down ... by SturgeonsLaw on Tuesday March 14, 2023 @07:52AM (#35150055)
Except, in your example, boiling water did get us closer to the moon :)
Re:Microsoft lays off an ethical AI team as it doubles down ... by Arkhaine_kupo on Tuesday March 14, 2023 @08:24AM (#35150344)
Yep, and there was a much mre promissing science on checking Ether, and the magical qualities of gravity defying physics.

Who is to say that our current ML models are not Ether studies and utterly useless.

If we crack AGI in 10 years, maybe our current models are the fuel engine. But if they are not even close, we might be boiling water. Or perhaps be wrong all together, thats why I said your observation of us getting closer thanks to those proyects is not right, at least not yet.

Re:Microsoft lays off an ethical AI team as it doubles down ... by CuriouslyC on Tuesday March 14, 2023 @08:51AM (#35150655)
We have lots of examples of generally intelligent systems, they're common and they evolve by themselves, so the idea that it might not be possible at all just seems ridiculous.

Large language models might not be capable of general intelligence by themselves, but a model network that includes a LLM as part of a generalized game playing model would certainly have the potential to get there.

Re:Microsoft lays off an ethical AI team as it doubles down ... by RandomLensman on Tuesday March 14, 2023 @09:34AM (#35151220)
I think it is fair to say that "not at all possible" is perhaps too strong. On the potential of certain approaches I am less certain we have or know what we need.

But we also have examples of things all around us we cannot make ourselves (e.g., a simple animal or lifeform from scratch).

Re:Microsoft lays off an ethical AI team as it doubles down ... by CuriouslyC on Tuesday March 14, 2023 @11:21AM (#35152942)
Except that we've already created synthetic genomes: https://www.nytimes.com/2019/05/15/science/synthetic-genome-...

If we can define what "generally intelligent" means in concrete terms, we can absolutely create it. The problem is that we don't have a good understanding/shared idea of what we're trying to achieve. We can create models that can learn to play many different games using the same weights, solve IQ/logic tests and pass the Turing test consistently, but people are going to move the goalposts and say those are just complex mashups of autocomplete and flowcharts.

To be honest I don't think AI skeptics are ever going to believe until there's a Skynet trying to exterminate them.

Re:Microsoft lays off an ethical AI team as it doubles down ... by RandomLensman on Tuesday March 14, 2023 @11:58AM (#35153628)
I don't share the view that all that is holding us back from creating AGI is a definition. Btw., that synthetic genome is a long way from making even something like an little nematode.

I rather think AGI "idealists" (maybe there is a better word) will forever hold out (any) algorithmic advances as a sign of the immanent creation of AGI proper.

Re:Microsoft lays off an ethical AI team as it doubles down ... by baq on Tuesday March 14, 2023 @03:49AM (#35148481)
AGI happens slowly then suddenly all at once, to paraphrase a saying. There is no other way it can happen.
Re:Microsoft lays off an ethical AI team as it doubles down ... by darkwater on Tuesday March 14, 2023 @09:16AM (#35150925)
Why not? I'd be interested in hearing the reasons.
Re:Microsoft lays off an ethical AI team as it doubles down ... by turtleyacht on Tuesday March 14, 2023 @01:50PM (#35155502)
It would have to be reproducible: one second of a snippet of a video clip of something; the next, the thing happening. You rewind it and play again. Scrub through and try to determine a different branch. At one timestamp, separation, peace, unknowing. In the next, flames and smoke and a flashing blur.

But no--the state of the world in that second versus the next could not have been predicted. It just happened, just so, and it would take longer to calculate the determinism than to adjust to the new.

Re:Microsoft lays off an ethical AI team as it doubles down ... by mcguire on Tuesday March 14, 2023 @03:04PM (#35156820)
Singularity-related religious beliefs.
Re:Microsoft lays off an ethical AI team as it doubles down ... by baq on Tuesday March 14, 2023 @03:36PM (#35157166)
1) progress doesn't happen linearly, it's very coarse stepwise function (remember alphago?) 2) acceptance of an AGI actually being created is also a non-linear function - look at current definitely-not-AGI LLMs: some people think it's close, some people say that absolutely not, these are big matrices of numbers regurgitating words in plausible combinations and that's all - and these people won't change their minds easily one way or the other, thus declaring delivering of AGI will be very controversial even if we could agree on the definition.
Re:Microsoft lays off an ethical AI team as it doubles down ... by 6nf on Tuesday March 14, 2023 @05:00AM (#35148885)
Ok so what’s your point? We can just ignore it? Or what.
Re:Microsoft lays off an ethical AI team as it doubles down ... by RandomLensman on Tuesday March 14, 2023 @05:48AM (#35149202)
AGI worries we probably could ignore until AGI exists but there is now an industry around them which has its own momentum.
Re:Microsoft lays off an ethical AI team as it doubles down ... by Jcowell on Tuesday March 14, 2023 @08:19AM (#35150291)
This seems dumb to me. It’s like if the Japanese had the clairvoyant ability to know that nuclear bombs were being built and used and the immediate future and said “We can ignore it until it exists)”. No. You leave Hiroshima and Nagasaki now and not when the freaking bombs are above your head.
Re:Microsoft lays off an ethical AI team as it doubles down ... by RandomLensman on Tuesday March 14, 2023 @08:33AM (#35150447)
We don't have the clairvoyant ability to know that AGI will exist in any future near enough to worry about. It might or it might not - and even if it exists we could get its nature wrong and worry about the wrong things.
Re:Microsoft lays off an ethical AI team as it doubles down ... by flangola7 on Tuesday March 14, 2023 @12:09PM (#35153782)
This isn't congruent with the AI research community consensus. There's debate over many points, but that we will develop AGI is seen as a forgone conclusion almost universally. The most pessimistic place it closer to 2100, others say 5 years, most say <25 years. But nobody is saying it won't happen at all, or even that it won't happen beyond the lifetimes of anyone currently living. At best we would be kicking the can down the road and making it out children's or grandchildren's problem.

Notably the people closest to the fire, those advancing SOTA AI capabilities and those deepest into alignment/existential risk research, both say it is coming this decade. They predict very different outcomes (crudely summarized: utopia vs extinction), but they both predict the event horizon is on our doorstep.

You're right that we might get it wrong, but getting things wrong with a new technology really always means negative outcomes. When you're figuring out rocket science and you realize you worried about the wrong things your rocket doesn't somehow still succeed and make it to space, it blows up in a fireball. It might not blow up because of the specific thing you were paying attention to, but it still blew up.

Re:Microsoft lays off an ethical AI team as it doubles down ... by RandomLensman on Tuesday March 14, 2023 @12:42PM (#35154313)
Serious question: do you have a track record of the long-term forecasts of the research community? If they are quite good at this - fair enough.
Re:Microsoft lays off an ethical AI team as it doubles down ... by nradov on Tuesday March 14, 2023 @12:29PM (#35154094)
So what. Research community "consensus" is meaningless when it comes to something that doesn't exist and which we don't even have a clear path to build. It's just a bunch of pontificating clowns engaging in idle speculation. Total waste of time, like trying to predict what space aliens will look like or whatever. It makes for fun sci-fi stories but it's not something that serious people actually care about.
Re:Microsoft lays off an ethical AI team as it doubles down ... by flangola7 on Thursday March 16, 2023 @11:26PM (#35192622)
Saying lies and denigration does not make it true.

Why should we believe you over the experts for whom this is their life's work? What are your credentials? Where is your experience? You seem to think they're unserious clowns so clearly you have a good reason.

Re:Microsoft lays off an ethical AI team as it doubles down ... by nradov on Tuesday March 14, 2023 @08:35AM (#35150469)
You are certainly welcome to leave Earth right now if you like. But the notion that we should waste time worrying about some hypothetical future AGI is just dumb.
Re:Microsoft lays off an ethical AI team as it doubles down ... by hgsgm on Tuesday March 14, 2023 @09:17AM (#35150946)
Existing AI is not AGI. Existing AI is dangerous. Existing AI is widely deployed and growing.

Existing AI is more important to worry about.

Re:Microsoft lays off an ethical AI team as it doubles down ... by nradov on Tuesday March 14, 2023 @07:38AM (#35149928)
Yes, that's my point. We can ignore it until there is some sign of progress towards a true AGI. Show me a machine as intelligent as a mouse.

This article is a bit dated but it gives a good explanation of the difference between the powerful but limited tools we have today versus a true AGI.

https://www.kurzweilai.net/what-is-artificial-intelligence

Re:Microsoft lays off an ethical AI team as it doubles down ... by dragonwriter on Tuesday March 14, 2023 @04:04AM (#35148552)
> AI alignment people are the group who are capable of bringing AGI to life, but want to do it responsibly.

AI alignment people are the group who believe “responsibly” means, centrally, bringing AGI to existence as fast as possible, and, critically, under the tight, opaque, authoritarian control of a narrow group of people whose ideology is, well, exemplified by figures like Sam Bankman-Fried, and doing everything possible to advance that goal, including fostering broad acceptance of sub-AGI AI under the same tight control for other purposes without concern for adverse social impact as a means of advancing and funding progress toward AGI. They believe that the ends (AGI) inherently justifies any means taken in its pursuit.

AI ethics people are the group who believes “responsibly”, withe regard to AI, isn’t restricted to pursuit of AGI (which is maybe a nice to have, but a non-essential goal), and means openness, transparency, and resolving bias and harm issues as rapidly as possible when, and ideally before, adopting AI – including AI far short of AGI – in roles with social impact (and most critically, resolving those for roles with the greatest social impact.) They believe that AI is, to the extent it at any level is desirable, is a means to improving concrete, present, immediate conditions, not an ends to be pursued in itself.

Re:Microsoft lays off an ethical AI team as it doubles down ... by circuit10 on Tuesday March 14, 2023 @05:15AM (#35148962)
> AI alignment people are the group who believe “responsibly” means, centrally, bringing AGI to existence as fast as possible

From what I’ve seen it’s the exact opposite? They want AI development to slow down so we can get it right and make it safe. There are people who are trying to speed it up but that seems like the opposite of alignment to me

Re:Microsoft lays off an ethical AI team as it doubles down ... by dragonwriter on Tuesday March 14, 2023 @05:44AM (#35149174)
> From what I’ve seen it’s the exact opposite? They want AI development to slow down so we can get it right and make it safe.

The only way that the alignment faction wants to slow things down is in that they want to maintain tight control of models and gated access to assure “proper” use. This does have an effect of slowing certain kinds of development, as broader access to build around models without centralized controls fosters some aspects of development (the “Stable Diffusion moment”) but in ideal terms, the alignment faction is about fast-but-narrowly-controlled development.

The ethics faction is more about slowing things down – though more about adoption, particularly in sensitive uses, than development – but is more oriented toward openness/transparency.

Re:Microsoft lays off an ethical AI team as it doubles down ... by contextfree on Tuesday March 14, 2023 @08:51AM (#35150648)
There's no single "alignment faction" in terms of these questions. There are people associated with the alignment research program who do advocate slowing things down, e.g., https://worldspiritsockpuppet.substack.com/p/lets-think-abou... . There are a variety of other different positions, e.g., OpenAI, Anthropic, and MIRI have all taken different public stances on this
Re:Microsoft lays off an ethical AI team as it doubles down ... by skinner_ on Tuesday March 14, 2023 @06:54PM (#35160395)
https://aisafety.world/tiles/ lists dozens of institutions. Each hold different positions on AI development, but almost none of them hold the position that you ascribe to their "faction", maybe with the exception of OpenAI.
Re:Microsoft lays off an ethical AI team as it doubles down ... by ben_w on Tuesday March 14, 2023 @07:17AM (#35149758)
Sam Bankman-Fried is a uniquely terrible example of ideology, given St. Petersburg: https://conversationswithtyler.com/episodes/sam-bankman-frie... and https://astralcodexten.substack.com/p/open-thread-250

(and, y'know, being a fraudster who temporarily pulled the wool over the eyes of the regulators not just the charities he promised money to)

Also: is SBF really the central example of alignment in your head, and not, say Yudkowsky? Because Yudkowsky is my central example, and he's the exact opposite on all counts, wanting us to go slow and explicitly saying that humans are terrible at reasoning properly when they allow themselves to think "ends justify the means": https://www.lesswrong.com/posts/K9ZaZXDnL3SEmYZqB/ends-don-t...

Likewise, there are plenty of examples of misaligned non-general AI, and have been since at least the 80s when it was GOFAI and databanks rather than neutral nets that led to new legislation: https://www.legislation.gov.uk/ukpga/1984/35/enacted

Re:Microsoft lays off an ethical AI team as it doubles down ... by polygamous_bat on Tuesday March 14, 2023 @09:20AM (#35150986)
> Also: is SBF really the central example of alignment in your head, and not, say Yudkowsky? Because Yudkowsky is my central example, and he's the exact opposite on all counts.

For normal people who don't spend six hours every day on lesswrong, they were literally in the same camp up until about six months ago (when all the "rationalists" violently disclaimed SBF after being funded by him for a few good years.)

Re:Microsoft lays off an ethical AI team as it doubles down ... by ben_w on Tuesday March 14, 2023 @11:43AM (#35153358)
Bearing in mind that I don't spend 6 hours a day on lesswrong (or even per month, not even if I include podcasts tangentially inspired by it), and I hadn't even heard of SBF until his crypto-thingie did what I expect every crypto-thingie to do, that caricature you're painting is going to be one of those ones that need a lot of labels on everything.
Re:Microsoft lays off an ethical AI team as it doubles down ... by dragonwriter on Tuesday March 14, 2023 @10:42AM (#35152290)
> Also: is SBF really the central example of alignment in your head,

No, he's a particularly well-known example of the broader ideology to which virtually all of tbe alignment camp subscribes, and which centers their desires for AI.

Re:Microsoft lays off an ethical AI team as it doubles down ... by hughesjj on Tuesday March 14, 2023 @11:01PM (#35163228)
IMHO, the world would be better off without the Paypal Mafia and all their ilk.
Re:Microsoft lays off an ethical AI team as it doubles down ... by yanderekko on Tuesday March 14, 2023 @10:32AM (#35152145)
>AI alignment people are the group who believe “responsibly” means, centrally, bringing AGI to existence as fast as possible, and, critically, under the tight, opaque, authoritarian control of a narrow group of people

Hey, let's be fair now, a lot of them also believe that we just need a world government that tightly controls GPU use.

Sarcasm aside, the alignment crowd definitely does not have a consensus about the desirability about creating AGI asap. Yudkowsky is clearly horrified by OpenAI.

Re:Microsoft lays off an ethical AI team as it doubles down ... by visarga on Tuesday March 14, 2023 @08:43AM (#35150560)
> but want to make sure that those who can do it responsibly or not at all.

Mostly not at all. They write papers like "Can Language Models Be Too Big?"

I say then: can people be too afraid of the unknown?

Re:Microsoft lays off an ethical AI team as it doubles down ... by mcguire on Tuesday March 14, 2023 @03:00PM (#35156751)
AI alignment people are dedicated to fending off the gigantic PR disasters like Tay (https://en.wikipedia.org/wiki/Tay_(bot)) in order to keep their funding.

Whether either is capable of "bringing AGI to life" remains to be seen, but seems very doubtful.