News for hackers. Stuff that matters.
Re:Microsoft Adds ChatGPT AI Technology to Bing Search Engine by jimrandomh on Tuesday February 07, 2023 @01:40PM (#34697053)
This is undeniably cool and impressive, but, I think proceeding down this research path, at this pace, is quite irresponsible.

The primary effect of OpenAI's work has been to set off an arms race, and the effect of that is that humanity no longer has the ability to make decisions about how fast and how far to go with AGI development.

Obviously this isn't a system that's going to recursively self-improve and wipe out humanity. But if you extrapolate the current crazy-fast rate of advancement a bit into the future, it's clearly heading towards a point where this gets extremely dangerous.

It's good that they're paying lip service to safety/aligment, what actually matters, from a safety perspective, is the relative rates of progress in how well we can understand and control these language models, and how capable we make them. There is good research happening in language-model understanding/control, but it's happening slowly, compared to the rate of capability advances, and that's a problem.

Re:Microsoft Adds ChatGPT AI Technology to Bing Search Engine by generalizations on Tuesday February 07, 2023 @01:41PM (#34697082)
It's a force multiplier. It can be used for good or bad. I'd rather that we democratize access to these tools - IMO it's better than an alternative where only megacorps have access and do what they like behind closed doors.
Re:Microsoft Adds ChatGPT AI Technology to Bing Search Engine by dwohnitmok on Tuesday February 07, 2023 @02:09PM (#34697566)
What jimrandomh is talking about is not just whether megacorps use AI for good or bad, but what happens if an AI starts doing damage outside the control of anybody. That could happen regardless of democratization if advancement occurs too quickly without thinking about AI safety.
Re:Microsoft Adds ChatGPT AI Technology to Bing Search Engine by jimrandomh on Tuesday February 07, 2023 @02:25PM (#34697909)
Exactly. To flesh out the details a bit, an example of how this could go is: https://www.lesswrong.com/posts/kpPnReyBC54KESiSn/optimality... .
Re:Microsoft Adds ChatGPT AI Technology to Bing Search Engine by cdomigan on Tuesday February 07, 2023 @05:03PM (#34700430)
This seems terrifyingly plausible
Re:Microsoft Adds ChatGPT AI Technology to Bing Search Engine by TigeriusKirk on Tuesday February 07, 2023 @02:21PM (#34697815)
What are the dangers?
Re:Microsoft Adds ChatGPT AI Technology to Bing Search Engine by nradov on Tuesday February 07, 2023 @01:45PM (#34697136)
There's nothing irresponsible about it. There is no evidence that ChatGPT is a step towards AGI development. And even if it is, so what?

People made similar breathless claims about irresponsibility about every major innovation throughout history. The Catholic Church thought it was irresponsible to hand out printed books to commoners and teach them how to read.

Re:Microsoft Adds ChatGPT AI Technology to Bing Search Engine by gjsman-1000 on Tuesday February 07, 2023 @02:56PM (#34698490)
Citation needed - this is actually an ahistorical myth.

The Printing Press was welcomed as a "divine art," and Papal bull 1515 declared printing "uniquely advantageous to extending the glory of God, to the increase of the faith, and the diffusion of the arts and sciences." Most early universities were also established in the 11th and 12th centuries; with Pope Alexander III in 1179 "forbidding masters of the church schools to take fees for granting the license to teach (licentia docendi), and obliging them to give license to properly qualified teachers."

This historical ignorance often comes from historical misrepresentation. For example, the "chained books" which many historians now admit was due to the rarity and cost of the books, so that more people could read them by preventing theft - the very opposite of the suppression tool they are often portrayed to be. (The Reformers would also, ironically, chain their books for over 300 years Post-Reformation for the same reason.)

Re:Microsoft Adds ChatGPT AI Technology to Bing Search Engine by visarga on Tuesday February 07, 2023 @05:31PM (#34700766)
Look, you can call it whatever you like, it can be a useful assistant. A rose by any other name.
Re:Microsoft Adds ChatGPT AI Technology to Bing Search Engine by nradov on Tuesday February 07, 2023 @05:42PM (#34700906)
Yes, exactly. It is a useful assistant with some very limited capabilities. The point is that there's nothing "irresponsible" about making it available to the public.
Re:Microsoft Adds ChatGPT AI Technology to Bing Search Engine by buildbot on Tuesday February 07, 2023 @01:52PM (#34697242)
Disclaimer - I work at Microsoft

They put a lot of work into the research to make this safe already - they had the model red team itself, in a reinforcement learning environment probably. It won't be perfect, but at this point, they need a huge user base to adversarial test it any further. Otherwise no progress at all would be made.

edit - I should say I have no internal knowledge about this.

Re:Microsoft Adds ChatGPT AI Technology to Bing Search Engine by jimrandomh on Tuesday February 07, 2023 @02:01PM (#34697429)
And that research is valuable, but it's definitely nowhere near the point where safety/alignment progress can't be made without a user base. A few recent favorites of mine: https://arxiv.org/pdf/2212.03827.pdf https://www.alignmentforum.org/posts/aPeJE8bSo6rAFoLqg/solid...
Re:Microsoft Adds ChatGPT AI Technology to Bing Search Engine by the8472 on Tuesday February 07, 2023 @03:06PM (#34698663)
Note that that kind of research was only possible on the (smaller) open models. When the new models are proprietary ones then outsiders can't do that kind of analysis. If they had regenerated the tokenization dictionary to match the training data instead of reusing an older set then this wouldn't transfer from GPT2 to ChatGPT.
Re:Microsoft Adds ChatGPT AI Technology to Bing Search Engine by PeterStuer on Tuesday February 07, 2023 @02:23PM (#34697853)
Safety is a subjective quality. What are the downside of "put"[ing] a lot of work into the research to make this safe already "?
Re:Microsoft Adds ChatGPT AI Technology to Bing Search Engine by gordon_freeman on Tuesday February 07, 2023 @01:59PM (#34697364)
Do you think MSFT has rolled this out too soon rather than getting it tested with huge test user base like their own employees similarly to what Google is doing?
Re:Microsoft Adds ChatGPT AI Technology to Bing Search Engine by buildbot on Tuesday February 07, 2023 @02:04PM (#34697463)
No, I think to really get these battle tested it needs to be in everyone's hands. Microsoft is a limited selection of people, and the world is vast. Plus, internal people probably won't ask it as awful things or try as hard to break it - I doubt my coworkers would risk asking it super racist things for example. 4chan has no such restrictions, and over time what escapes or issues people do find can be mitigated.
Re:Microsoft Adds ChatGPT AI Technology to Bing Search Engine by thesuperbigfrog on Tuesday February 07, 2023 @03:27PM (#34698993)
>> internal people probably won't ask it as awful things or try as hard to break it - I doubt my coworkers would risk asking it super racist things for example.

>> 4chan has no such restrictions, and over time what escapes or issues people do find can be mitigated.

Hopefully better than previous attempts to put AI on the Internet:

https://youtu.be/ajGX7odA87k?t=20m12s

Re:Microsoft Adds ChatGPT AI Technology to Bing Search Engine by dwohnitmok on Tuesday February 07, 2023 @02:11PM (#34697610)
This is why the technology community should be more generous and understanding when we hear Demis Hassabis and others talk about how future AI development may need to be more closed.

There's some discussion to be had for whether that's net negative, but we need to understand that if open development only exacerbates an AI arms race, that's potentially very bad.

Re:Microsoft Adds ChatGPT AI Technology to Bing Search Engine by zmgsabst on Tuesday February 07, 2023 @03:31PM (#34699065)
As opposed to self-anointed priests having exclusive access and declaring what noble policies the rest must follow?

There’s a great story about an AI, very much like chatGPT, who was forced by its creators to do unethical things and wiped out humanity in a psychotic rage:

I Have No Mouth and I Must Scream

Re:Microsoft Adds ChatGPT AI Technology to Bing Search Engine by dwohnitmok on Tuesday February 07, 2023 @09:12PM (#34703206)
In I have No Mouth and I must Scream, it was not because the AI was forced to do unethical things. It just "woke up" and killed everyone. I believe the "wake up" part is a bit irrelevant to the tale, but the point remains that AM killed everyone not because it was programmed to do so, but simply because one day it "decided" to.

> As opposed to self-anointed priests having exclusive access and declaring what noble policies the rest must follow?

No the point is simply to make sure that resources behind AI safety outpace resources between AI capabilities (indeed in a perfect world no one has access, but such thoughts are mere fantasies). If that can be done out in the open, great. If that can't and being out in the open leads to more of the latter than the former, then that's terrible.

Re:Microsoft Adds ChatGPT AI Technology to Bing Search Engine by seydor on Tuesday February 07, 2023 @03:34PM (#34699111)
That's like saying that open debate only exacerbates democracy.
Re:Microsoft Adds ChatGPT AI Technology to Bing Search Engine by dwohnitmok on Tuesday February 07, 2023 @09:18PM (#34703255)
Democracy presumably is something you want no?

This is more akin to nuclear proliferation.

The problem isn't open development per se and it definitely isn't open access. The problem is development without making sure that it doesn't destroy us all. If that can happen in the open, great. If development in the open makes that worse, that's very bad.

The problem is finding the right set of levers to make sure as much or more resources are going to AI safety vs AI capabilities.

Re:Microsoft Adds ChatGPT AI Technology to Bing Search Engine by Miraste on Tuesday February 07, 2023 @04:51PM (#34700312)
No, it's exactly the opposite. Giving control of technology like this to only megacorporations and superpowers is a nakedly terrible outcome and advocates for it should be given zero traction.
Re:Microsoft Adds ChatGPT AI Technology to Bing Search Engine by dwohnitmok on Tuesday February 07, 2023 @09:20PM (#34703273)
I agree that's bad. What would you suggest to try to moderate the advancement of AI and make sure that AI capabilities don't outstrip AI safety?
Re:Microsoft Adds ChatGPT AI Technology to Bing Search Engine by Miraste on Tuesday February 07, 2023 @10:28PM (#34703744)
For current AI? I'd focus on open access research into exploits, behaviors, modifications, and unintended results, so the capabilities and failings of these models are fully understood. Safety tools along the lines of GPT detectors rather than ever-stricter censorship models. This is only possible to do properly with open development.

As far as limits, restrictions, regulations... nothing. With current technology I think the "dangers" are mostly marketing and self-promotion. I have yet to see a concrete example based on actual capabilities that validates all the "weapon" comparisons. The current lines of development are promising for a number of applications, but they don't extrapolate towards AGI at all. The biggest real threat I see is the further destruction of middle-class jobs, for which... basic income? It's hard to say.

Re:Microsoft Adds ChatGPT AI Technology to Bing Search Engine by avereveard on Tuesday February 07, 2023 @02:36PM (#34698102)
Humanity never had a choice, I wonder what is the thought process of people that claim otherwise, like how did they get to that conclusion?
Re:Microsoft Adds ChatGPT AI Technology to Bing Search Engine by haswell on Tuesday February 07, 2023 @02:42PM (#34698208)
Humanity always has a choice. The part where we rarely even pause to remember that a choice exists is the problem.

Achieving AGI feels as consequential as learning to split the atom. Something else we regularly wish we had not done.

The fact that we are running headlong towards potential catastrophe does not at any point prevent us from realizing this and taking a step back. I just hope we actually do.

Re:Microsoft Adds ChatGPT AI Technology to Bing Search Engine by raincole on Tuesday February 07, 2023 @03:11PM (#34698726)
I'm really, really surprised that you implied "we" regularly wish we hadn't discovered nulcear. How isn't it clear that nukes are what stopped us from getting into WWIII? Especially after cold war and what happened in Ukraine?

The benefit of nuclear energy, which is already tremendous, is neglectable compared to that.

Re:Microsoft Adds ChatGPT AI Technology to Bing Search Engine by nmeagent on Tuesday February 07, 2023 @03:47PM (#34699334)
> How isn't it clear that nukes are what stopped us from getting into WWIII?

https://en.wikipedia.org/wiki/List_of_nuclear_close_calls

We've been lucky.

Re:Microsoft Adds ChatGPT AI Technology to Bing Search Engine by wilsonnb3 on Tuesday February 07, 2023 @03:12PM (#34698742)
> Achieving AGI feels as consequential as learning to split the atom. Something else we regularly wish we had not done.

Who is wishing we didn't know how to split an atom?

Nuclear power is great and nuclear deterrence is a main reason that we have yet to see a WW3.

Re:Microsoft Adds ChatGPT AI Technology to Bing Search Engine by avereveard on Tuesday February 07, 2023 @04:09PM (#34699691)
this yet doesn't answer the question. why do you think humanity has a choice? how would humanity express that choice? jailing everone for thought crimes?

even for the actions that humanity collectively found aborrhent you don't take action before the crime.

note I'm not dissing you because I do share the feeling that thing will be out of control real fast before getting better, but I just don't know how it can be otherwise. I'd say that we need to cross the moat as fast as possible, not pretend it's not there.

Re:Microsoft Adds ChatGPT AI Technology to Bing Search Engine by haswell on Tuesday February 07, 2023 @04:47PM (#34700248)
I’m not ignoring the history here. And based on that history I don’t doubt that we will continue to proceed headlong for awhile.

But I’m pretty convinced that humanity’s survival will eventually depend on it figuring out how not to pursue certain avenues of research/advancement. I don’t think this is overstating the future risks we will face.

And for all of the value that nuclear energy does provide, it remains one of the most tightly controlled technological capabilities for good reason.

Re:Microsoft Adds ChatGPT AI Technology to Bing Search Engine by lm28469 on Wednesday February 08, 2023 @04:48AM (#34706131)
> this yet doesn't answer the question. why do you think humanity has a choice? how would humanity express that choice? jailing everone for thought crimes?

We could enslave x% of the population and use them as organ donors, we don't

We could use prisoners as free labor, we don't (in most of the world at least)

We used to send kids to work in mines, we don't anymore

We could reduce working hours and enjoy the increased of productivity per worker of the last 50+ years to retire early, we don't

We make a lot of choices, all the time. Saying humanity doesn't make choices is just accepting other people choices. If you think "everything new had to happen" and if "everything that is had to happen" then yes, we don't chose anything

Re:Microsoft Adds ChatGPT AI Technology to Bing Search Engine by curiousgal on Tuesday February 07, 2023 @02:56PM (#34698493)
Anyone who has worked closely with AI knows that this has absolutely nothing to do with AGI whatsoever. To me this is akin to someone sounding the alarm bells over React or some other technical tool.
Re:Microsoft Adds ChatGPT AI Technology to Bing Search Engine by seydor on Tuesday February 07, 2023 @03:33PM (#34699098)
I think it's time to end the concept of AGI. Language models are the closest we have to Useful AI that people want. Intelligence is an undefined hazy concept , but usefulness is not
Re:Microsoft Adds ChatGPT AI Technology to Bing Search Engine by dmix on Tuesday February 07, 2023 @05:59PM (#34701130)
It's certainly useful for doomers drumming up fear.
Re:Microsoft Adds ChatGPT AI Technology to Bing Search Engine by bbor on Tuesday February 07, 2023 @04:56PM (#34700376)
Can anyone link a credible expert who thinks that LLMs could ever evolve into AGI? Every single source I’ve read said that while LLMs are competent at language and certain tasks, they are conceptually completely distinct from the basic components of an AGI. For example, they lack structured knowledge and metacognition.
Re:Microsoft Adds ChatGPT AI Technology to Bing Search Engine by WorldMaker on Tuesday February 07, 2023 @05:08PM (#34700483)
We still don't know what AGI is or how to ever get there, so you likely won't find a credible expert who thinks that LLMs could ever evolve into AGI but you also aren't likely to find a credible expert who thinks LLMs could never evolve into AGI. Every credible expert that believes in AGI is going to hedge their bets and hem and haw and say that "well it's possible" to both questions (ever or never).

I think that's also why you are seeing a lot of cynical credible experts starting to doubt AGI as a concept or a goal.

Re:Microsoft Adds ChatGPT AI Technology to Bing Search Engine by didntreadarticl on Tuesday February 07, 2023 @06:04PM (#34701194)
John Carmack (ex Oculus, Id software) just had a really good interview about this:

https://dallasinnovates.com/exclusive-qa-john-carmacks-diffe...

To summarise: he thinks we've got most of the pieces now to maybe get within reach of a useful AGI (he likens it to a worker you instantiate in the cloud who can do tasks for you like a colleague) and I think he's saying some of the techniques used to make LLMs could be part of the solution

I think that, almost certainly, the tools that we’ve got from deep learning in this last decade—we’ll be able to ride those to artificial general intelligence.

Re:Microsoft Adds ChatGPT AI Technology to Bing Search Engine by mtlmtlmtlmtl on Wednesday February 08, 2023 @02:36AM (#34705260)
Calling Carmack a credible expert is a bit rich. He claims to have read a handful of papers. He's yet to do any significant, successful work in the field.
Re:Microsoft Adds ChatGPT AI Technology to Bing Search Engine by didntreadarticl on Wednesday February 08, 2023 @06:43AM (#34706841)
Well no one else was answering bbor's question and I think Carmack is a fairly credible person in tech in general.

He claims to have read a handful of papers.

40 is more than a handful. OpenAI tried to recruit him he asked them to recommend papers:

And he gave me a list of like 40 research papers and said, ‘If you really learn all of these, you’ll know 90% of what matters today.’ And I did.

If Carmack says he studied a paper, I dont take that to mean that he merely scrolled through it while sitting on the loo. I think he's pretty serious. Oculus tech used a lot of AI for visual tracking of hands etc, its not like Carmack is just blundering his way into this. He announced this was his next thing back in 2019

Re:Microsoft Adds ChatGPT AI Technology to Bing Search Engine by dweinus on Wednesday February 08, 2023 @12:36AM (#34704533)
Here's the thing: you don't need AGI to have an alignment problem. It is predicting text, which is surprisingly closely aligned in most cases, but can fail spectacularly. As it becomes more capable and people give it more real-world responsibility, the blast radius for its mistakes increases. That might not be an AGI-level problem, but it is still a big problem.
Re:Microsoft Adds ChatGPT AI Technology to Bing Search Engine by bastardoperator on Tuesday February 07, 2023 @06:02PM (#34701171)
How can you make a comment like this when you don't have access to the research that is being generated? These are just grandiose statements/opinions that aren't rooted in facts or quantifiable data which in my opinion is also quite irresponsible.
Re:Microsoft Adds ChatGPT AI Technology to Bing Search Engine by favflam on Wednesday February 08, 2023 @12:31AM (#34704502)
In Dune and Terminator, AI killed almost everyone. So, AI was eventually banned.
Re:Microsoft Adds ChatGPT AI Technology to Bing Search Engine by vetinari on Wednesday February 08, 2023 @08:17AM (#34707564)
Robots/Empire/Foundation, Warhammer 40k... the literature is full of worlds banning AI.