News for hackers. Stuff that matters.
Re:A.I. Poses ‘Risk of Extinction,’ Industry Leaders Warn by sf4lifer on Tuesday May 30, 2023 @10:07AM (#36125038)
LLMs are just text prediction. I don't see the linear path from LLM to AGI. Was there similar hysteria when the calculator or PC first came out?
Re:A.I. Poses ‘Risk of Extinction,’ Industry Leaders Warn by HDThoreaun on Tuesday May 30, 2023 @10:40AM (#36125410)
We don't need AGI to have an extinction risk. Dumb AI might be even more dangerous
Re:A.I. Poses ‘Risk of Extinction,’ Industry Leaders Warn by clnq on Tuesday May 30, 2023 @10:25AM (#36125238)
> similar hysteria when the calculator

https://twitter.com/mrgreen/status/1640075654417862657

Re:A.I. Poses ‘Risk of Extinction,’ Industry Leaders Warn by api on Tuesday May 30, 2023 @10:21AM (#36125196)
I know there were similar hysterias when automated weaving looms and other extreme labor saving machines came out. These machines did actually put a lot of people out of work, but they grew the economy so much that the net number of jobs increased.

In a way it's actually a bit dystopian that the "everyone will be put out of work" predictions never come true, because it means we never get that promised age of leisure. Here we are with something like a hundred thousand times the productivity of a medieval peasant working as much or more than a medieval peasant. The hedonic treadmill and the bullshit job creating effects of commerce and bureaucracy eat all our productivity gains.

The economy is actually a red queen's race:

https://en.wikipedia.org/wiki/Red_Queen%27s_race

Re:A.I. Poses ‘Risk of Extinction,’ Industry Leaders Warn by musicale on Tuesday May 30, 2023 @10:29AM (#36125276)
Automation has increased income inequality for the past 40 years or more, and is likely to continue to do so as more tech jobs and service jobs are automated in addition to office jobs and manufacturing jobs.

> In a way it's actually a bit dystopian that the "everyone will be put out of work" predictions never come true, because it means we never get that promised age of leisure.

It's disappointing that the economy seems to be structured in such a way for most people "leisure" is equivalent to "unemployment." It probably doesn't help that increases in housing, health care, and higher education costs have outpaced inflation for decades.

Re:A.I. Poses ‘Risk of Extinction,’ Industry Leaders Warn by api on Tuesday May 30, 2023 @10:33AM (#36125327)
Not globally: https://kagi.com/proxy/th?c=lUfv1nYBTMKYtKYO-rQ4Vg_QAA9uQJ07...

Outsourcing has stagnated wages in the developed world, but the cost of manufactured goods has also plummeted. The only reason people aren't better off is that the developed world (especially the anglosphere) has a "cost disease" around things like real estate that prevents people from benefiting from global scale price deflation. It doesn't help you much if gadgets are super cheap but housing is insanely expensive. The high cost of housing is unrelated to automation.

Re:A.I. Poses ‘Risk of Extinction,’ Industry Leaders Warn by musicale on Tuesday May 30, 2023 @10:50AM (#36125542)
> The high cost of housing is unrelated to automation

Housing, health care, higher education... all drastically more expensive.

The point about outsourcing is a good one.

However, automation still appears to drive income inequality (at least in the US.)

"Job-replacing tech has directly driven the income gap since the late 1980s, economists report."[1]

[1] https://news.mit.edu/2020/study-inks-automation-inequality-0...

Re:A.I. Poses ‘Risk of Extinction,’ Industry Leaders Warn by dwaltrip on Tuesday May 30, 2023 @10:43AM (#36125460)
It’s not just random text, they are predicting writings and documents produced by humans. They are “language” models.

Language is used to say things about the world. This means that predicting language extremely well is best done through acquiring an understanding of the world.

Take a rigorous, well-written textbook. Predicting a textbook is like writing a textbook. To write a good textbook, you need to be an expert in the subject. There’s no way around this.

The best language models (eg GPT-4) have some understanding of the world. It isn’t perfect, or even very deep in many ways. It fails in ways that we find quite strange and stupid. It isn’t capable of writing an entire textbook yet.

But there is still a model of the world in there. It wouldn’t be able to do everything it is capable of otherwise.

Re:A.I. Poses ‘Risk of Extinction,’ Industry Leaders Warn by somewhereoutth on Tuesday May 30, 2023 @11:27AM (#36125991)
To be more precise, it holds a model of the text that has been fed to it. That will be, at very best, a pale reflection of the underlying model of the world.
Re:A.I. Poses ‘Risk of Extinction,’ Industry Leaders Warn by tech_ken on Tuesday May 30, 2023 @03:55PM (#36129788)
>Predicting a textbook is like writing a textbook

HN AGI discourse is full of statements like this (eg. all the stuff about stochastic parrots), but to me this seems massively non-obvious. Mimicking and rephrasing pre-written text is very different from conceiving of and organizing information in new ways. Textbook authors are not simply transcribing their grad school notes down into a book and selling it. They are surveying a field, prioritizing its knowledge content based on an intended audience, organizing said information based on their own experience with and opinions on the learning process, and presenting the knowledge in a way which engages the audience. LLMs are a long way off from this latter behavior, as far as I can tell.

> The best language models (eg GPT-4) have some understanding of the world

This is another statement that I see variants of a lot, but which seems to way overstate the case. IMO it's like saying that a linear regression "understands" econometrics or a series of coupled ODEs "understands" epidemiology; it's at best an abuse of terminology and at worst a complete misapplication of the term. If I take a picture of a page of a textbook the resulting JPEG is "reproducing" the text, but it doesn't understand the content it's presenting to me in a meaningful way. Sure it has primitives with which it can store the content, but human understanding covers a far richer set of behaviors than merely storing/compressing training inputs. It implies being able to generalize and extrapolate the digested information in novel situations, effectively growing one's own training data. I don't see that behavior in GPT-4

Re:A.I. Poses ‘Risk of Extinction,’ Industry Leaders Warn by Henk0 on Wednesday May 31, 2023 @01:51PM (#36141326)
I took the liberty of asking GPT-4 to take the second statement and your response to it, and turning it into a fable:

"Once upon a time, in a village nestled by a grand mountain, lived a wise Sage. The Sage was known throughout the lands for his vast knowledge and understanding, for he had spent his life studying the texts of old and the secrets of the world.

In the same village, an Artisan, skilled in the craft of making extraordinary mirrors, lived. These were no ordinary mirrors, for they were said to reflect not just physical appearances, but also knowledge and experiences. Intrigued by the wisdom of the Sage, the Artisan decided to make a mirror that would reflect the Sage's knowledge.

After many days and nights of meticulous work, the Artisan finally crafted a mirror so clear and pure, it held a reflection of all the knowledge and wisdom the Sage had gathered throughout his life. The mirror could answer questions about the world, cite ancient texts, and reflect the wisdom it was imbued with.

Word quickly spread about the Sage's Mirror, and villagers began to claim, "This mirror is as wise as the Sage himself! It knows and understands as much as he does!"

However, a wise Old Woman of the village, known for her insightful observations, gently corrected them, "This mirror, as remarkable as it is, contains a reflection of the Sage's knowledge. It can share what the Sage knows but doesn't truly understand the way the Sage does."

The Old Woman continued, "The Sage has spent years learning, pondering, and experiencing life, which the mirror cannot replicate. The Sage's understanding implies the ability to think, reason, and learn in ways that the mirror, no matter how complete its reflection, simply cannot. The mirror's reflection is static, a snapshot of a moment in time, while the Sage's wisdom continues to grow and adapt."

The villagers learned a valuable lesson that day. They realized the mirror was an extraordinary tool that held vast knowledge, but it was not a substitute for the genuine understanding and wisdom of the Sage."

- Not too bad for a mirror.

I'd be interested to hear what you think is so special about human understanding? We also just absorb a lot of data and make connections and inferences from it, and spit it out when prompted, or spontaneously due to some kind of cognitive loop. Most of it happens subconsciously, and if you stop to observe it, you may notice that you have no control of what your next conscious thought will be. We do have a FEELING that we associate with the cognitive event of understanding something though, and I think many of us are prone to read a lot more into that feeling than is warranted

Re:A.I. Poses ‘Risk of Extinction,’ Industry Leaders Warn by Henk0 on Wednesday May 31, 2023 @02:00PM (#36141476)
A preemptive addendum to my previous reply:

Yes, LLM:s currently only deal with text information. But GPT-5 will supposedly be multimodal, so then it will also have visual and sound data to associate with many of the concepts it currently only knows as words. How many more modalities will we need to give it to be able to say that it understands something?

Also, GPT-4 indeed doesn't do any additional training in real-time. However, it is being trained on the interactions people have with it. Most likely, near future models will be able to train themselves continuously, so that's another step closer to how we function

Re:A.I. Poses ‘Risk of Extinction,’ Industry Leaders Warn by dwaltrip on Saturday June 10, 2023 @01:47PM (#36272815)
Hey, I wanted to reply to this, but just didn't find the time. I disagree with a lot of what you wrote, but your arguments are stated clearly. I appreciate that. Cheers :)
Re:A.I. Poses ‘Risk of Extinction,’ Industry Leaders Warn by tech_ken on Monday June 12, 2023 @12:17PM (#36295965)
> Hey, I wanted to reply to this, but just didn't find the time.

Story of my life, glad you found things clear if not persuasive! Hope you're having a nice start to your week

Re:A.I. Poses ‘Risk of Extinction,’ Industry Leaders Warn by kordlessagain on Tuesday May 30, 2023 @10:55AM (#36125597)
Careful, LLMs such as ChatGPT are not solely limited to text prediction. While text prediction is one of their capabilities, these models are designed to understand and generate human-like text based on the context and prompts they receive. Chaining, for example, allows interactive and innovative problem solving on a corpus of relevant documents: https://github.com/EagleW/CLBD
Re:A.I. Poses ‘Risk of Extinction,’ Industry Leaders Warn by mitthrowaway2 on Tuesday May 30, 2023 @12:38PM (#36126991)
The statement isn't about LLMs. It doesn't refer to them even once.
Re:A.I. Poses ‘Risk of Extinction,’ Industry Leaders Warn by EamonnMR on Tuesday May 30, 2023 @02:06PM (#36128282)
But if LLMs hadn't captured the popular imagination I doubt this would have been written this year and gotten the attention of enough prominent signatories to frontpage on HN.
Re:A.I. Poses ‘Risk of Extinction,’ Industry Leaders Warn by mitthrowaway2 on Tuesday May 30, 2023 @02:16PM (#36128406)
Maybe. It's happened before. [1] And several of the signatories have expressed views about AI risk for many years.

That said, the renewed anxiety is probably not because these experts think that LLMs per se will become generally intelligent. It's more that each time we find out that the human brain does that we thought were impossible for computers to do turn out to be easy, each time we find that it takes 3~5 years for AI researchers to crack a problem we thought would take centuries[2], people sort of have to adjust their perception of how high the remaining barriers to general intelligence might be. And then when billions of investment dollars pour in at the same time, directing a lot more research into that field, that's another factor that shortens timelines.

[1] https://news.ycombinator.com/item?id=14780752

[2] https://kotaku.com/humans-triumph-over-machines-in-protein-f...