News for hackers. Stuff that matters.
Re:Douglas Hofstadter changes his mind on Deep Learning and ... by startupsfail on Monday July 03, 2023 @02:02PM (#36576604)
Don’t forget that most of these engineers down in the trenches can’t see further than their battlefield.

You actually need someone with vision and track record of doing right predictions and placing right technology bets.

Ask engineers that had placed their bet on deep learning and generative models back when discriminative models and support vector machines were a rage of dat, a few years before Alexnet (I’m one of such engineers). I’d bet the answer will be different.

Re:Douglas Hofstadter changes his mind on Deep Learning and ... by ke88y on Monday July 03, 2023 @02:09PM (#36576733)
> Don’t forget that most of these engineers down in the trenches can’t see further than their battlefield.

I don't think this is the case at all. I'm not primarily talking about a junior or even senior engineer with a decade of experience working on product features. On the contrary, many of these people have PhDs, have been leading research agendas in this field for decades, have been in senior leadership roles for a long time, have launched successful products, etc. etc.

> Ask engineers that had placed their bet on deep learning and generative models back when discriminative models and support vector machines were a rage of dat, a few years before Alexnet (I’m one of such engineers). I’d bet the answer will be different.

And at that time half of the "Great Minds And Great Leaders" prognosticating on X-Risk were doing social web or whatever else was peak hype cycle back then.

Re:Douglas Hofstadter changes his mind on Deep Learning and ... by jrumbut on Monday July 03, 2023 @04:01PM (#36578043)
> On the contrary, many of these people have PhDs, have been leading research agendas in this field for decades, have been in senior leadership roles for a long time, have launched successful products, etc. etc.

Society should definitely hear from those whose careers depend on continued AI research (and I fall into this group myself), but we are a hopelessly biased group.

Advancements in AI are going to change everyone's lives, so everyone deserves a say in this.

For the last 20-30 years tech has raced ahead of our government and civic processes and we're starting to feel the consequences of that. New technologies are being experienced by people as changes that are inflicted upon them instead of new options open to them.

They might have very little understanding of the technology, but everyone is the expert on what impact it is having on their lives. That's something that we shouldn't ignore.

Re:Douglas Hofstadter changes his mind on Deep Learning and ... by ke88y on Monday July 03, 2023 @04:07PM (#36578100)
FWIW, my bias leans the opposite direction. I benefit from AI risk hype.

I agree with everything you said and I think exaggerating capabilities and risks distracts society from doing that important work.

Re:Douglas Hofstadter changes his mind on Deep Learning and ... by lacrimacida on Monday July 03, 2023 @11:19PM (#36581781)
Cashing in on hype is short sighted and selfish.
Re:Douglas Hofstadter changes his mind on Deep Learning and ... by zone411 on Monday July 03, 2023 @02:26PM (#36576988)
I agree. If you speak with ML engineers, many simply haven't deeply considered longer-term risks and things like self-improvement until recently. I think we'll see more concern, not less. For me personally, GPT-3 wasn't that surprising, but GPT-4 and code generation capabilities were.
Re:Douglas Hofstadter changes his mind on Deep Learning and ... by OkayPhysicist on Monday July 03, 2023 @02:56PM (#36577367)
If you have a PHD, you scouted the battlefield.
Re:Douglas Hofstadter changes his mind on Deep Learning and ... by red-iron-pine on Tuesday July 04, 2023 @09:33AM (#36586406)
But often a very small battlefield, and located away from other efforts.

Literature reviews are meant to get broad down to the hyperfocused, and stay focused. But many of these issues are related to marketing, governance, economics, etc.

I'm sure plenty of PhDs would love to weigh-in on those, but that's more "engineer's disease" than real expertise.

Re:Douglas Hofstadter changes his mind on Deep Learning and ... by dlkf on Monday July 03, 2023 @05:09PM (#36578864)
> You actually need someone with vision and track record of doing right predictions and placing right technology bets.

Hofstadter does not exactly fit this description.

Re:Douglas Hofstadter changes his mind on Deep Learning and ... by TedDoesntTalk on Monday July 03, 2023 @07:44PM (#36580294)
Why not? Serious question
Re:Douglas Hofstadter changes his mind on Deep Learning and ... by lacrimacida on Monday July 03, 2023 @11:24PM (#36581807)
I suspect OP that your respnding to thinks something like … Hofstadter is not known to have made himself uber-rich therefore he’s not worth taking advice from or something along those lines
Re:Douglas Hofstadter changes his mind on Deep Learning and ... by dlkf on Tuesday July 04, 2023 @02:07AM (#36582786)
Enlighten me. What has his research agenda resulted in? What did Hofstadter get right that others did not?
Re:Douglas Hofstadter changes his mind on Deep Learning and ... by ChatGTP on Monday July 10, 2023 @09:44AM (#36666327)
Well, you got that guy...
Re:Douglas Hofstadter changes his mind on Deep Learning and ... by dlkf on Tuesday July 04, 2023 @01:53AM (#36582706)
What did he get right? While Hinton, LeCun, and others were laying the foundations of deep learning in the 80s, Hofstadter was writing expert systems in Lisp.
Re:Douglas Hofstadter changes his mind on Deep Learning and ... by lispm on Tuesday July 04, 2023 @02:20AM (#36582890)
Just for the record: Hinton was a Lisp user at some point in time. 1984 he got a Symbolics 3600. LeCun co-wrote Lush, a specialized Lisp for numerical computing. https://lush.sourceforge.net
Re:Douglas Hofstadter changes his mind on Deep Learning and ... by dlkf on Tuesday July 04, 2023 @05:13AM (#36584120)
That you have to reach for such obscure examples kind of illustrates my point, no? Geoff Hinton isn’t known for ”using Lisp at some earlier point in time.”

It’s not my goal to denigrate Lisp. I think Lisp is great. I learned programming with Dr Racket. But Hofstadter’s contribution to our understanding of intelligence is somewhere between ”negligible” and ”counterproductive.”

Re:Douglas Hofstadter changes his mind on Deep Learning and ... by disgruntledphd2 on Tuesday July 04, 2023 @05:28AM (#36584225)
One could say that same about Hinton and LeCun.
Re:Douglas Hofstadter changes his mind on Deep Learning and ... by dlkf on Tuesday July 04, 2023 @05:31AM (#36584244)
Are you joking? Learning nonlinearities by hidden representations ranks among the most important scientific discoveries of the past fifty years.
Re:Douglas Hofstadter changes his mind on Deep Learning and ... by disgruntledphd2 on Tuesday July 04, 2023 @06:43AM (#36584850)
How does it relate to intelligence though?

Certainly it's very useful for training ML models but their relationship to intelligence has yet to be determined.

Re:Douglas Hofstadter changes his mind on Deep Learning and ... by dlkf on Tuesday July 04, 2023 @09:27AM (#36586350)
The idea that knowledge can be expressed in distributed representations rather than sequences of symbols is a huge advancement in our understanding of intelligence. Obviously we haven’t ”solved intelligence” but now we at least have some idea of what the right questions to ask are.
Re:Douglas Hofstadter changes his mind on Deep Learning and ... by disgruntledphd2 on Tuesday July 04, 2023 @10:20AM (#36587076)
Can you share a link to the paper please? I feel like I get you, but would like to disambiguate with the mathematical representation.
Re:Douglas Hofstadter changes his mind on Deep Learning and ... by dlkf on Tuesday July 04, 2023 @10:33AM (#36587263)
I’m not talking about a single paper, I’m talking about a whole research programme. But Rumelhart, Hinton, Williams 1986 is very improtant. A lot of the foundational work is collected in the PDP report.
Re:Douglas Hofstadter changes his mind on Deep Learning and ... by disgruntledphd2 on Tuesday July 04, 2023 @10:48AM (#36587478)
Links would be greatly appreciated. I'm definitely not familiar with NN work from the 80s.
Re:Douglas Hofstadter changes his mind on Deep Learning and ... by disgruntledphd2 on Tuesday July 04, 2023 @10:57AM (#36587555)
> But Rumelhart, Hinton, Williams 1986 is very improtant. A lot of the foundational work is collected in the PDP report.

Thanks!

OK, so this is basically a connectionist model of mind approach?

I can definitely see this as an ancestor of current neural network approaches, and I now have some idea of what you mean by distributed representations.

I mean no disrespect here, but this has not contributed to our understanding of intelligence at all. It's proved useful in getting large datasets to perform certain actions (like vision and speech), but those are not necessarily the same thing at all.

It's a massive, massive advancement in the field of statistics and learning from data, but doesn't seem to map to my conception of intelligence at all.

(as you may have guessed, I'm sceptical that statistical learning approaches will lead to human-level intelligence).

Re:Douglas Hofstadter changes his mind on Deep Learning and ... by dlkf on Tuesday July 04, 2023 @12:45PM (#36589152)
> It's a massive, massive advancement in the field of statistics and learning from data, but doesn't seem to map to my conception of intelligence at all.

You think ”learning from data” has nothing to do with ”intelligence” ?