News for hackers. Stuff that matters.
Re:What I learned as a hired consultant to autodidact physic... by photochemsyn on Monday April 04, 2022 @02:06PM (#30909968)
For a while I held onto this archaic 1950 textbook on geology, solely because the introductory chapter included something like "the now thoroughly discredited crackpot theory of plate tectonics, which we will not consider here."
Re:What I learned as a hired consultant to autodidact physic... by antognini on Monday April 04, 2022 @04:20PM (#30911515)
When I was a kid I read a book published in the 1980s called Yes We Have No Neutrons. It covered various episodes in pseudoscience like N-rays, cold fusion, and Freudian psychology. But there was also a chapter on neural networks that did not age very well at all.
Re:What I learned as a hired consultant to autodidact physic... by mjburgess on Monday April 04, 2022 @04:47PM (#30911767)
The phrase "neural network" is itself a kind of pseudoscience, in being neither neural nor a network -- but an essentially pseudoscientific description of an algorithm better called "ensembled regression" or something of that kind.

NNs do not model the brain, and have basically nothing to do with it. I'd imagine rereading that chapter would be a good idea.

Re:What I learned as a hired consultant to autodidact physic... by martopix on Tuesday April 05, 2022 @06:34AM (#30917316)
I'm an academic working at the edge between biological neural networks in neuroscience, and artificial neural networks in machine learning. It is true that NNs don't model the brain, but it is not true that they have nothing to do with it. There is a whole fascinating field that compares the two and discusses their commonalities and differences.

But above all, "pseudoscience" is something different and is certainly not the right word to describe this issue of terminology.

Re:What I learned as a hired consultant to autodidact physic... by fivea on Wednesday April 06, 2022 @04:21AM (#30929739)
> But above all, "pseudoscience" is something different and is certainly not the right word to describe this issue of terminology.

The definition of pseudoscience is "a collection of beliefs or practices mistakenly regarded as being based on scientific method."

Arguably the idea that neural networks reproduce how brains work is pseudoscience.

In the very least, it's a deeply misguided marketing blurb. It's now considered a metaphor, just like the ones driving some corners of the artificial intelligence field.

Re:What I learned as a hired consultant to autodidact physic... by martopix on Wednesday April 06, 2022 @09:04AM (#30931511)
Pseudoscience is an issue of method, not of wrongly held beliefs. Before Einstein, Newton's law of gravitation was believed to be correct. It was not, but it wasn't "pseudoscience", because it was falsifiable and the theory was correct up to a certain error value. "Wrong" and "pseudoscience" are not synonymous.

Moreover, you're dismissing an entire extremely active research field, which tries to understand up to which point artificial and biological NNs are or aren't similar. It's the field, for example, of Turing laureate Yoshua Bengio.

Neural networks CAN reproduce how the brain work (if you're doing computational neuroscience, in which case what you call NNs is something different, and it's an issue of terminology). Even simple binary NNs were originally born to understand the cognitive functions of the brain (McCulloch and Pitts, 1943). The field later diverged (with the advent of backprop), but still today they have a lot in common even inadvertently, for example the representations they learn in navigational tasks (Banino et al. Nature 2018) or in the visual cortex vs. CNNs for computer vision.

Re:What I learned as a hired consultant to autodidact physic... by canjobear on Monday April 04, 2022 @04:58PM (#30911852)
This is a terminological question which has nothing to do with whether a textbook in the 80s was right or not in calling neural networks a crockpot theory.

The terminology is now fixed and anyone who has more than a passing acquaintance with these things knows that neural networks have almost nothing to do with biological neurons.

Re:What I learned as a hired consultant to autodidact physic... by rmatt2000 on Monday April 04, 2022 @09:32PM (#30914262)
If it's a crockpot theory, it probably just hasn't simmered long enough.
Re:What I learned as a hired consultant to autodidact physic... by mjburgess on Monday April 04, 2022 @05:10PM (#30911970)
It's a 1997, I've just skim read the chapeter -- and it seems entirely correct. Its views here are consistent with my own, NN is just a name for a technique -- baring little relation to the claims made about it in a hysterical press.
Re:What I learned as a hired consultant to autodidact physic... by munificent on Monday April 04, 2022 @07:58PM (#30913549)
Strings, fibers, ropes, and threads do not model textiles. Trees have basically nothing to do with botany but we find their names acceptable. We're programmers, metaphors are our bread and butter.
Re:What I learned as a hired consultant to autodidact physic... by zozbot234 on Monday April 04, 2022 @04:55PM (#30911824)
Ensemble methods have basically nothing to do with neural networks. The output of a NN is not some kind of "average" or "best pick" taken from the outputs of individual neurons. Rather, there are multiple layers each of which performs a kind of generalized multivariate regression on the outputs of the previous layer, and the parameterization for the whole hierarchy of layers is fitted as a whole. Very different.
Re:What I learned as a hired consultant to autodidact physic... by mjburgess on Monday April 04, 2022 @05:14PM (#30912012)
NNs with dropout are, trivially, an ensembling. And I think it's not so hard to show NNs, by default, meet a criterion like it -- namely that if we have something like batch normalization between the layers, so they are something PMF-like, then each is taking an expectation.

either way, the technique has absolutely nothing to do with the biological cells we call neurones -- as much as decision tress have to do with forests.

It is metaphorical mumbojumbo taken up by a credulous press and repeated in research grant proposals by the present generation of young jobbing PhDs soon to be out of a job.

The whole structure is, as it has ever been, is on the verge of a winter brought about by this shysterism. Self-driving cars, due in 2016, are likewise "just around the corner".

Re:What I learned as a hired consultant to autodidact physic... by vajrabum on Monday April 04, 2022 @05:43PM (#30912344)
I sympathesize about the overhyping. I certainly don't know if it's a good idea or not, but if you work for Google driverless cars are already on the road. https://www.theverge.com/2022/3/30/23002082/waymo-driverless...
Re:What I learned as a hired consultant to autodidact physic... by recursive on Tuesday April 05, 2022 @12:34AM (#30915327)
I don't know if you can use the word "already" for something that's been nearly here for so long.
Re:What I learned as a hired consultant to autodidact physic... by zarzavat on Monday April 04, 2022 @06:09PM (#30912616)
> [A]NNs do not model the brain, and have basically nothing to do with it.

ANNs don't yet model the structure of the brain but it seems plausible that they could do in the future as the result of some "convergent evolution".

ANNs have a fair model of individual neurons. Artificial and biological neurons do roughly the same thing when evaluated, but they are connected and trained very differently.

For me it's too much of a coincidence that the two most generally intelligent systems (ANNs and BNNs) are both "linear networks of activation functions".

We have not managed to build general intelligence from any other formalism, and neither has nature.

Viewing ANNs as a poor model of BNNs may be looking at the question backwards. You could say that BNNs are trying desperately hard to model the pure mathematics of ANNs within the confines of biochemistry. The fact that a biological neuron is not exactly a ReLU may say more about the limitations of biology rather than the limitations of ReLUs.

Re:What I learned as a hired consultant to autodidact physic... by peterlk on Monday April 04, 2022 @06:36PM (#30912881)
> For me it's too much of a coincidence that the two most generally intelligent systems (ANNs and BNNs) are both "linear networks of activation functions".

I am unconvinced that "linear network" appropriately defines BNNs. Can you clarify this?

Re:What I learned as a hired consultant to autodidact physic... by tsimionescu on Tuesday April 05, 2022 @03:34AM (#30916272)
> Artificial and biological neurons do roughly the same thing when evaluated

They most certainly do not. The idea that biological neurons are a kind of programmable on/off switch is completely wrong. There is plenty of computation happening inside a single neuron.

The fact that we once thought only the interconnect of neurons is important, and not the individual neurons themselves, is actually pretty strange. We know full well that every single cell is capable of computation, as it must react to its internal and external environment to be able to function. Neurons are more specialized for computation than other cells, and of course the biological neural network is a still a huge part of animal intelligence, but the simple model where we ignored what was going in inside each neuron should have been seen as extremely unlikely to be the full picture from the start.

Re:What I learned as a hired consultant to autodidact physic... by Khelavaster on Tuesday April 05, 2022 @12:21PM (#30920961)
You're confused if you think residual neural net motifs don't underlie how human brains reason about analogies, and if you think they don't control vision processing categorization.
Re:What I learned as a hired consultant to autodidact physic... by FeepingCreature on Tuesday April 05, 2022 @01:46AM (#30915653)
Neural networks model the brain via convergent evolution. I don't have a link for this offhand, but I remember seeing several studies that discovered that if you train ANNs on at least some of the same data sources that the developing brain trains on, they develop the same sorts of structures and match to the same features.
Re:What I learned as a hired consultant to autodidact physic... by np_tedious on Monday April 04, 2022 @03:43PM (#30911139)
That's fun. I had the converse experience when I discovered and read an old pop science article that discussed the light ether as if it were established fact. I was quite young and it was several years until I learned in high school physics class that this was complete bunk.
Re:What I learned as a hired consultant to autodidact physic... by Khelavaster on Tuesday April 05, 2022 @12:24PM (#30921023)
And it's not until further studying that you learn about the Anderson-Higgs effect in superconductors--how a gravity wave burst is generated every time a type 2 superconductor becomes superconducting.

Light-ether is real. It's [very oversimplifying] the EM pressure from maintaining every EMF-free zone (the insides of particles superconductors), plus pressure of all the cosmic background radiation.

Re:What I learned as a hired consultant to autodidact physic... by temporalparts on Monday April 04, 2022 @05:09PM (#30911962)
Do you have a picture of the book? Can you post a picture of the passage and cover? That would be fascinating to see! (I believe you fwiw, not doing the whole "pics or it didn't happen" :) )
Re:What I learned as a hired consultant to autodidact physic... by photochemsyn on Monday April 04, 2022 @08:16PM (#30913700)
I think the title included "Geomorphology" but I can't seem to find the exact instance. To be fair to geologists of that era, there wasn't a ton of evidence for plate tectonics at the time other than the matchup between continental boundaries. The mid-ocean spreading discoveries of a decade later kind of sealed the deal.
Re:What I learned as a hired consultant to autodidact physic... by wolverine876 on Monday April 04, 2022 @06:34PM (#30912855)
This theory of crackpot theories is popular with the crackpots, and I suspect it's a good signal of them. Some people do win the lottery, but your lottery ticket is still worthless.