News for hackers. Stuff that matters.
Re:The Purpose of Sleep? To Forget, Scientists Say by tprice7 on Friday February 03, 2017 @11:57AM (#13561179)
As someone with no neuroscience background I’ve suspected this for a while just based on how Boltzmann machines work. These are a type of unsupervised artificial neural network.

The training algorithm for Boltzmann machines has two phases:

1. (Awake phase) Feed the network training data, and increase the weights between neurons that tend to get activated simultaneously in this phase.

2. (Dreaming phase) Let the network run without any training input, and decrease the weights between neurons that tend to get activated simultaneously in this phase.

This training algorithm wasn’t designed to emulate how human brains work, it just sort of falls out of the math. And yet (1) resembles the Hebbian principle from neuroscience (neurons that fire together, wire together), and (2) resembles the process described in this article. Actually I suspect something stronger than what the article claims based on this: that the connections that get pruned are the ones connecting neurons that fire together while dreaming. It would be nice to have someone who knows stuff about neuroscience comment on this.

Re:The Purpose of Sleep? To Forget, Scientists Say by dontreact on Friday February 03, 2017 @02:02PM (#13562251)
I wouldn't be so sure that Boltzmann machines were designed without how the brain operated in mind. Terry Seknowski, one of the inventors, is a computational neuroscientist.

In general, I think the influence of neuroscience on neural network research has been subtle and perhaps underrated. For example, modern day convolutional neural networks Havea lineage going back to Fukushima's neocognitron, which was heavily inspired by Hubel an Wiesel's simple/complex cell model of visual cortex based on single cell electrophysiology.

Re:The Purpose of Sleep? To Forget, Scientists Say by mlechha on Friday February 03, 2017 @08:39PM (#13565072)
They weren't. They were a generalization of the Hopfield networks. Boltzmann machines are a stochastic version of the Hopfield network. The training algorithm simply tries to minimize the KL divergence between the network activity and real data. So it was quite surprising when it turned out that the algorithm needed a "dream phase" as they call it. Francis Crick was inspired by this and proposed a theory of sleep.
Re:The Purpose of Sleep? To Forget, Scientists Say by redler on Friday February 03, 2017 @11:59PM (#13565709)
Unpacking a comment like this is one of the quieter pleasures of reading HN.
Re:The Purpose of Sleep? To Forget, Scientists Say by mlechha on Saturday February 04, 2017 @08:09AM (#13566846)
Haha I'm not sure if you're being sarcastic so I'll try to unpack the comment. Hopfield networks were one of the first models of associative memory. They themselves were based on a model of simple magnets called ising model (generalized). Basically a group of binary units, each connected its nearest neighbors with a coupling strength. Each unit prefers to be like their neighbors. Hopfield developed a clever method to change the coupling so that the networks can store and retrieve patterns of activity. In the Hopfield network everything is deterministic, Hopfield himself realized that if this constraint was relaxed this model could become a very powerful computational machine. Which means that if instead of being always on or off, the units had a probability of being on or off the networks could perform very general computational tools [1]. Unfortunately, training these general stochastic systems was not easy. With their Boltzmann machines Sejnowski and Hinton proposed a possible solution. The activity of stochastic binary units effectively encodes a probability distribution, so all they had to do was make sure that the probability distribution being encoded by the activity of the units was the same as that of the input. They did this by changing the connection strengths between the units such that the activity pattern minimized something called the Kullback-Leibler or KL divergence, which is a measure of how close two probability distributions are (the one encoded by the network activity, or the dream activity of the network and the probability distribution of the real data e.g. a set of natural images). If two distributions match exactly then the KLd is zero and if not it's large. When they wrote out the math it turned out that the algorithm required two phases, an awake phase where the connections were changed according to the real data, and the sleep phase where the connections were pruned by the spontaneous activity of the network without any input (or dreams). This analogy got a lot of people excited, including Francis Crick and several others tried to test this idea in real brains, but we are still waiting for a convincing result.
Re:The Purpose of Sleep? To Forget, Scientists Say by p1esk on Saturday February 04, 2017 @12:39PM (#13567908)
Wow, this is a really good explanation! Thank you. I wonder though, are RBMs being used in practice? Are they worth any future research?
Re:The Purpose of Sleep? To Forget, Scientists Say by mlechha on Saturday February 04, 2017 @07:12PM (#13569940)
In applied machine learning not so much. They feel ancient! But some use them to study the physics of computation. They were used to make the connection between renormalization group (RG) and machine learning. RG is one of the main workhorses of quantum field theory and condensed matter physics. The fact that there's a mapping from RG to RBMs means that we can understand how deep learning works by using the same techniques that modern physicists use to understand the world! Here's a nice article on this topic if you're interested https://www.quantamagazine.org/20141204-a-common-logic-to-se...
Re:The Purpose of Sleep? To Forget, Scientists Say by vkreso on Saturday February 04, 2017 @12:26AM (#13565782)
Couldn't agree more
Re:The Purpose of Sleep? To Forget, Scientists Say by dontreact on Saturday February 11, 2017 @08:31PM (#13626045)
I don't disagree with this, but I also think that part of the interest in Hopfield networks and generalizing them to have stochastic outputs was because they seemed like a good model for how neurons might store memories or compute.
Re:The Purpose of Sleep? To Forget, Scientists Say by espeed on Saturday February 04, 2017 @12:54AM (#13565849)
Numenta [1] (co-founded by Jeff Hawkins, author of "On Intelligence") has been working on a model of ANN designed around how the brain operates. Hierarchical temporal memory (HTM) [2] is one aspect of their model.

1. http://numenta.com

2. https://en.wikipedia.org/wiki/Hierarchical_temporal_memory

Re:The Purpose of Sleep? To Forget, Scientists Say by tprice7 on Friday February 03, 2017 @02:17PM (#13562398)
"I wouldn't be so sure that Boltzmann machines were designed without how the brain operated in mind."

Sure, actually it seems highly unlikely to me that they were designed without considering at all how they brain works. Let me clarify what I meant: given the definition of how a Boltzmann machine infers (i.e. how to determine which neurons are on or off), the training algorithm can be derived from purely mathematical considerations.

Re:The Purpose of Sleep? To Forget, Scientists Say by jayajay on Friday February 03, 2017 @03:16PM (#13562923)
When I told Hinton about SPWs and that they are related to RBMs, he told me that he had a theory which required the existence of SPWs. That is not a coincidence, they seem to have been inspired by neuroscience and real neural network phenomena.
Re:The Purpose of Sleep? To Forget, Scientists Say by giardini on Friday February 03, 2017 @05:29PM (#13563912)
What's an SPW?
Re:The Purpose of Sleep? To Forget, Scientists Say by jayajay on Friday February 03, 2017 @06:41PM (#13564385)
SPW: High frequency oscillation in the hippocampus which occurs during rest and sleep. The characteristics would suggest they are critical in memory management.

If you google "SPW neuroscience", you will find many resources.

https://en.wikipedia.org/wiki/Sharp_waves_and_ripples

Brain science is highly interdisciplinary. Each expert should spend time to familiarize themselves with the various approaches used by other experts. In particular, machine learning theorists should spend time to understand the latest results (their implications, at least) in neuroscience and neuron biophysics. At some point, an ML theorist is not too different than a biophysics theorist. They just use different abstractions. This is why I am particularly interested in Google's and Hinton's work. That work seems to be subtly motivated by neuroscience and natural science...

Re:The Purpose of Sleep? To Forget, Scientists Say by jayajay on Friday February 03, 2017 @03:11PM (#13562873)
There is evidence to suggest that sleep and rest are important in verifying which memories to keep, as well. I did some undergraduate research (as a biophysics student) which actually looks at this from a neuroscience perspective. If you are interested in memory and neural networks, read about SPWs, from some scientists in the field:

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2039924/

http://neuralcircuits.uwm.edu/sharp-wave-ripples/

In summary, the hippocampus will organize neurons into clumps which encode spatial dimension. As a rat runs around throughout a track, certain groups of neurons fire[1] depending on where the rat is located. Before the task even starts, hippocampal neurons in a rat actually fire in quick succession (e.g. ABCDEFG). When the task is completed, the same pattern fires in the reverse order (e.g. GFEDCBA). Scientists believe that these SPWs play an important role in memory consolidation.

I have always thought of this as "the rat is thinking about where it's going to run to". So I started learning machine learning and landed on things like RBMs. I was so excited because RBMs are reminiscent of this actual phenomenon that occurs in the brain. So, I asked Hinton about it because this is no mere coincidence, and he told me that he had a theory that required reverse order firing during sleep state.

If you get a ton of new memories and patterns during wake, sleep may be responsible for not only pruning less important memories, but strengthening more important memories. For example, sleep might be asking the brain for a ranked list of memories, and then strengthens the memories above some threshold, and prunes the rest (just an idea).

[1] Synchronized firing, see Kuramoto coupled differential equations: https://en.wikipedia.org/wiki/Kuramoto_model

Re:The Purpose of Sleep? To Forget, Scientists Say by rectangletangle on Friday February 03, 2017 @06:38PM (#13564366)
Working memory, and concentration are adversely effected by sleep deprivation, which seems like another piece of evidence.
Re:The Purpose of Sleep? To Forget, Scientists Say by giardini on Friday February 03, 2017 @05:50PM (#13564047)
Anyone know what jayajay means by "SPWs"? Neither article speaks of SPWs but both speak of "sharp waves" or "sharp wave ripples"(designated "SWRs").

Maybe jayajay needs a nap!8-))

Re:The Purpose of Sleep? To Forget, Scientists Say by jayajay on Friday February 03, 2017 @07:14PM (#13564596)
I am not sure why you are attacking me. If you do not understand jargon, please read some more literature. SPW-R complexes. Look it up!

Edit: SPWs, Sharp Waves, SWR, etc. are just labels for sharp waves. I can't believe I am getting attacked for using jargon. The names don't matter. The physics and science matters. Richard Feynman.

Edit 2: The link I sent uses the same jargon that I use. So, honestly mate, what is your deal here? Are you trolling?

Re:The Purpose of Sleep? To Forget, Scientists Say by giardini on Friday February 03, 2017 @08:23PM (#13565001)
I am not attacking you. I don't work in neuroscience or biology.

You used a term (SPW) I'm not familiar with without defining the term, the three articles you referenced did not include the term, in the end we find the term is not an acronym, I have no knowledge of your training or background and my background is different from yours.

I was intrigued enough to briefly search for SPW but seeing nothing clearly enlightening, I asked for clarification.

My reference to your needing a nap is because you didn't initially define the term SPW, since most reading your post were likely unfamiliar with. No harm was intended. Not everyone knows what you know or thinks about what you think about, especially technical jargon.

Re:The Purpose of Sleep? To Forget, Scientists Say by jayajay on Friday February 03, 2017 @09:10PM (#13565221)
A simple CTRL+F (CMD+F Mac) "SPW" comes up with 7 results in the Wikipedia article I sent you. I didn't define it, but I mentioned why it was interesting, and I linked to an entire paper discussing the phenomenon! Anyways, I hope you read more and share your thoughts. Please no more personal remarks, just focus on the content!
Re:The Purpose of Sleep? To Forget, Scientists Say by giardini on Friday February 03, 2017 @11:57PM (#13565706)
When I searched for the string "SPW" at the wikipedia URL you originally posted (https://en.wikipedia.org/wiki/Kuramoto_model ) I found nothing. Hence my queries.

It is true that the article you posted later in response to my queries does indeed contain references to the term "SPW". Thank you for providing that clarifying link.

Unfortunately the terminology and origin of the term "SPW" (or "SPW-R")remains unknown to me, despite the second article using it freely. The article says, for example:

"... John O'Keefe investigated SPW-Rs in more detail in 1978 while studying the spatial memory of rat..."

Queries on Google seem to indicate that the term "SPW-R" refers to the same concept as the acronym "SWR" (a true acronym), but I'll never know, being too impatient to consider the matter further.

Re:The Purpose of Sleep? To Forget, Scientists Say by jayajay on Saturday February 04, 2017 @12:26AM (#13565781)
It is up to you. I tried to point you to the best of my ability. Let it be known. Anyways, I would appreciate if you removed the downvotes -- there is no good reason you have downvoted my responses to you.
Re:The Purpose of Sleep? To Forget, Scientists Say by yorwba on Saturday February 04, 2017 @01:38AM (#13565951)
AFAIK, no one can downvote responses to their own comments.

Apparently someone else found your posts to be dissatisfactory.

Re:The Purpose of Sleep? To Forget, Scientists Say by jayajay on Saturday February 04, 2017 @01:53AM (#13565993)
Well, what are you supposed to do when a user who is asking for clarification is really just trolling you?
Re:The Purpose of Sleep? To Forget, Scientists Say by pitaj on Saturday February 04, 2017 @02:54AM (#13566142)
I'm not sure if English is your first language, but giardini's first comment in response to yours does not seem to be in any way "trolling" or "attacking" you. None of his later responses seem to, either. You may want to check yourself, because there was no good reason to accuse him of trolling you. You are the one out of line. You started this with your overly-confrontational attitude.

> Anyways, I would appreciate if you removed the downvotes -- there is no good reason you have downvoted my responses to you.

As for this, obviously somebody thought your responses were inappropriate. I agree with them, and would have done the same if I had the necessary reputation.

Quoting the HN Guidelines: > Please resist commenting about being downvoted. It never does any good, and it makes boring reading.

Re:The Purpose of Sleep? To Forget, Scientists Say by jayajay on Saturday February 04, 2017 @03:38AM (#13566251)
I have every right to feel provoked if I believe someone else is intentionally wasting my time. That user, from my point of view, did not seem genuine in his interest in what I had to say.
Re:The Purpose of Sleep? To Forget, Scientists Say by grzm on Saturday February 04, 2017 @10:53AM (#13567355)
Don't respond. Don't feed the troll.
Re:The Purpose of Sleep? To Forget, Scientists Say by erikbye on Saturday February 04, 2017 @02:16PM (#13568478)
For what it's worth, at this point, both SPW and SPW-R are commonly used. SPW == Hippocampal sharp waves, SPW-R == Hippocampal sharp wave ripples. Often referred to as just sharp waves or sharp wave ripples.
Re:The Purpose of Sleep? To Forget, Scientists Say by ryukafalz on Friday February 03, 2017 @02:53PM (#13562697)
That's really interesting. From a naïve and purely pop-science perspective... if true, wouldn't this explain why it's hard to remember dreams after waking up?
Re:The Purpose of Sleep? To Forget, Scientists Say by spacehacker on Friday February 03, 2017 @07:38PM (#13564736)
There are brief sections about this in the Deep Learning book by Bengio, Courville and Goodfellow (2016):

> 18.2

Because the negative phase involves drawing samples from the model’s distri- bution, we can think of it as finding points that the model believes in strongly. Because the negative phase acts to reduce the probability of those points, they are generally considered to represent the model’s incorrect beliefs about the world. They are frequently referred to in the literature as “hallucinations” or “fantasy particles.” In fact, the negative phase has been proposed as a possible explanation for dreaming in humans and other animals (Crick and Mitchison, 1983), the idea being that the brain maintains a probabilistic model of the world and follows the gradient of log p ̃ while experiencing real events while awake and follows the negative gradient of log p ̃ to minimize log Z while sleeping and experiencing events sampled from the current model. This view explains much of the language used to describe algorithms with a positive and negative phase, but it has not been proven to be correct with neuroscientific experiments. In machine learning models, it is usually necessary to use the positive and negative phase simultaneously, rather than in separate time periods of wakefulness and REM sleep. As we will see in Sec. 19.5, other machine learning algorithms draw samples from the model distribution for other purposes and such algorithms could also provide an account for the function of dream sleep.

> 19.5.1 Wake-Sleep

One of the main difficulties with training a model to infer h from v is that we do not have a supervised training set with which to train the model. Given a v,we do not know the appropriate h. The mapping from v to h depends on the choice of model family, and evolves throughout the learning process as θ changes. The wake-sleep algorithm (Hinton et al., 1995b; Frey et al., 1996) resolves this problem by drawing samples of both h and v from the model distribution. For example, in a directed model, this can be done cheaply by performing ancestral sampling beginning at h and ending at v. The inference network can then be trained to perform the reverse mapping: predicting which h caused the present v. The main drawback to this approach is that we will only be able to train the inference network on values of v that have high probability under the model. Early in learning, the model distribution will not resemble the data distribution, so the inference network will not have an opportunity to learn on samples that resemble data.

Another possible explanation for biological dreaming is that it is providing samples from p(h,v) which can be used to train an inference network to predict h given v. In some senses, this explanation is more satisfying than the partition function explanation. Monte Carlo algorithms generally do not perform well if they are run using only the positive phase of the gradient for several steps then with only the negative phase of the gradient for several steps. Human beings and animals are usually awake for several consecutive hours then asleep for several consecutive hours. It is not readily apparent how this schedule could support Monte Carlo training of an undirected model. Learning algorithms based on maximizing L can be run with prolonged periods of improving q and prolonged periods of improving θ, however. If the role of biological dreaming is to train networks for predicting q, then this explains how animals are able to remain awake for several hours (the longer they are awake, the greater the gap between L and log p(v), but L will remain a lower bound) and to remain asleep for several hours (the generative model itself is not modified during sleep) without damaging their internal models. Of course, these ideas are purely speculative, and there is no hard evidence to suggest that dreaming accomplishes either of these goals. Dreaming may also serve reinforcement learning rather than probabilistic modeling, by sampling synthetic experiences from the animal’s transition model, on which to train the animal’s policy. Or sleep may serve some other purpose not yet anticipated by the machine learning community.

Re:The Purpose of Sleep? To Forget, Scientists Say by tprice7 on Friday February 03, 2017 @08:44PM (#13565106)
Hmm interesting. So it seems the Boltzmann machine explanation of dreams that I described becomes less convincing when you consider that running the waking and dreaming phases of the training algorithm in separate long "chunks", instead of simultaneously, seems to not work well in practice.
Re:The Purpose of Sleep? To Forget, Scientists Say by dicroce on Friday February 03, 2017 @04:49PM (#13563633)
(first, I should say I didn't read the article... I know, I know)...

BUT, something about this reminds me of simulated annealing. Just throwing that out there.