News for hackers. Stuff that matters.
Re:My phone case, the hidden state of an RNN by only_as_i_fall on Tuesday September 07, 2021 @03:52PM (#28449060)
Looks great but I'm surprised they aren't way noisier. Is that just a result of having a limited number of neurons compared to the number of pixels, or is there some specific trick to it?
Re:My phone case, the hidden state of an RNN by douglasorr on Thursday September 09, 2021 @04:05AM (#28467413)
I think at least some of the structure is down to the update equation, roughly 'new = sign(gate * old + ...)'. Since 'gate' is always positive, this means that even when the network is randomly initialized, the state is more likely to stay the same between two time steps than to change.

I tried "deep transitions" which repeats this equation a couple of times, and the results were indeed noisier.

Re:My phone case, the hidden state of an RNN by maxwells-daemon on Wednesday September 08, 2021 @12:22AM (#28453004)
It's mostly that these are very narrow and shallow networks -- if I remember right, 5 layers and 30 units wide?
Re:My phone case, the hidden state of an RNN by iaw on Tuesday September 07, 2021 @07:13PM (#28450993)
Not OP, but I imagine you can set maximum limits on the rate change of the RGB vector so it doesn't create a static but a more fluidic appearing image.