Re:Applications of Deep Neural Networks v2 [pdf]
by ohnemint on Monday January 25, 2021 @05:42AM
(#25901741)
> Training a neural network is like training a dog, and the loss function just describes when you'll shout "no!" and how loudly.
This is a pretty good ELI5 neural networks.
Re:Applications of Deep Neural Networks v2 [pdf]
by thermistokles on Tuesday January 26, 2021 @02:40PM
(#25920182)
> But turns out, a hierarchical structure can NOT correctly represent some movement patterns in the real world, like branches on a tree moving or overhead cables.
Could you expand a bit on this? Do you mean that it might miss small fast moving objects due to losing fidelity at the coarse resolutions. Or is there actually some sort of movement that the hierarchical structure can't interpret.
Re:Applications of Deep Neural Networks v2 [pdf]
by fxtentacle on Friday January 29, 2021 @04:52AM
(#25955716)
It might miss anything where the average structure size is smaller than the large hierarchical blocks. For most SOTA, that means 32px minimum size. So it'll also miss fences, for example, because the wires are too thin and it'll not treat it as a whole but as separate tiny objects.
However, I believe that coding the network is very challenging unless you do a task that has been widely explored already. For optical flow, there was a wide consensus among SOTA papers for some years that convolutional filters, warping of the input data, and a hierarchical structure was the correct way, e.g. everything descended from FlowNet.
But turns out, a hierarchical structure can NOT correctly represent some movement patterns in the real world, like branches on a tree moving or overhead cables. So now we have a category of AI solutions that all fail in the same way in the same circumstances, plus commercial products (e.g. Skydio Drone) with the exact same issues.
The correct approach seems to be a iterative solver approach, which has been attempted with RAFT, but nobody has yet managed to design a suitable network architecture that does not require hierarchical undersampling.
Just like in your failure story, tiny mistakes in the network can prevent success for good. And you need lots of attention to detail and plenty of experience to avoid those mistakes.