I consider deep networks as experiments. People try incrementally different models , and the results keep getting better. At some point the theory behind them will advance to the point where we can analytically describe them.