I don't think there has to be. Many libraries for example overkill it and make everything a layer when it should be a layer with a hyper parameter. A good example of this is drop out. Why not make it a knob rather than a layer?

I think people should be given high level primitives like "layers", allow them to make their own where necessary, but allow the defaults to be: Layer with x (dropout,momentum,..) trained by optimization algo: LBFGS,Hessian Free,.. This allows people to experiment with different configurations without having to dive deep to achieve some basic problems.

Relevant to julia: it's a great language and what I wish production code could look like (while being fast!)

Like rust, it's in a pretty alpha state right now. I'm watching the language heavily though.