I am a complete machine learning noob, so this could just be my lack of skill. Having looked at most of the popular ML frameworks, none of them seem to provide an easy, functional way to implement a custom 'neuron'. That is:

  function (someinput):
      output = do_stuff
      return output

Which then gets passed on to the next layer. Again, it's quite possible I've just missed something, or there is some inherent ML limitation that makes this impossible. But if such a thing is possible, it would be a rather awesome feature from my POV.

Many do. For example, the `Lambda` Layer from Keras:

https://keras.io/layers/core/#lambda

Yeah tried that one a while ago but didn't have much success. I'm a bit of a dummy when it comes to ML. That said, it looks like they've updated the documentation or something, so might be time to give it another whirl. Thanks for the reply.

This is another area CNTK's 'BrainScript' excels. You can pretty much express it as normal maths:

https://github.com/Microsoft/CNTK/wiki/BrainScript-Activatio...

List of functions:

https://github.com/Microsoft/CNTK/wiki/BrainScript-Full-Func...

tensorflow's py_func allows you to do that, you can also write your own C++ operators, their API interface for that is pretty nice. In fact, you should be able to pretty much write any op you need by combining existing ops (maybe not as efficient as you'd like but yields the results you want).

Sorry I'm late to reply. This looks perfect for what I have in mind! Thanks!

The recurrent network work is under way - we are revisiting the designs to consciously balance between performance and API niceness. We'll share more tech details in the upcoming days. Right now, this is an example RNN:

https://github.com/caffe2/caffe2/blob/master/caffe2/python/e...

We use it for machine translation so the perf is nice.

If you have design feedbacks please let us know - we'll be really grateful.

I know one of the biggest struggles of training RNNs in TensorFlow, for instance, is the fact that the graph is static. Other frameworks, like PyTorch, have dynamic graphs.

Will Caffe2 support dynamic graphs (perhaps down the line)? What is your reasoning behind whichever decision?