RE the performance - many frameworks nowadays enjoy the high-performance libraries such as CuDNN for optimized runtime on different platforms. For example, we've been collaborating with NVidia on optimized distributed training on GPUs, and you can check out more details here: https://blogs.nvidia.com/blog/2017/04/18/caffe2/

Thanks for the link. Are there any public specifics comparing distributed tf to Caffe2's distributed implementation?

I am not aware of that right now. Internally we measure the performance by looking at the theoretical peak possible (like the scaling efficiency when you use distributed training). C2 has been doing pretty well; if you find performance degradations, send an issue to us and we are usually very good to figure out the perf.

MKL is going to be merged very soon - a full MKL pull request is here if you'd like to track it:

https://github.com/caffe2/caffe2/pull/270

And thanks so much for the awesome work from Intel!