Tensorflow is getting dynamic jit optimization too. I think part of the reason that some dynamic optimizations might perform better is that the results of the optimization can be cached for most other batches and they can specialize to utilize batch/shape/input specific properties

I'm kind of getting the sense that TF is presently being optimized for its own massive development: that is, the engineers are yanking out chunks of code and replacing them, quickly, and at varying scales.

A nitpick, but an important one: Only Pytorch uses a dynamic computation graph. Torch doesn't have any concept of a computation graph.

TF has runtime cost based optimization, which is inside Session.run()