News for hackers. Stuff that matters.
Re:TensorFlow Fold: Deep Learning with Dynamic Computation G... by moshe on Tuesday February 07, 2017 @11:38PM (#13595797)
Thanks, insightful questions.

You're absolutely right about combinators and long-range dependencies. We have a special block type in the high-level API (https://github.com/tensorflow/fold/blob/master/tensorflow_fo...) for accumulating results here without having to explicitly shuttle them around; not perfect but very handy in many cases.

Regarding your second question, the equivalent of padding in dynamic batching is "pass through" do-nothing ops, which are introduced transparently but worth being aware of if you want to understand the machinery. The worst case scenario here is chains of wildly varying lengths, where we need to add pass-throughs to the shorter chains to match the length of the longest chain.

There are no comments on this comment yet.