Hm, please correct me if I'm wrong, but isn't the Zuker algorithm inherently a dynamic programming algorithm? If so, I would have liked to see this paper compare to directly optimizing the input space using a differentiable form of the Zuker algorithm, e.g., using ideas related to https://arxiv.org/abs/1802.03676.

IMO it's unnatural to rely on deep reinforcement learning when we know so much about the mapping between input space and reward space (here, because Zuker's algorithm is not a black box). To me, deep RL is primarily for the domain where the mapping is highly complex and largely black box. Deep RL might be more useful if the authors didn't use Zuker's approximate algorithm but instead performed molecular simulations to determine secondary structure folding that is more faithful to reality.