Both images have their patterns extracted by the NN, and the optimization then tries to match the best patches from one image with the other, performing gradient descent to adjust the pixel values from a random start image.