Here's a psuedocode rundown of exactly what it does:

  network = new NeuralNetwork()
  
  targetOutput = readFile('./images/zebra-500x500-corrupted.jpg')
  input = generateNoise(500, 500)

  while(iterationCount < overfitThreshold) {
    networkOutput = network.getOutput(input)
    loss = getLoss(targetOutput, networkOutput)
    network.backPropogate(loss)

    iterationCount++
  }

  writeFile('./images/zebra-500x500-denoised.jpg', networkOutput)

> Can someone explain this in a way so that an ordinary mortal computer scientist can understand it?

I'll try.

Instead of the common approach that tries to search for image pixels to minimize e.g. a denoising objective, they realize that they can instead search for the weights of an image generator network such that the generated image matches the objective.

They argue that the structure of the network then constitutes some prior knowledge over what a natural image should look like.

My (probably wrong) interpretation: since a convolutional neural network essentially works by looking for some spatial patterns at different resolutions, their optimization process boils down to finding the high and mid-resolution patterns that best match the input image, and then re-using that information to fill in the missing information (or replacing "noise" that does not match the extracted pattern).

What do they mean by ‘the structure of the network’ is this the network topology?

There's a great explanation of the idea here in the comments : https://news.ycombinator.com/item?id=15820994

There exist neural network structures that can solve the image processing problems shown when weights are randomly applied.

This means that for these problems the solution is encoded entirely in the network structure and not the weights.

We could use linear regression as an analogy: y = b f(x).

This is akin to saying that for some class of problems, it's the case that you can get as good of a solution from the function you pick alone (x^2, log(x)) as you could from picking a function and then computing the best coefficient. Note this isn't true for linear regression for any meaningful problems, but I feel like it's a decent analogy.