To me the simplest place to start is Fourier Optics. A carefully set up collection of lenses, illuminators, samples, and cameras can compute (well, physically evaluate) the fourier transorm of a 2D image. https://en.wikipedia.org/wiki/Fourier_optics (so you can for example do convolutions)

From what I can see in this area, the physical set up mainly computes forward passes by passing light through the SLM, while the computer computes the backward pass (using a mathematical equation that approximates the forward optics and computes its derivative). The nice thing is that SLMs are simply programmed to have a specific intensity pattern, it doesn't need a whole manufacturing pass to update the SLM.