Wavelength specificity. You only get the original focal depth at the wavelength you used to make the hologram, and it changes proportional to the wavelength. When you consider that blue light is 2/3 the wavelength of red light, you can see that this is chromatic aberrations big brother.