Probably not all these things need per-pixel metadata, but anisotropy exists and that means many of the variables you'd think are per-exposure are actually dependent on where in the exposure the individual pixel is. For instance, illumination level isn't uniform across a field of view for all cameras, and may need to be normalized.