'The importance of being linear' is the best introduction to the topic I know: http://http.developer.nvidia.com/GPUGems3/gpugems3_ch24.html
https://en.wikipedia.org/wiki/SRGB
However, one cannot do computations on an image in sRGB colorspace since the colorspace is not linear - the values will turn out wrong. I.e. imaging algorithms expect source data to be in linear colorspace, and if fed sRGB data the result will turn out more or less wrong.
Thus, when doing any weighting calculations etc. on a set of samples the samples must be in linear (e.g. RGB) colorspace.
Your OpenGL implementation may or may not handle this automatically for you if you specify the image to be in one of the sRGB texture formats. https://www.opengl.org/registry/specs/EXT/texture_sRGB.txt
So, in the general sense, if doing any handcoded imaging routines, and reading e.g. PNG image, one needs to convert it to linear colorspace before processing, and back to sRGB before writing to disk/displaying.