A smartphone camera, DSLR or other camera does not work any differently though. It's just that an array of color filters is fixed above the sensor's pixels which would also be monochrome if it were not there. The readout of these pixels is automatically converted to a color image then, or stored as RAW for later processing.

This color filter array is called a "Bayer filter".

https://en.wikipedia.org/wiki/Bayer_filter

I'd say there's still a difference in that while DSLRs and smartphone sensors have individual readouts for each colour channel, scientific instruments do not. So a DSLR or smartphone camera still outputs a multichannel image with a single sensor while scientific instruments are strictly monochrome.

They don't take multi-channel pictures in a single shot and each full image represents a single channel. HST for example has three cameras - one per channel - while the MER cameras have a single sensor with multiple filters.

From what I know sensors do not have different readouts for the different color channels. All pixels are read out the same way and, with knowledge of the pattern of the Bayer filter in use, are then processed ("de-mosaiced") into a color image.

If one removed the Bayer filter from a sensor (as difficult as that is) and skipped demosaicing while processing the image, one would end up with a monochrome image just fine. Using physical color filters for different channels and taking the same scene for each of them, one could create a single color image with even better resolution than the camera normally would. Why better? - this might disappoint you now! - consumer cameras count each differently colored 'sub'pixel as a full pixel. So half of the "megapixels" of your camera are e.g. green and just a quarter of them each red and blue (that's a common ratio for the colors). The camera then 'fakes' having a full set of RGB pixels by demosaicing, see https://en.wikipedia.org/wiki/Demosaicing ). Taking monochrome image with physical color filters would allow to use the full resolution of the camera for each color channel.

That's exactly what I wrote, though.

> From what I know sensors do not have different readouts for the different color channels.

You know wrong then. HST uses three camera sensors precisely to have three different filters at the same time and at full resolution: https://www.stsci.edu/hst/instrumentation/legacy/wfpc2

So yes, the WF/PC2 does indeed have different readouts per channels because each channel has its own dedicated sensor.

Maybe you simply misinterpreted what I wrote. The fact of the matter is that consumer level sensors return multi-channel images (the sub-pixel filtering is an irrelevant technical detail), while scientific instruments use dedicated sensors per channel.

edit: just to clarify even further - there are no subpixel shenanigans with scientific instruments for both higher precision and to avoid cross-talk between frequency bands; hence separate sensors per frequency filter. if you want multiple channels at once (e.g. space telescopes), you therefore need multiple sensors as opposed subpixel filtering.

OK, I think I understand the confusion now. I understood "readout" as hardware implementation detail (think: "readout circuit") while it seems to refer to the data of a particular channel, right? My lack of familiarity with the lingo seems to blame then.

That "subpixel shenanigans" on scientific apparatuses are inacceptable should be understood.

What you are saying doesn’t make much sense. It’s obvious that the images are taken on different wave lengths and recombined. Cameras with a Bayer filter use exactly the same principle but have worse spatial resolution because they have micro-filters on each pixel and the result is interpolated. Foveon sensors work in the same way with different layers sensible to a specific wave length. Even our eyes work in the same way with the rods and cones that are sensible to luminance and different wavelengths. By your definition what we see are not colour images since they are processed after the fact by our brain from the separate wave length signals coming from the receptors in our eyes.

You misinterpreted what I wrote then.

Yes, a consumer-level camera sensor uses Bayer filtering, but that's just an implementation detail just like the rods and cones inside the retina - you don't perceive each channel individually after all, lest you wouldn't be able to perceive "brown" or "pink".

So any analogy between CCD sensor output and human eyes ends right there.

The difference lies in the output of the sensory mechanism (be that human eyes or CCD sensors). While consumer level hardware outputs multichannel images (yes, by means of subpixels and Bayer filtering, but that's irrelevant - for the sake of argument it could as well be pixies), scientific instruments use separate CCD sensors for each recorded channel to a) keep the full resolution and b) avoid cross-talk between different frequency bands , which is unavoidable with the subpixel setup used in commodity hardware.

So while your smartphone or DSLR's RAW output will be a multichannel image, instruments onboard of spacecraft will only ever output a single channel per sensor. No subpixels, no Bayer filters. Either one dedicated sensor per recorded channel (e.g. HST WF/PC2 camera) or multiple images taken in series using a single sensor but multiple filters (e.g. cameras onboard MER and MSL rovers).

I hope that was a little clearer.

The Foveon sensor that I mentioned before works exactly in the same way getting 3 full resolution images at different wave lengths, exactly like the scientific cameras. And I’d argue that the only difference between this approach and the eye/Bayer sensor is the resolution, the underlying mechanism is exactly the same.

But does the Foveon sensor use RGB filters or does it work with different frequency bands? That's a major issue with science data - it doesn't contain RGB data for the most part, because that's not what scientists are interested in.

What's the correct colour channel to use for for near UV (yes, I know that some people - mainly women - can perceive that, but the vast majority of people can't)? What matches near IR or even X-rays? How would you colourise the frequency bands that match different elements (those include very narrow bandpass filters) but don't really make sense in terms of RGB?

That's the difference right there. If you have a picture that uses channels of different filter bands that correspond to hydrogen bands, organic molecules or UV radiation, none of that data correlates to human colour perception. So someone just chose shades of green for hydrogen or shades of blue for organics.

So what I'm trying to say is that the underlying mechanism is irrelevant - the data simply isn't colour data as seen by humans.

I'd argue if you have sensors that take images in multiple wavelengths, and they are combined, it amounts to a imaging system (=camera) that senses colors - vs upthread argument of "colored black and white images".

The issue is that the multi-wavelength combined image doesn't correspond to RGB like the RAW output of DSLR or smartphone camera.

The different bands received from spacecraft, rovers and (space-)telescopes are matched with red, green and blue, even though they're actually taken in near infrared, UV, or even X-ray spectra.

The "truth" is that the images are only monochrome representations of recorded photons of different energies. Most people don't see near UV light for example and some images are taken in very narrow bands and then "stretched" into "full" RGB, etc.

The point is that the recorded data doesn't correspond to the human vision apparatus (or at least not well) an thus any multi-colour representations are necessarily fabricated and sometimes don't even try to reflect what a human would have seen.