When you do computer vision, the first step you do is convert your color image into a black and white image, and run your CV algorithms on the black and white image. This is because when you're looking at objects and shapes and stuff, it's contrast that tells you where the boundaries between things are. This is true even in a human world of human objects, which tend to be many colored. It's even more true on Mars where basically everything is varying shades of orange. So having color doesn't help a whole lot, and you also have to do the additional step of converting the color image to black and white, which takes CPU power and adds latency. Remember, the purpose is hazard avoidance- latency is bad.
Additionally, color camera sensors aren't actually color sensors. They're black and white sensors. In front of every pixel on the black and white sensor is a filter that is either red, green, or blue. Pixels are grouped into sets of four, and there are two pixels with green filters, one pixel with a blue filter, and one filter with a red filter. (sometimes one of the green filters is omitted, giving red, green, blue, and b&w, or sometimes one of the green filters is a filter that allows IR, or something like that.) So if you have a 16MP camera, the camera has 8M green, 4M red, and 4M blue pixels. This means two things; first of all, if you just wanted a black and white image in the first place, a color sensor gives less detail than the equivalent black and white sensor, and second, you need to do additional processing to convert the raw output from the sensor into an image that's usable for anything. The additional processing adds latency.
I have a feeling I'd be the angry guy in the meeting who wouldn't accept the consensus. "but what about latency! what about the descend and landing!" shakes fist
-Worked at JPL for a few years and have dozens of friends, a few in the vision system.
Long answer: Colour is a very rabbithole topic but Captain Disillusion has a summary of it (https://youtu.be/FTKP0Y9MVus) and Technology Connections has a discussion (https://youtu.be/uYbdx4I7STg).
We can notice that when people say they perceive "yellow" that the spectral intensity graph has certain patterns. This is the physical phenomenon that produces the sensation of "yellow."
Humans are not good at judging reality introspectively. We experience everything heavily filtered through a variety of lenses. Our feeling that color is "concrete" is not predictive or explanatory... we cannot build mechanisms based on it. The idea that our perception of color is a result of interactions between certain wavelengths of light and certain photosensitive tissues in our eyes is both predictive and explanatory. We can design systems that have similar types of wavelength intensity sensitivity components and measure the physical response of those systems. That's how cameras work.
We can reverse the process and take those measured wavelength intensities and re-emit them from variable-wavelength light sources and produce images. That's how you're reading what I've typed right now - the images produced by the display you're looking at were generated in this fashion.
I'm not sure what you mean by the “wavelength theory” of color perception.
Of course we can. We can capture the signal sent through the optical nerve and then reproduce it as a stimulus which will make the brain “see” yellow color.
Besides, humans are capable of distinguishing literally millions of colors, of which just a tiny fraction can be attributed to measuring particular wavelengths (or, more accurately, particular energies of the incident photons). In that way the eye is different from the ear (which performs a kind of Fourier analysis of the sound wave).
I agree that there are sensory perceptions humans are capable of perceiving and labeling as colors that cannot be attributed to external physical phenomena, but those are largely artifacts of the way our brain processes signals. For example if you stare at a purple dot for some time, then look away, you'll perceive a yellow dot where there is no external set of photons corresponding to the wavelengths that normally trigger the sensation of yellow striking your retina.
This is just more explanation about how "yellowness" is a characteristic of our brains, not of the external world.
Or did you mean something other than what I'm referring to here? I think that for the vast bulk of humans, the vast bulk of the colors they perceive regularly are due to photons striking rods and cones in their eyes at various intensities, causing color sensations to occur in the brain. Do you think something else is happening?
You seem to understand how the eye works, and some neuroscience, so I don't understand how you can have the questions that you raise about whether we can build cameras that sense "color" instead of "light"
The human eye has four basic cell types, rod cells and cone cells, and there are three subtypes of cones, short, medium, and long. The three subtypes of cone cells sense blue, green, and red light more or less directly. Medium and long cone cells, which directly detect green and red light, almost entirely overlap. [0] It is more accurate to say that long cone cells detect yellow light than it is to say it detects red light. There is a brain system which measures the difference in response between the long (red) and medium (green) cells and uses the difference to say "aha! this must be red!"
The ratio of short (blue) medium (green) and long (red (yellow)) cone cells are roughly 2%, 2/3, and 1/3. The cells in your eye which detect blue light are more or less a rounding error. The cells which detect green light are roughly twice as numerous as the cells which detect red (well, yellow) light. If you see a thing and think, "man, that's awfully blue," it's not because your eyes are telling you "hey, this thing is awfully blue". The "blue" signal is barely noticeable in the overall signal; but your brain jacks up its responsiveness to the minuscule blue signal.
One of the side effects of the completely fucked ratios between the three types of cones is that your perception of the overall brightness of a thing is mostly down to how green it is. This shows up in lots of standards; NTSC, JPEG, the whole nine yards. If you've ever implemented a conversion between RGB and any luminosity-chroma colorspace (YUV, YCbCr, YIQ, NTSC, any of them) there's a moment where you'll go "wait a minute this doesn't make any fucking sense". You look at the numbers and the luminosity channel is just... green, and you know that the other two chroma channels are quartered in resolution. And you'll think that makes no sense. But that's how it works.
Then you'll remember that color sensors have their pixels arranged in groups of four, with two green, one red, and one blue channel. There must be some green conspiracy.
And there is. It's your brain. It's your eyeballs with 2/3 of its cone cells being green sensitive ones.
Those are your cone cells. Rod cells are entirely different. It's trivial to say well, cone cells see color, rod cells see black and white, but it's more complicated than that. Rod cells are excellent in low light conditions, cone cells not so much. Cone cells see motion very well, rod cells not so much. Cone cells can discern fine detail, rod cells do not. Rods and cones are not evenly distributed across the retina either; cone cells are densely packed in the center, rod cells are more common in peripheral vision.
Look at a colorful thing directly; take a note of how colorful it is. Now look away from it, so it's only in your peripheral vision; take a note of how colorful it is. Does it seem just as colorful? It isn't. That's your brain fucking with you. Your brain knows it's in your peripheral vision and all the colors are muted out there, so your brain exaggerates the colorfulness. Cone cells are 30 times as dense in the center of your vision as they are just outside the center of your vision. [1] That's why you can read a word directly where you're looking but it's very difficult to read elsewhere.
The reality is that your retinas give a fucking mess of bullshit to your brain, and the brain is the most incredible image processing system conceivable. It takes bullshit that makes no damn sense and -- holy shit I forgot to talk about blind spots.
Ok, so your rods and cones have a light sensitive thing, with a wire in the back, and all the wires get bundled up in the optic nerve that goes to the brain. Here's the thing: they're fucking plugged in backwards. The wires go forward, and are bundled up between your retinas and the stuff you're looking at. The big fat optic nerve therefore constitutes a large chunk of your vision where you can't see anything. Your brain just.. invents stuff where the optic nerve burrows through your retina.
Other weird stuff. If it's bright, the rods and cones send no signal, if it's dark, they send a strong signal. It's inverted. There's apparently a very good reason for this but I don't remember what it is. Also, the rods continuously produce a light sensitive substance that amplifies the light sensitivity but is destroyed in the process. It takes a long time to build up a reserve. This is why it takes time to "build up" your dark vision, and why it's so easily destroyed by lighting a cigarette. The physiology of "ow it's bright" as opposed to "it's bright" isn't just on your retinas, it's also on your eyelids and your iris, but more importantly, it's shared between your two eyes. This is why closing one eye makes it less painful when you go from a dark place to a bright place.
The point is, the study of human vision is not the study of the human eye. The study of human vision is the study of the human brain.
Much of what we do with color spaces and image compression is dictated by our stupid smart eyeballs and our stupid smart brains. Video codecs compress with 4:2:0 chroma subsampling because the brain's gonna decompress that shit better than a computer can anyway. Cameras have twice as many green sensitive pixels as blur or red pixels because the eye resolution is much sharper in green than other colors. More advanced image and video compression schemes will try harder to account for human eye-brain physiology.
[0] https://upload.wikimedia.org/wikipedia/commons/0/04/Cone-fun...
[1] https://upload.wikimedia.org/wikipedia/commons/3/3c/Human_ph...
The reason is to prevent light fatigue in eyes. Ears and nose experience a quick fatigue when exposed to the same stimulant for a long time. With inverted arrangement in eyes, you have a naturally stimulated inhibition rather than a fatigue inhibition.
After you get done exploring how we perceive colors associated with different wave lengths of light, and how nobody really knows whether these are common somehow, or unique to each of us, that sentence should bring you both a chuckle and some wonder about perception.
I am inclined to believe it is, but we do not really know.
The lower "HazCams" hazard avoidance cameras (which captured those initial photos) are there to detect hazards (rocks, trenches, etc.). They are stereoscopic, lightweight, and high resolution.
My guess is that using color sensors would have either increased the 3D mapping precision or added weight/power/bandwidth requirements, or otherwise been less robust in that environment.
Those cameras were also pre-deployed for the landing phase and likely transmit more quickly due to the lower data information. The other cameras were shielded for the landing phase.
The navigation and other cameras are in color, and I expect we'll be seeing better images shortly.
[1] This comes to mind whenever a question like that is asked: http://4.bp.blogspot.com/-CWM1zDcmWXs/TroD0VsX4WI/AAAAAAAAAV...
I think you meant to say decreased? In which case I think you would be correct! Camera pixels are made up of these things called photosites which don't by themselves record color, only brightness. In order to record color information, the photosites are placed behind a Bayer filter[1], which effectively reduces the resolution of the camera by 3, because in order to get the color of a pixel you need its red, green and blue component. Bayer filters also frequently have a small blurring filter in front of them to make sure that nearby photosites with different color filters get the information they need.
If you're looking for the highest resolution image possible, black and white is the way to go!
That way you get high regulation as well as color. You can also have some special (infrared, ultraviolet, etc.) Filters on the carousel, not just RGB.
and BAM, false color! FTFY
I did, thank you. I think my brain had already skipped ahead to the added weight/complexity concept while my fingers were stuck on that part of the sentence.
I should probably read things after I type them...
What are they going to do next ? Put on board a solar powered Mars helicopter ?? ;-)
These are hazard cameras, designed to be inputs into the guidance algorithms on board. It might make sense for such a camera to be B/W to reduce on board processing required. There's also a glass cover on them, and a lot of dust from the landing, so that may be obscuring true color if the cameras do in fact take color images.
Also they may have just transmitted a lower quality B/W image to get something back to Earth quickly, since higher res images take longer to uplink.
also is it technically correct to call the Martian atmosphere "air"?
https://www.imdb.com/title/tt0080745/goofs
Flash Gordon (1980) Goofs
At the very beginning of the film, Ming and his henchman are discussing "an obscure body in the SK system", which the inhabitants refer to as the planet "Earth", pronounced as if the word is completely foreign to them. However, at that moment, Ming activates a button on his console labeled "Earth Quake".
http://bobcanada92.blogspot.com/2020/10/flash-gordon-logic.h...
> the mixture of invisible odorless tasteless gases (such as nitrogen and oxygen) that surrounds the earth
> also : the equivalent mix of gases on another planet
I would naively guess yes to part one but it's complicated: Mars has less gravity, much less atmospheric pressure, colder temps, and greater gravitational influence from its moons than Earth. Wikipedia says the mechanism of the planet's dust storms isn't well understood.
https://en.wikipedia.org/wiki/Atmosphere_of_Mars#Dust_and_ot...
Also doesn't help that there is a (transparent) lens cover in front of the lens obscuring the view.
Elon Musk needs to provide some Starlink sats for a better connection.
What I could imagine is having Starlink satellites around Mars that allow to route data from rovers anywhere on the planet to a dedicated high-performance communications platform that handles communication with Earth.
It's just that since there have never been more than a handful of spacecraft active on Mars at any given time, there's currently no point in spending huge amounts of money to launch a whole constellation of satellites for continuous coverage.
https://mars.nasa.gov/msl/mission/communications/#data
"The data rate direct-to-Earth [from Mars] varies from about 500 bits per second to 32,000 bits per second"
> 160/500 bits per second or faster to/from the Deep Space Network's 112-foot-diameter (34-meter-diameter) antennas or at 800/3000 bits per second or faster to/from the Deep Space Network's 230-foot-diameter (70 meter-diameter)
for high-gain antenna, and
> Approximately 10 bits per second or faster from the Deep Space Network's 112-foot-diameter (34-meter-diameter) antennas or approximately 30 bits per second or faster from the Deep Space Network's 230-foot-diameter (70-meter-diameter) antenna
for the low-gain antenna, which I believe the first two images were sent through
https://mars.nasa.gov/mars2020/spacecraft/rover/communicatio...
It seems that NASA is being awesome and making all raw images available as they get them. So far just the 2-ish.
Guessing its black and white/high contrast to help see rocks etc. And probably much lower res, smaller file size too for transferring.