And speaking of SNR, there are a few tricks astro-photographers have been using for years that occasionally find their way into mainstream photography. One is image stacking, where you combine multiple photographs of the same scene (useful for landscapes if a little blur isn't a problem for you) into a single image. This pushes the shot noise (the random distribution of noise) down even further. I've gotten clean images out of cheap cameras, and even from cheap cell phone cameras (given enough images).
The other technique is dark frame subtraction, where you take an photo with the lens covered at the same ISO/shutter/aperture as the rest of the set. Digital sensors have readout noise and heat noise which tends to be very predicable. Subtracting this noise gives you a cleaner shot as well.
On a side note, I do wish when talking about ETTR and even digital photography in general that the term "overexpose" wasn't used when explaining the technique of overriding the cameras default metering with exposure compensation. Overexposure would be blowing the highlights, or at least the highlights of details that the photographer considers important (e.g. blowing the highlights of street lights is usually okay). The "over" portion makes it sound like the photographer is doing too much of something, when in reality ETTR is a technique of giving enough exposure, but not too much.
Since you seem to have experimented with this, I'd like to ask: don't you need a certain "minimum" time to capture the faintest of stars, because their signal otherwise wouldn't rise above the noise floor? I've been curious about this with stacking in atrophotography for a while.
> The other technique is dark frame subtraction, where you take an photo with the lens covered at the same ISO/shutter/aperture as the rest of the set. Digital sensors have readout noise and heat noise which tends to be very predicable. Subtracting this noise gives you a cleaner shot as well.
For long exposures this heat noise becomes more problematic: long exposures heat up the sensor quite a bit. Have you tried building your own customised electric coolbox to keep temperatures down? It can be a fun project!
I actually used this in a different context: during long Skype sessions at home with my people abroad, my phone heats up a lot and the video would start stutter. Putting it on a cooling block actually fixes that. This had more to do with the system not having to scale back CPU performance to let the phone cool down, but a side-effect according to people on the other end of the line, was that the image quality improved too
I'm only guessing, but there must be a minimum time for the exposure. Stacking won't change that. My assumption (and I'm not a statistician) is that the noise floor moves lower with stacking. Taking multiple images and stacking them is basically taking multiple samples. The random (Poisson shot noise) is averaged out across multiple images while the signal (the star) is a constant across the images.
As you can see from the linked article, there's an amazing amount of detail that can be recovered from the shadows if you know how to get to it. In a dark sky environment, you'd be limited to the camera's sensitivity (the analog gain applied using the ISO setting), but in the city environment, almost certainly the sky light is the limiting factor.
I have not experimented with custom cooling. Aside from some moon photos and photos of the recent partial (in my area) solar eclipse, I'll leave astrophotography to others.
I'm not an expert or even very knowledgeable on image stacking, but I'll share my experience.
I've taken a stacked picture of the Orion constellation. 15 exposures at 5 seconds each, and combined them to get this photo:
https://www.flickr.com/photos/abliskovsky/31756623694/
In none of the individual exposures is the Orion Nebula visible, but in this photo it's pretty clear. So in the end I'm fairly sure that your statement is true, but the signal required is very, very faint.
Ok, just for fun: I'll try to make a quick back-of-an-envelope calculation and see if we can make an educated guess...
Let's use the old Sunny 16 rule, and its equivalent for moonlight, which also gives us an estimate for difference in light between day and night[0]:
> Daylight (sunny day): Correct exposure for this case is given by the Sunny 16 Rule: 1 / ISO [seconds] @ f/16. So at ISO 100, a typical exposure is 1/125 @ f/16 (that's 1/3 stop less exposure than the rule calls for, but it's the closest standard shutter speed.)
> Full moon: to get an equivalent exposure at night with a full moon, the rule is: 1 / ISO [days] @ f/4. That's right, DAYS. For ISO 100, that means 1/100 of a day @ f/4.
> That works out to something like 14.7 minutes; I just round it up to 15 minutes. So 15 minutes @ f/4, or FOUR HOURS @ f/16.
> That's 21 stops difference from sunlight to moonlight.
So on a sunny day you need f/16, 100 ISO, shutter speed 1/100.
flickr says you shot at f/5.6, unknown ISO (I'll go with quasi-conservative 6400 ISO), 5 seconds.
Aperture increased by 1.5 stops
ISO increase by 6 stops
500x longer shutter time => log2(500) -> 2^9 = 512 => bit less than 9 stops.
So that's between 16-17 stops of extra light sensitivity
That's five stops short of the 21 stops mentioned above. However, we're not aiming for a properly exposed single picture, we're aiming for minimum exposure to capture a signal!
Even the first digital SLRs had a dynamic range of five to six stops, so theoretically should be able to capture something in the shadow regions. Nowadays ten stops is about the norm I believe, with some of the better cameras going up to twelve. So that should clearly be above the minimum signal required
Also, the moonlight-rule above is very conservative: it's about exposing a scene in moonlight, not about capturing stars. Stars are light sources themselves, and probably brighter than objects reflecting moon light, even in a full moon. Also, the rule-of-thumb was based on film, which actually suffered from something called "reciprocity failure"[1] so it might exaggerate the required exposure.
So yes, this should work out fine, and as your picture shows it clearly does :)
[0] https://www.flickr.com/groups/11947580@N00/discuss/721576207...
[1] https://en.wikipedia.org/wiki/Reciprocity_(photography)#Reci...
I feel that this is worth expanding on.
Because a raw file isn't actually image data and instead just straight readouts from the sensor, it doesn't make sense to show the histogram of raw data; the data has to be manipulated into a jpeg first. The histogram in the camera will show the lightness values for what the in-camera image processing computer does to convert the raw file into a jpeg, but the in-camera image processor is usually not as advanced as a desktop image processor. It's probably running older image processing algorithms and has to worry about battery life among other things.
So generally what this means is that if your camera is telling you that parts of your image are over or under exposed and therefore clipped, it might still be possible to recover those parts of the image in a more powerful image processor. It's also worthwhile, if you've got really old raw files, to go back and reprocess them with modern software. Raw file processing has come quite a long way in the past decade!
Imagine back to when digital photography just started, when we were stuck with 10-bit RAW images (before processing them to 8 bit JPGs). That's 1024 different values per channel, not a whole lot of room to begin with. Because we're converting a linear signal to a logarithmic one, that means that highest stop in the image takes up 512 of those values.
EDIT: As rightfully pointed out in a reply, 8-bit JPGs are not in linear colour space any more - so this logic does not apply there (however, JPG compression is more harsh on shadow details, since it's lossy, and it tries to move that loss of information to the parts of the image where you're less likely to notice, like the shadows, but we're digressing)
By that logic, the low light data should have a lot more banding than the highly exposed data. Try it with level tools on a RAW file and see for yourself.
So you can imagine that when the dynamic range of (part of) the scene that you want to capture is less than the dynamic range of your sensor, you want to be on the right edge of the histogram (all else being equal - if this causes motion blur due to long exposure it's pretty useless)
Caveat: this is potentially outdated knowledge, perhaps the analog/digital converter in modern cameras actually converts the voltage of a CCD to a (pseudo)logarithmic scale for the digital output, which might partially circumvent this issue (although the sensor is still linear). And nowadays many of the better cameras have 14-bit A/D converters - the usefulness of which also largely depends on the quality of the sensor - so there's a lot more detail in the shadows than before.
The same thing holds true for a processed RAW image as well.
I guess there's also the visual "symmetry" with JPG or JPEG.
When you say processed RAW, you mean after importing to image editing software, right? Which implies you wouldn't see much of a difference in posterisation in the high and low exposure areas
If you want a good website in general for explaining photography techniques plus the physics behind it, I highly recommend Cambridge in Colour[3].
By the way, per the example of the second linked essay: if we try to capture 10 stops in 12 bit channels, the darkest stop only has 32 discrete available in the RAW file.
However, this might be somewhat less of an issue these days because most cameras have a really high resolution, and most images are resized for screen displays. So we can average out some banding (and image noise, for that matter).
Example: I have a 20 megapixel camera. A 1080p screens is just 2.1 megapixels. So even for a full-screen export, that's roughly ten "source pixels" per "output pixel". Which would mean that the output pixel, being an, could have 300 different values. The distribution won't be like that of course, but it still should help.
[0] https://en.wikipedia.org/wiki/Exposing_to_the_right
[1] https://web.archive.org/web/20150209012804/http://www.lumino...
[2] https://web.archive.org/web/20150213002736/http://www.lumino...
This is really neat. I've got a full frame camera with a sensor that has very good dynamic range and I'd learned to under-expose and then pull detail from the shadows in post processing as information can't be saved from blown highlights. This sort of flips that on its head, except the featured overexposed shot didn't have blown highlights, it just looked like it did. Instead it had a wealth of low-light data.