You are right, the final output dynamic range is limited by your output device, you can't circumvent that.
The 1,000,000:1 is the contrast ratio you might see on a sunny day, and represents the state of the world that your eyes are seeing.
Your eyes aren't capable of 1,000,000:1 contrast either, at least not globally across your field of vision, so we can play tricks in rendering to simulate what you would perceive with your limited eyeballs on a 1M:1 contrast ratio day, and this is what HDR rendering does.
Internally, the rendering code handles luminance well in excess of its ability to display on the final output device. All the game rendering happens internally in HDR, preserving this high level of contrast. The final steps of producing a displayable image are doing something to convert non-displayable "real" contrast ratios into post-processed simulation to fool you into perceiving high contrast scenes. This last part is subjective and artistic, since you are trying to figure out a way to show that high dynamic range scene. Common tricks played here are the light bloom from bright areas, the artificial amplification of darks areas, etc.
All these in-between rendering steps compose the result of multiple rendering passes into an image. Each pass which is blended with the results of a previous pass is subject to quantization error. The more passes you do (and modern games do lots), the more this quantization error accumulates, and leads to pretty ugly banding artifacts. You want to do your internal processing in the highest color resolution that you can, so that you only have to worry about the error of quantizing the 1M:1 scene into a 256:1 output device. You don't need the full 20 bits, and common practice is to use 16-bit S10E5 floats since hardware handles this quickly.
HDR televisions, while nicer than nothing, don't help you all that much. Displays have generally been 8-bit per channel, giving you 256 values distributed along a gamma curve which maps to a device with perhaps 3,000:1 contrast capability. HDR tv's give you 1024 values along a gamma curve that spans a higher range of contrast, but not all that much higher.
In the real world, the total contrast ratio between the brightest highlights
and darkest shadows during a sunny day is on the order of 1,000,000:1.
We would need 20 bits of just luminance to represent those illumination rangesIn practice modeling vision precisely gets really complicated because there are several types of spatial and temporal adaptation and contrast effects that the human visual system can undergo / be affected by, and something like a video game definitely doesn’t have the computation available to handle it perfectly (not to mention the programmers don’t want to implement anything that complex and finicky). There is also a complication that display black is never especially close to zero light, and depends a lot on the light hitting the display. Etc.
So for practical purposes you just have to come up with some kind of “good enough” heuristic.
The assumptions seems to be a linear illumination scale with the darkest value at 1 (with true absence of illumination at zero); 1,000,000:1 needs about 20 bits for that.
So for practical purposes you just have to come up with some kind of “good enough” heuristic
Ok I agree ; that's what I'm saying to chrisseatonThe article says that the highest light level in a scene is 1,000,000 times brighter than the lowest. It does not say, or even imply, that you can see the difference between those levels of light to a one in a million, which is what 20 bits needs.
Can you see what I mean? The factor between the lowest and the highest is entirely separate from the question of how many levels between you can see, which is what directs how many bits of resolution you need.
One thing to add: Wikipedia sayeth "the eye senses brightness approximately logarithmically over a moderate range". If we go with that, then you presumably want to encode brightness logarithmically, and the number of bits you have available will determine the ratio between your adjacent quantized levels. In that case I believe the ratio between adjacent levels would be exp(ln(max_range_ratio)/(2^bits)).
I explained it in another comment but basically the "20 bit" value is based on an idealized digital image sensor rather than a game rendering pipeline (which operates in floating point). It has admittedly proven somewhat confusing.
All we know is that the range is N to 1000000N, but we do not know in and of itself that N is the smallest delta perceivable or possible.
I don't know anything about games but at least in cameras the relation you want between the bit depth and dynamic range is determined by what you want to measure rather than a formula. An eye tracker I have is a 10-bit IR camera because a normal 8 bits are insufficient to both note that IR LED reflections are way brighter than everything else, while also having sufficient detail in the lower values that the edges of the pupil are easy to detect. At least, without having the scale be logarithmic or discontinuous or otherwise compressed.
The real benefit of HDR is in the "more values" part since, as the author notes, our displays have a very limited dynamic range anyway.
Hell, I like to think I'm somewhat knowledgeable about these things and I read straight past that line thinking "that's a million, that's about twenty bits, okay".
But you are absolutely right, now that you pointed it out it's obvious.
And indeed you can get the same range using only one bit (per channel, that is) and if you had high enough (very high) resolution and proper dithering, you'd totally get away with it, too. In that case, the tonemapping goes just before the dithering.
I didn't interpret the articles premise that if you had 20 bits, you could 100% reproduce the light in a scene.
You only need 1 bit if you declare an encoding scheme where 1 == 1 million. You need 20 bits to define 1 million DIFFERENT luminances. Their ratio is subject to an arbitrary scaling.
> To express a ratio of 1000000:1, you need at least 20 bits.
You mean to express all the integer factors in the range 1000000:1, you need at least 20 bits. With 20 bits you can represent 1 times brighter, 2 times brighter ... 1000000 times brighter.
But there's nothing special about those coefficients. 20 bits does not allow you to represent 1.5 times brighter. That's still in the range 1000000:1 but 20 bits isn't enough to represent it.
If we're happy to skip 1.5 times brighter, why can't we skip all the even integer times brighter values, and use 19 bits?
Things get wonky if you don't have a linear scale with a true zero; in such a scale the low end of your N:1 contrast ratio (in the smallest representation) has a value of 1, and the high end has a value of N.
Also, if that type of "wonky" throws off your rendering pipeline, you're bound to get something else wrong.
Such as ever having a linear scale with a small number of bits in your pipeline. The linear scaled brightness stays afloat all the way through (cause floats have this handy feature of being transparently sorta-logarithmic in the way they use their bits, even 16-bit floats beat 20-bit ints for that purpose), only at the very end you apply the tonemap+gamma function(s), then dither, then truncate to fixed (8) bit integer.
edit: And aren’t output color spaces already highly nonlinear due to gamma correction?
2^19 = 524,288
2^20 = 1,048,576
Another argument for this formula might come from Shannon information entropy but I'm not 100% sure
EDIT: I might as well link to the author's clarification. I missed it in the noise: https://news.ycombinator.com/item?id=15538738
It is a little odd. I don't think a linear, fixed-precision scale for luminance is used much, though I could be quite mistaken. I'm not very familiar with HDR. Still, I might as well explain a bit further...
Light accumulation is linear. If you have two lights, the amount of light reflecting off the surface is twice as much. Twice as many photons.
Human perception, however, is logorithmic. Something that is twice as bright in terms of photons emitted actually only looks a little brighter to the human eye. To get the same perceived brightness increment again, you need to double the photon count again.
With that in mind, what's the best numeric representation to use? Well, you want a simple way to add up the total amount of light from all sources. The amount of light may vary by orders of magnitude, but it's manageable because your need for precision scales with the magnitude you're dealing with. Basically, you want floating point.
In practice, a float for each color channel means 3 floats (96 bits) per pixel, which is a lot. So, while that might be used during calculations, it's probably stored compressed. For example, RGBE is a 32 bit per pixel format. You use 8 bits per color channel and multiply them all by 2^(E+128), where E is an 8 bit signed integer. It's kind of inaccurate for the dim channels if luminance varies significantly between colors, but the brightest one is probably going to overwhelm the other channels anyways.
Imagine a gradient that smoothly transitions from black to white. 8 bit color means only 256 shades of luminance, which would quickly show banding artifacts.
In fact, the entire reason we gamma encode images is because 256 is too small for even a low dynamic range monitor. Gamma encoding is a clever technique to give us more bits to represent shadows, which humans are more sensitive to than highlights.
You can't take a real range and give a number of bits needed to represent it without also stating what resolution you need. And I think it's unlikely that the resolution that a human can perceive is coincidentally the same as the factor of one extreme to the other.
Much of the post comes from a general assumption that the goal of computer graphics is primarily to replicate how a camera sees the world around it. Thus I think it's easiest to start from the idea of real world light entering a digital image sensor. Light in this setting is not continuous! Each subpixel in an image sensor acts as a photon counter. One photon hits the sensor, the count ticks up by one. There's no question of being able to perceive the values between 1 and 2 because they don't even exist. Either the sensor counted one photon or two. If you were going to literally create a digital camera that can process the entire world, you need 20 bits to count up to a million photons without losing any along the way. So if you were to build the hypothetical rendering pipeline that works on "real world" data about the scene, that 20 bit value would be the input.
As a practical matter, nearly all modern games store lighting levels internally in floating point, in arbitrary units chosen by the developers. Lighting pipelines are not integer based, but they're linear and not perceptual. The conversion to perceptual 8 bit (gamma curve) happens as part of the tone map stage. Doing things in floating point physical units is a better idea than the photon counter anyway, but the line you're confused about was really written with idealized cameras in mind.
(Technically an image sensor is an analog device and the voltage increases with each photon detection by an increment that is subject to noise of all sorts and pre-amplification before hitting ADC. Don't jump me on the photon counter thing.)
But it's rather important, and not even for those reasons. This is not meant to jump on you! (and I really loved your article, like you said it's not a crucial point at all)
The 1:1000000 or 1:2^20 contrast ratio only corresponds to exactly 20 bits if the 1 on the low end of this ratio corresponds to exactly one discrete unit of light (photon). If it's off by a factor of 0.5, 1.618 or whatever, that's what the whole argument is about.
First, the sensor counts not photons but a value relating to photons per second (because exposure time). If the 1 on the low end of the range corresponds to some exact minimum number of photons, it's going to be "one discrete unit of light per <exposure time>". Making the whole thing analog from the start.
Second, those sensors most probably aren't able to count individual photons any way[0]. The human eye, after about 30 minutes to get optimally adjusted to darkness, can sort of perceive individual photons, or small bunches of maybe 2 or 3, kind of. Those barely-perceptible specks of light in the utter darkness aren't the sort of resolution issues we're worrying about in the dark end of these types of scenes. And, as soon as you make a light source that can be described as "emitting single photons" in a certain context, you get uncertainty effects and all that quantum jazz (show me a photon/path tracer renderer that gets the slit experiment correct[1] and you can have your integer photon counters :) ).
So the sensor output values can (and should), for all intents and purposes, be assumed to be an analogue value.
The amount of bits you represent it with just puts an upper bound on your signal-to-noise ratio (as per Shannon entropy). But since we're dealing with 2D images, the distribution of this noise over the spatial resolution (either as a result of sensor noise at the input or explicit noise shaping dithering at the end of the pipeline) also comes into play when considering the quality of the output.
If the signal-to-noise ratio of a sensor output happens to allow for 20 bits of precision, for a sensor that happens to have a 1:2^20 brightness range, that's coincidental. Sure it correlates because higher-end sensors tend to perform better in both range and SNR. But I don't believe that the 1 on the very low end of a discrete range represents precisely one photon per <power-of-two times minimum exposure time>.
[0] correct me if I'm wrong about this btw. There might be specialized scientific equipment that can, but I doubt even high-end cameras bother to go to the accuracy of single photons. But, I mostly know about digital signal processing, not about state of the art of camera hardware. Yet even if they are able to detect individual photons, that's going to be a probabilistic and per-unit-of-time measurement, so the rest of my argument holds.
[1] these probably exist, but aren't used for games or photorealistic rendering purposes
To put it in mathematical terms, there is no reason to use the log base 2, but the formulae is definitely a logarithm. So, again, 20 bits is a minimum bit count conveniently used to compare with others situation (e.g. "if you are shooting not in direct sunlight, you will only need 12 bit minimum, not 20 bit minimum so this method of shooting is OK") and make decision about your color pipeline
You will need 20 bits if a human can distinguish 1,000,000 levels between those two extremes
It's not about what we can perceive but how we store and manipulate amount of lights in our pipelinesSo, for display purposes, 8 bits (with a non-linear mapping) has been sufficient for the brightness range of TV and monitors until recently. Now we have displays going High Dynamic Range. If you tried to display a 10x wider range of brightness using the old 8-bit format, you'd start to see a lot of ugly banding (posterization) much more than you have seen before because each of those 255 flat steps would be associated with a much larger section of the brightness curve. You'll need at least a couple more bits to keep your brighter&darker image looking as smooth as the mid-range-only images did in the past.
For storage purposes, the 16-bit per channel floating point OpenEXR format has been used by the film industry as a intermediate format between image processing tools for quite some time now. I think that demonstrates that 16 bit floats are sufficient for storage.
For computational purposes, 16 bit floats can build up a lot of numerical error very quickly. It's OK for small bit of math (moderate real-time shaders), but when things get complicated, it's necessary to go full 32-bit float.
Tell that to D/A converters everywhere.
> If you tried to display a 10x wider range of brightness using the old 8-bit format, you'd start to see a lot of ugly banding (posterization) much more than you have seen before because--
Because you didn't dither, is why.
The only thing you lose with less bits is SNR, and thanks to the wonders of noise-shaping you can pretty much decide where to put this noise. And I'm just going to make a very broad statement that ever since we managed to climb out of 320x200 resolutions, spatial resolution is where you should put it, not in banding. Maybe even one or two bits in the time resolution if you want to get fancy about it.
Basic dithering can be as easy as just adding a uniform 0..1 random number just before truncation. Triangle noise (difference of two uniforms) can give a bit better visual accuracy for graphics, but has one or two snags that take attention to get right.
What kind of graphics calculation would build up enough error to matter? You don't exactly need to do A + 10000 - 10000 in a rendering pipeline.
If you're summing huge amounts of numbers in a raytracer it's appropriate. Where else?
And .. even a raytracer doesn't sum that many numbers, maybe a few thousand[0]? So even then it's still a very reasonable question to ask.
[0] assuming a Monte Carlo pathtracer type of thing with 5-10 bounces or so. BTW, do you think you'd ever need more than five? Unless you're rendering close-to-parallel reflective surfaces, but the appearance of those is often as confusing in real life as it is when rendered correctly.
Only if you don't dither. Which is like free bits of precision given today's monitors resolutions.
I don't disagree with the rest of your point (although I do believe that just because there's a 20-bit dynamic range one should represent it by 20 bits of precision that's an arbitrary choice) (not even the best choice, IMO).
Just want to emphasize that, a statement being made all over this thread, that "less bits = you gonna get banding", just means you're doing it wrong, thinking about it wrong, and might therefore even end up with banding if you had enough bits but wasted them at some point in the pipeline.
Imagine you have a lightbulb (producing let's say max 1000 units of light/brightness [which are two subtly different things]) connected to a dimmer that has for example 8 discrete levels. What would you say the dynamic range of this setup would be? Obviously measuring the contrast ratio from when the dimmer is in the zero setting (=light off) to max setting would result infinite contrast ratio/dynamic range. That is not very useful though, so the next best thing is to measure brightness difference from the lowest on setting ("1") to max, which gives you actually a reasonable number, in this scenario 125:1000 or 1:8. I hope this example was illuminating.
So I guess the idea is that bit depth fundamentally determines what is the minimum brightness representable without going to zero (where be dragons), and as such determines the max usable dynamic range.
You could have a dimmer that goes from one level of lightness, up to another, with 8 steps in between. And you could have another dimmer that goes between exactly the same two levels of lightness. The same range. but with 1,000,000 steps in between. The range of the two dimmer switches is the same. They both go from the same minimum to the same maximum. Just the number of steps in between is different. The number of bits represents the number of steps.
Why is 1,000,000 steps the required number? 1,000,000 is just the number of integral factors in the range in base 10. But why is an integral factor more important than a fractional one? They could both be perceptible to a human. The range is the same in both cases, and the given explanation was given that the range was so high.
The thing is that you can't set the minimum value arbitrarily, because the scale is anchored to zero. So "1" is always one step above zero, and the step size is a function of bit depth.
So going with our example here, a dimmer with 1000000 steps would not have the same range as dimmer with 8 steps, because the the minimum brightness of one is 1/1000000th of max, and 1/8th of the max for the other (assuming linear scale for simplicity's sake, the principle applies to nonlinear scales too)
(Pure black might be zero, but that's useless for computing a ratio, so it doesn't give you a resolution.)
Then you need 20 bits, because log_2 1e6 rounds up to 20 (equivalently, 2^20 <= 1,000,000 < 2^21).
If you used only 8 bits, the ratio between highest and lowest (non-zero) luminance would be only 256:1, or 1,000,000:3,906.
20 bits lets you represent all the values which are integral in base 10.... but so what? Why is the fact that a value is integral in a particular base make it the minimum resolution difference that you need to represent? Why do you need to quantise at that level? There is a luminance of 1.5 times n, isn't there? But 20 bits doesn't let me represent that. 21 bits would. See what I'm saying?
The fact is that if you want to have good highlights (beautiful clouds, candles, sun glares etc..) in your image when shooting with a camera, you have to "expose by the right" i.e. if the scene is more dynamic => reduce the amount of light incoming into the camera => your shadows will be crushed down so less precise => you need better color resolution if you want to "save" your shadows OR you need to use another specific colorspace (S log shaped)
It's a problem of presenting that contrast ratio on a screen
that is not capable of a contrast ratio of 1,000,000:1
If you have such a high contrast ratio, it also means you have discretized amount of light in such amount or more. Talking about the number of bits minimum needed indeed makes the assumption you are storing and manipulating light in a linear way (hypothesis not true for most 8 bpc rgb values).The short answer is: You're right that it's wrong to say that you need 20 bits to cover the range. Because clearly you can cover the range with 10 bits or 100 bits by subdividing differently. More bits is better, but there's no rule of required multiplication of bits for fabricated metrics like integral brightness multiples. 20 bits is just required to cover the range with integers without skipping any, which is an entirely arbitrary decision.
The long answer is:
The reason you want more bits to represent the larger range is exactly as you say because one doesn't actually care about _range_ but rather about the utilization of increments within the range. Now I'm going to describe something that you clearly already intuit for completeness...
Say, for example, that you have only 3 steps (let's call them 1, 2, and 3) in your range instead of a million, but you want to represent the full range of darkest dark to brightest bright with those 3 steps (for simplicity you could also be thinking in black and white instead of color). Now, you take a picture of a normal indoor environment with some gentle indirect light, and maybe all of external values would be, on the million range, between 350k and 650k. If you linearly divide your range onto 3 steps, when you look at your resulting image you will have a flat solid block of 2s with no 1s or 3s at all. And yet your original setting had an absolutely massive 300k spread. But it didn't have significant brights or significant darks, because you were indoors with some indirect light and there was no fireball sun or deep cavern of despair in the image, so you didn't measure any differentiation in values.
So you decide to switch it up and go inside a dark cathedral with some wonderful stained glass windows (everyone loves cathedrals with stained glass when looking at HDR). This time you do the same thing, but instead of measuring all 2s, you measure all 1s with a chunk of all 3s for the sunlit windows, but no 2s at all.
So then you say, "What I should do is take some of those 3s and some of those 1s and pretend that they're actually 2s when differentiation is too high, and take some of those 2s and split them into 1s and 3s when differentiation is too low! That way I'll see more details, because there will be more efficient utilization of the segments of my available output spectrum!"
But _then_ you say, "Oh crap! I can't do that! I didn't actually capture the differences! I captured what I captured, and what I captured was the aforementioned terrible undifferentiated data!"
So the reason you want to be able to represent values on a _densely_ _segmented_ range is just so that you can differentiate things that are only slightly different in absolute overall term (think photon counts, I guess) because aesthetically those slight differences matter. It means you can do things like know when some of your 2s should become 1s and 3s, because the 2s aren't actually 2s anymore; they're a bunch of tiny fractional variations between 1.5 and 2.5, and that's data you can work with.
You can perform tone mapping with any number of bits. If record 10 increments instead of only 3, you can obviously convert those 10 down to 3 in a way that gives a better image than just starting with 3. And the more bits you have, the more linearly you can record the environment without losing the differentiability between things that are in absolute terms on a large scale very similar. And then you get to do fun things like add a light bloom effect around something that is _very_ bright compared to its surroundings without losing details within the very bright thing and without losing details within the very dark surroundings.
Having more representational bits just means you can tone map scenes that have really bright parts, really dark parts, and really medium parts without losing any of the details inside those parts.
I don't understand this statement. Why do you need a certain number of bits to represent a large range? You can encode a range using any number of bits you want, but if you use fewer bits the resolution within may be lower.
Representing a large luminance range isn't a problem of resolution though is it? It's a problem of presenting that contrast ratio on a screen that is not capable of a contrast ratio of 1,000,000:1. If I represented the same luminance range with a thousand bits it doesn't do anything to solve the core problem, does it? So why does the number of bits matter?
Not a graphics programmer, so genuinely asking, not saying the article is wrong!