News for hackers. Stuff that matters.
Re:A Pixel Is Not a Little Square (1995) [pdf] by kragen on Tuesday April 27, 2021 @02:58AM (#26952889)
No, there are two fundamental misunderstandings in your comment.

You posit an opposition between "[arrays of] point samples with a convolution" and "[arrays of] exceedingly crisp squares". This is incorrect. If the kernel you convolve your array of point samples with is an exceedingly crisp square, the result is an array of exceedingly crisp squares. Convolution with a square is just a special case. Smith explains this, perhaps not very clearly, in Fig. 3, p. 5.

Conceptualizing this as a convolution allows you not only to handle the special case you're saying is "a reasonable first-order approximation," but also to analyze what effects that display or sensor convolution is having on your image, and possibly to correct them. Moreover, it can handle some other kinds of image degradation as well that go beyond that first-order approximation, such as bokeh, diffraction, defocus, and nonuniformity within pixels, as long as it's the same for every pixel.

That's how you find out what those "further adjustments to avoid antialiasing artifacts, desired sharpness, and so forth" are.

And it's how you can go about analyzing what happens to the image when you, for example, resample to a different sampling grid, as Smith explains on p. 4. If you analyze different linear resampling algorithms using the theory of convolutions, you can predict precisely how good the results will be; if you do not use this theory, you will be puzzled and will not understand your results at all. Believe me, I know; I wandered in that desert for years. But you don't have to. Learn from my mistakes.

Second, audio samples are frequently handled exactly the same way as pixels in this sense, using a "zero-order hold". If you oversample by taking 64 consecutive samples from your ADC and summing them, or set the PWM duty cycle on your Arduino to a new value every 125 μs and leave it there until the next sample, or shift a new sample into the shift register connected to your R-2R DAC and leave it there until the next sample, you're doing exactly the same thing as trying to draw a little square for each pixel—just in one temporal dimension instead of two spatial dimensions. A zero-order hold is a perfectly reasonable thing to do with audio samples for both input and output, although it's not the only option, and it can produce audible artifacts. The math for analyzing this—and correcting it, if that's what you want—is exactly the same as the analogous math for displays or camera sensors made out of little squares.

Far from being "far more misleading", the analogy between pixels and audio samples is deeply insightful and unlocks a wealth of mathematical tools that can be applied in both domains, as well as, for example, to linear control theory and GPS position estimation. The results this analogy leads you to are profound truths, not falsehoods as you seem to think. And some of them are very beautiful.

None of this has changed in 25 years (except the detail that now we have LCDs with little rectangles on them to complement the little squares of dye-sublimation printers that Smith mentions) and little of it has changed in 50 years.

Re:A Pixel Is Not a Little Square (1995) [pdf] by crazygringo on Tuesday April 27, 2021 @02:10PM (#26959387)
I think we're agreeing in all our facts but simply disagreeing on how to most usefully interpret them.

You call convolution with a square a special case, I'm calling it a reasonable first-order approximation for modern displays.

And using a zero-order hold on audio samples isn't something most people frequently need to conceptualize. In fact, it's responsible for many people's great misconception that sampling rates above ~40 kHz result in noticeably higher-fidelity audio, because they think the DAC's output will be smoother. It's clear to me that you know better, but most people don't.

I'm in full agreement with you on all the math involved. It's simply that going around saying "pixels are point samples not squares" seems like it's going to be more misleading than helpful to 99% of people.

If someone wants to understand the basics of antialiasing, or reducing image resolution with bilinear interpolation, the conceptualization of square pixels works perfectly. More advanced antialiasing techniques are ultimately just subtle adjustments to that in terms of measurable output differences, which is why I stand by calling square pixels a reasonable "first-order approximation".

Re:A Pixel Is Not a Little Square (1995) [pdf] by kragen on Tuesday April 27, 2021 @11:07PM (#26965064)
80+% of people would respond to a statement like "Pixels are little squares" with something like "我不会说英语", "أنا لا أتحدث الإنجليزية. ", or "मुझे अंग्रेजी नहीं आती। ". Of the remaining 20%, 99% would have no idea what a "pixel" was supposed to be, perhaps thinking it's a Google product. But the fact that the garden is mostly bare brown earth today does not imply that sowing the seeds of knowledge is futile. (I have certainly explained the Dirac-comb sampled image model in Spanish on more than one occasion, and if I have not done so in Hindi or Mandarin, it is only because of the untilled wastelands of my own withered, stunted mind.)

> If someone wants to understand the basics of antialiasing, or reducing image resolution with bilinear interpolation, the conceptualization of square pixels works perfectly

As Smith explains in the paper and I explained in the comment above, it's easy to get extremely conspicuous artifacts if you design a downsampling algorithm with the little-squares model, although if you're increasing image resolution you can frequently get away with it.

If you don't understand the formalization of the imaging problem that Smith is laying out—and it's clear that you don't—you aren't in a position to opine on how useful that formalization is. Applying that formalization will show you whether it's useful or not, and you can't try that until you understand it. My opinion from my experience is that I wasted a lot of time on the kind of seat-of-the-pants approach you're advocating, and now that I understand the rudiments of sampling theory, many problems that were intractable before have become easy. Learn from my mistakes instead of urging other people to repeat them.

> You call convolution with a square a special case, I'm calling it a reasonable first-order approximation for modern displays.

You are positing another opposition that simply doesn't exist. Perhaps you don't know what the phrase "a special case" means? A mathematical operation can simultaneously be a special case of a more general mathematical operation and useful for something, for example approximating a given display. And that is the case here. Being "a special case" of a more general case just means that everything that is true of the more general case is also true of the special case. It's not a way to deprecate the special case. It's a way to pointing you at a deeper understanding of it.

(Again, though, it's a zero-order approximation. First-order approximations are bilinear sampling and similar things. I think you have a lot of work to do before you can be in full agreement with me on all the math involved.)