It's exactly the same problem as is caused by thinking of a digital audio signal as a series of steps, like http://upload.wikimedia.org/wikipedia/commons/thumb/1/15/Zer... . This too is completely wrong. A digital image DOES NOT DEFINE the space between the discrete points just as a sampled sound does not define the level in between each point.

It's why signal processing folks will insist on plotting sampled signals as lollipop plots, NOT step (aka zero-order-hold) signals. http://www.ling.upenn.edu/courses/ling525/swave2.gif

The reason why is it treats appropriately an ambiguity in a sampled signal... the fact that anything at all could happen between two samples, but all you see is the value at that point. Stated another way, any line that goes through those green circles could have been the original sampled signal. I might also add that the zero-order-hold isn't even a minimal energy or GOOD interpolation of the original signal. Typically when sampling, you band limit the input to the nyquist frequency. That way a PERFECT reconstruction can be done. Yes I know that's weird. If you band limit a signal and discretely sample it, you may reconstruct it PERFECTLY. The perfect reconstruction may be generated through the Whittaker–Shannon interpolation formula. A zero-order hold will interpolate the signal in a way that will create tons of harmonics outside the band of interest. It's wrong. It's the wrong way to think about sampling.

It's not quite the same, and here is why: In audio, the samples from an ADC are generally very close to point-samples indeed. Not quite, because ADCs can not sample the input at infinite small times. Or actually really point-like if generated algorithmically.

However, this is not true for pixels. The meaning of the pixel values in a source image is arbitrary. It can be point-like, but I would argue it mostly isn't.

Pixels in a display device /never/ are point-like. They always fill an area. That's just a physical reality.

We also can not use a simple band pass to limit the (spatial) frequency content. If you take the videolinked in the sister comment, and assume that t is a spatial direction.

A pattern like __^^^___^^^___^^, that is black-white-black-white... would show ringing when the slopes don't match pixel borders. What do we do instead? Either align it, or integrate over the pixel area. I.e., the pixel overlapping the slopes would have grey values.

(Edit: realized parent post didn't link the video)

With pixel data, we have low enough resolutions that our thinking is biased towards the display device the majority of the time; we can author "pixel art" that expects the results to look square, or very close to square. And when we resize that art, we don't interpolate the color values(unless we want it to look like shit), we use a simple drop-sample or nearest-neighbor. But when we resize photographic images, suddenly we have a reason to do other things. Take for example this Stack Overflow answer on Lanczos resampling. [0] It describes exactly what you say we can't and don't use, but is actually used every day in Photoshop: a filter kernel that produces ringing artifacts on black-white patterns. As I allude to with the pixel art, it depends on the source content. When we get an image off of a camera, it's sampled in a way that assumes the pixels are points as impulses, and thus should be processed "as if" they are convolutions of the sinc function, even though the data itself is "just" points. Digital photographs always look blurry or noisy when examined pixel-by-pixel, and only form coherency to our eyes when viewed more distantly. For similar reasons, pixel artists can't downsample and process arbitrary photos and pass them off as legitimate pixel art, because the original content isn't designed towards the display of square or near-square pixels.

[0] http://gis.stackexchange.com/questions/10931/what-is-lanczos...

Also watch this extremely informative video from xiph.org: http://xiph.org/video/vid2.shtml

Thanks, excellent video to understand what people are talking about in this thread.

My favorite video that drives this point home: http://xiph.org/video/vid2.shtml

Using actual signal generators and oscilloscopes, this guy proves by experiment everything you said. Goes into differences between sampling rate and quantization depth as well. Well worth the watch.

Programmers would implicitly use a box filter when sampling from an underlying process (like 3d rendering, image scaling, scanning) into screen coordinates.

It's one thing when writing a compositing window manager or blitting sprites to the screen; it's another thing entirely when doing any kind of image processing to make that assumption.

It's similar to assuming that RGB (0-255) is perceptually linear and consistent among monitors (not understanding monitor profiles and color spaces)

There is no issue when the image resolution and display resolution are the same. But you shouldn't be tying these together: the image is some approximation of a continuous image, and the displayed version is the best visualization your monitor can provide of this approximation. When the display resolution is higher, thinking in terms of "little squares" causes you to interpolate the image in a suboptimal way.

Indeed. As far as I could tell, the article was an exercise in "TMTOWTDI" - sometimes squares, sometimes subdivided squares (trinitron), sometimes little circles with gaps or overlaps, sometimes linear interpolation of a sample.

OK, it's an abstraction, but the article didn't seem to cover the strength of weakness of various implementations.

I think there is a slight overload of terminology here.

The samples in the source image, and the tiny light sources that vary in intensity and are driven by a voltage after the information in the bits in the source image has traveled through the display pipeline are two different things.

The author discusses the best way to think about the source image. It will get really awkward if we want to discuss imaging algorithms and have to define "my pixel is a square... if it's a bunch of weird shapes the transform is this... etc." It's easiest from this point of view to think of the source image as a lattice of point samples and figure out what the sampling function (our physical pixels) do to the image after we have figured out the math on the point sample source.

And sometimes, it was an asterisk in a 60 x 80 grid (with a messed up aspect ratio) on a piece of printer paper, back in the stone age :-)