News for hackers. Stuff that matters.
Re:Pixel is a unit of length and area by tomhow on Wednesday April 23, 2025 @08:31AM (#43771327)
See also:

A Pixel Is Not a Little Square (1995) [pdf] – http://alvyray.com/Memos/CG/Microsoft/6_pixel.pdf

Re:Pixel is a unit of length and area by glitchc on Wednesday April 23, 2025 @09:33AM (#43771989)
This is written from a rather narrow perspective (of signal processing) and is clearly wrong in other contexts. For an image sensor designer, gate length, photosensitive area and pixel pitch are all real-valued measurements. That pixels are laid out on a grid simply reflects the ease of constructing electronic circuits this way. Alternative arrangements are possible, Sigma's depth pixels being one commercial example.
Re:Pixel is a unit of length and area by grandempire on Wednesday April 23, 2025 @10:05AM (#43772381)
Ok, but that’s not what an digital image is. Images are designed to be invariant across camera capture and display hardware. The panel driver should interpret the dsp representation into an appropriate electronic pixel output.
Re:Pixel is a unit of length and area by fluoridation on Wednesday April 23, 2025 @10:32AM (#43772688)
Well, "what a digital image is" is a sequence of numbers. There's no single correct way to interpret the numbers, it depends on what you want to accomplish. If your digital image is a representation of, say, the dead components in an array of sensors, the signal processing theoretic interpretation of samples may not be useful as far as figuring out which sensors you should replace.
Re:Pixel is a unit of length and area by grandempire on Wednesday April 23, 2025 @10:35AM (#43772716)
> There's no single correct way to interpret the numbers

They are just bits in a computer. But there is a correct way of to interpret them in a particular context. For example 32 bits can be meaningless - or it can have an interpretation as a twos complement integer which is well defined.

If you are looking to understand how an operating system will display images, or how graphics drivers work, or how photoshop will edit them, or what digital cameras produce, then it’s the point sample definition.

Re:Pixel is a unit of length and area by Dylan16807 on Wednesday April 23, 2025 @03:24PM (#43775760)
Cameras don't take point samples. That's an approximation, just as inaccurate as a rectangle approximation.

And for pixel art, the intent is usually far from points on a smooth color territory.

Multiple interpretations matter within different contexts inside the computer context.

Re:Pixel is a unit of length and area by grandempire on Wednesday April 23, 2025 @04:10PM (#43776163)
> Cameras don't take point samples. That's an approximation

They use a physical process to attempt to determine light at a single point. That’s their model they try to approximate.

> And for pixel art, the intent is usually far from points on a smooth color territory.

And notice that to display pixel art you need to tell it to interpret the image data differently.

Also it has a vastly different appearance on a CRT where it was designed which is less like a rectangle.

Re:Pixel is a unit of length and area by Dylan16807 on Wednesday April 23, 2025 @07:03PM (#43777620)
> They use a physical process to attempt to determine light at a single point. That’s their model they try to approximate.

According to who?

A naked camera sensor with lens sure doesn't do that, it collects squares of light, usually in a mosaic of different colors. Any point approximation would have to be in software.

Re:Pixel is a unit of length and area by grandempire on Wednesday April 23, 2025 @08:30PM (#43778168)
Yep, cameras filter the input from sensors.

> Any point approximation would have to be in software.

Circuits can process signals too.

Re:Pixel is a unit of length and area by Dylan16807 on Wednesday April 23, 2025 @08:51PM (#43778290)
They usually do, but their software decisions are not gospel. They don't change the nature of the underlying sensor, which grabs areas that are pretty square.
Re:Pixel is a unit of length and area by grandempire on Wednesday April 23, 2025 @09:09PM (#43778380)
> which grabs areas

And outputs what? Just because the input is an area does not mean the output is an area.

What it if it outputs the peak of the distribution across the area?

> that are pretty square.

If we look at a camera sensor and do not see a uniform grid of packed area elements would that convince you?

I notice you haven’t shared any criticism of the point model - widely understood by the field.

Re:Pixel is a unit of length and area by Dylan16807 on Wednesday April 23, 2025 @10:29PM (#43778750)
> And outputs what? Just because the input is an area does not mean the output is an area.

> What it if it outputs the peak of the distribution across the area?

It outputs a voltage proportional to the (filtered) photon count across the entire area.

> If we look at a camera sensor and do not see a uniform grid of packed area elements would that convince you?

Non-uniformity won't convince me points are a better fit, but if the median camera doesn't use a grid I'll be interested in what you have to show.

> I notice you haven’t shared any criticism of the point model - widely understood by the field.

This whole comment line is a criticism of the input being modeled as points, and my criticism of the output is implied by my pixel art comment above (because point-like upscaling causes a giant blur) and also exists in other comments like this one: https://news.ycombinator.com/item?id=43777957

Re:Pixel is a unit of length and area by grandempire on Wednesday April 23, 2025 @10:55PM (#43778877)
> It outputs a voltage proportional to the (filtered) photon count across the entire area.

This is not true. And it’s even debunked in the original article.

Re:Pixel is a unit of length and area by Dylan16807 on Wednesday April 23, 2025 @11:13PM (#43778948)
> And it’s even debunked in the original article.

No, it's not. That article does not mention digital cameras anywhere. It briefly says that scanners give a gaussian, and I don't want to do enough research to see how accurate that is, but that's the only input device that gets detailed.

It also gives the impression that computer rendering uses boxes, when usually it's the opposite and rendering uses points.

Re:Pixel is a unit of length and area by glitchc on Wednesday April 23, 2025 @10:48AM (#43772853)
Yeah but the article is about a pixel, which has different meanings. Making blanket statements is not helpful in resolving definitions.

Truth is, a pixel is both a sample and a transducer. And in transduction, a pixel is both an integrator and an emitter.

Re:Pixel is a unit of length and area by grandempire on Wednesday April 23, 2025 @10:55AM (#43772926)
I’ll quote my other comment:

> If you are looking to understand how your operating system will display images, or how your graphics drivers work, or how photoshop will edit them, or what digital cameras aim to produce, then it’s the point sample definition.

Re:Pixel is a unit of length and area by tobr on Wednesday April 23, 2025 @11:37AM (#43773367)
Sometimes yes. Sometimes no! There are certainly situations where a pixel will be scaled, displayed, edited or otherwise treated as a little square.
Re:Pixel is a unit of length and area by seba_dos1 on Wednesday April 23, 2025 @01:50PM (#43774746)
A little rectangle even.
Re:Pixel is a unit of length and area by grandempire on Wednesday April 23, 2025 @02:44PM (#43775349)
What is a little rectangle? Physical display pixels are separate RGB leds with a non-rectangular shape.
Re:Pixel is a unit of length and area by afandian on Wednesday April 23, 2025 @03:35PM (#43775858)
BBC Micro in graphics mode 2 the pixel is a 2:1 rectangle. Other geometries for different modes. It's going back a bit, but very much living memory!

https://beebwiki.mdfs.net/MODE_2

Re:Pixel is a unit of length and area by Dylan16807 on Wednesday April 23, 2025 @03:16PM (#43775691)
They're rectangles on my monitor.
Re:Pixel is a unit of length and area by grandempire on Wednesday April 23, 2025 @04:08PM (#43776149)
Do the 3 separate sub pixels look like a photoshop pixel?
Re:Pixel is a unit of length and area by Dylan16807 on Wednesday April 23, 2025 @07:00PM (#43777588)
I'm not sure what you mean by Photoshop pixel.

They also look more like a square when I back away. And the mismatch of the square model doesn't mean the point model is good.

Re:Pixel is a unit of length and area by grandempire on Wednesday April 23, 2025 @08:59PM (#43778328)
> And the mismatch of the square model

So your intuition for why squares makes sense is wrong, but you’re still holding on to it.

> doesn't mean the point model is good.

What does show it’s a good model is all the theory of image processing and the implementation of this theory in camera display systems.

You’re welcome to propose an alternative theory, and if that is consistent, try to get manufacturers to adopt it.

Re:Pixel is a unit of length and area by Dylan16807 on Wednesday April 23, 2025 @10:40PM (#43778802)
> So your intuition for why squares makes sense is wrong, but you’re still holding on to it.

I said subpixels are rectangles. Because they are.

If the point model was all you need, then objects small enough to slip between points would be invisible. Which is not the case.

In particular a shot of the night sky would look pure black.

So if being wrong means we should abandon the model, then we can't use squares or points.

Re:Pixel is a unit of length and area by grandempire on Wednesday April 23, 2025 @11:18PM (#43778966)
> I said subpixels are rectangles. Because they are

https://upload.wikimedia.org/wikipedia/commons/4/4d/Pixel_ge...

I look forward to your paper about a superior digital image representation.

Re:Pixel is a unit of length and area by Dylan16807 on Wednesday April 23, 2025 @11:29PM (#43779012)
So is this a cheap gotcha because I only said "my monitor" and "most screens" the first couple times and didn't repeat it a third time? It's the one labeled just "LCD".

Or are you arguing that the slightly rounded corners on the rectangles make a significant difference in how the filtering math works out? It doesn't. On a scale between "gaussian" and "perfect rectangles", the filtering for this shape is 95% toward the latter.

Re:Pixel is a unit of length and area by cma on Wednesday April 23, 2025 @03:30PM (#43775813)
If black and white or some other cases, but there are typically subpixels that can make things like sparse enough text stroke transitions get 3X horizontal resolution.
Re:Pixel is a unit of length and area by Dylan16807 on Wednesday April 23, 2025 @03:32PM (#43775828)
The subpixels are rectangles on most screens.
Re:Pixel is a unit of length and area by grandempire on Wednesday April 23, 2025 @04:06PM (#43776131)
They tend to have rounded or cut corners and are not uniform.
Re:Pixel is a unit of length and area by SimplyUnknown on Thursday April 24, 2025 @02:25AM (#43779822)
In medical imaging, data are often acquired using anisotropic resolution. So a pixel (or voxel in 3D) can be an averaged signal sample originating from 2mm of tissue in one direction and 0.9mm in another direction.
Re:Pixel is a unit of length and area by grandempire on Thursday April 24, 2025 @09:32AM (#43782529)
and it’s displayed with a completely different algorithm…
Re:Pixel is a unit of length and area by seba_dos1 on Thursday April 24, 2025 @06:44AM (#43781124)
Many video formats use rectangular pixels. It's less common these days, but there were no squares on DVDs for example.

Even some VGA modes have non-square pixels, and these were used by many games and... Windows 9x splash screen.

Re:Pixel is a unit of length and area by kjkjadksj on Wednesday April 23, 2025 @01:02PM (#43774197)
Well the camera sensor captures a greater dynamic range than the display or print media or perhaps even your eyes, so something has to give. If you ever worked with a linear file without gamma correction you will understand what I mean.
Re:Pixel is a unit of length and area by grandempire on Wednesday April 23, 2025 @04:13PM (#43776184)
And that full dynamic range is in the images’s point samples, ready to be remapped for a physical output.
Re:Pixel is a unit of length and area by pornel on Wednesday April 23, 2025 @09:17PM (#43778419)
We commonly use hardware like LCDs and printers that render a sharp transition between pixels without the Gibbs' phenomenon. CRT scanlines were close to an actual 1D signal (but not directly controlled by the pixels, which the video cards still tried to make square-ish), but AFAIK we've never had a display that is a 2D signal that we assume in image processing.

In signal processing you have a finite number of samples of an infinitely precise contiguous signal, but in image processing you have a discrete representation mapped to a discrete output. It's contiguous only when you choose to model it that way. Discrete → contiguous → discrete conversion is a useful tool in some cases, but it's not the whole story.

There are images designed for very specific hardware, like sprites for CRT monitors, or font glyphs rendered for LCD subpixels. More generally, nearly all bitmap graphics assumes that pixel alignment is meaningful (and that has been true even in the CRT era before the pixel grid could be aligned with the display's subpixels). Boxes and line widths, especially in GUIs, tend to be designed for integer multiples of pixels. Fonts have/had hinting for aligning to the pixel grid.

Lack of grid alignment, an equivalent of a phase shift that wouldn't matter in pure signal processing, is visually quite noticeable at resolutions where the hardware pixels are little squares to the naked eye.

Re:Pixel is a unit of length and area by grandempire on Wednesday April 23, 2025 @11:53PM (#43779116)
I think you are saying there are other kinds of displays which are not typical monitors and those displays show different kinds of images - and I don’t disagree.
Re:Pixel is a unit of length and area by pornel on Thursday April 24, 2025 @09:38AM (#43782629)
I'm saying "digital images" are captured by and created for hardware that has the "little squares". This defines what their pixels really are. Pixels in these digital images actually represent discrete units, and not infinitesimal samples of waveforms.

Since the pixels never were a waveform, never were sampled from such signal (even light in camera sensors isn't sampled along these axis), and don't get displayed as a 2D waveform, the pixels-as-points model from the article at the top of this thread is just an arbitrary abstract model, but it's not an accurate representation of what pixels are.

Re:Pixel is a unit of length and area by nextaccountic on Thursday April 24, 2025 @04:54AM (#43780581)
That's only for images coming directly from a camera. If the images were generated in another way, the idea that a pixel is a little square is sometimes ok (example, pixel art)
Re:Pixel is a unit of length and area by calibas on Wednesday April 23, 2025 @10:27AM (#43772631)
If you want a good example of what happens when you treat pixels like they're just little squares, disable font smoothing. Anti-aliasing, fonts that look good, and smooth animation are all dependent upon subpixel rending.

https://en.wikipedia.org/wiki/Subpixel_rendering

Edit: For the record, I'm on Win 10 with a 1440p monitor and disabling font smoothing makes a very noticeable difference.

Re:Pixel is a unit of length and area by dahart on Wednesday April 23, 2025 @12:16PM (#43773770)
You’re conflating different topics. Subpixel rendering and font smoothing is often implemented by treating the subpixels as little rectangles, which is the same mistake as treating pixels as squares.

Anti-aliasing can be and is done on squares routinely. It’s called ‘greyscale antialiasing’ to differentiate from LCD subpixel antialiasing, but the name is confusing since it works and is most often used on colors.

The problem Alvy-Ray is talking about is far more subtle. You can do anti-aliasing with little squares, but the result isn’t 100% correct and is not the best result possible no matter how many samples you take. What he’s really referring to is what signal processing people call a box filter, versus something better like a sinc or Gaussian or Mitchell filter.

Re:Pixel is a unit of length and area by calibas on Wednesday April 23, 2025 @06:16PM (#43777262)
We're going off on tangents here, I only brought this up to reinforce the idea that a pixel is not a little square.

And I guess I should have explicitly stated that I'm not talking about high-DPI displays, subpixel rendering obviously doesn't do much good there!

My point is simply this, if you don't treat pixel like discrete little boxes that display a single color, you can use subpixels to effectively increase the resolution on low-DPI monitors. Yes, you can use greyscale antialiasing instead, you will even get better performance, but the visual quality will suffer on your standard desktop PC monitor.

Re:Pixel is a unit of length and area by dahart on Thursday April 24, 2025 @02:06AM (#43779738)
Yes, LCD subpixel rendering is a tangent, it’s not relevant to Alvy-Ray’s point. And again, LCD subpixel rendering is treating pixels as little squares; cutting a square pixel into 3 rectangles doesn’t change anything with respect to what Alvy Ray was talking about. So-called grayscale antialiasing on low-DPI displays is nearly as good as LCD subpixel, the quality differences are pretty minor. (Plus it’s a tradeoff and you get the downside of chromatic aberration.) I think you’re suggesting visual quality suffers when you don’t do any antialiasing at all, which is true, but that’s not what Alvy-Ray was talking about. Treating a pixel like a square does not imply lack of antialiasing.
Re:Pixel is a unit of length and area by calibas on Thursday April 24, 2025 @10:54AM (#43783580)
I should have been more clear, I'm not responding to Alvy-Ray's article, I was simply adding to the idea that a pixel is not a little square.

If you don't treat a logical pixel like a discrete little square, you can take advantage of subpixel geometry to effectively increase the resolution. It's not the same as antialiasing, even though it can be used for antialiasing.

Arguably, instead of treating pixels as squares, you're treating them as three times as many rectangles, but that should still contradict the mental model of pixels as little squares.

Re:Pixel is a unit of length and area by dahart on Thursday April 24, 2025 @09:59PM (#43789465)
> that should still contradict the mental model of pixels as little squares.

No, it doesn’t. You seem to be missing this point. Using subpixel rectangles does not in any way correct the misconception, it perpetuates it.

Re:Pixel is a unit of length and area by kccqzy on Wednesday April 23, 2025 @10:34AM (#43772712)
This might have been a good example fifteen years ago. These days with high-DPI displays you can't perceive a difference between font smoothing being turned on and off. On macOS for example font smoothing adds some faux bold to the fonts, and it's long been recommended to turn it off. See for example the influential article https://tonsky.me/blog/monitors/ which explains that font smoothing used to do subpixel antialiasing, but the whole feature was removed in 2018. It also explains that this checkbox doesn't even control regular grayscale antialiasing, and I'm guessing it's because downscaling a rendered @2x framebuffer down to the physical resolution inherently introduces antialiasing.
Re:Pixel is a unit of length and area by calibas on Wednesday April 23, 2025 @11:16AM (#43773151)
Maybe true for Mac users, but the average Win 10 desktop is still using a 1080p monitor.
Re:Pixel is a unit of length and area by hulium on Wednesday April 23, 2025 @01:40PM (#43774649)
The users may have 1080p monitors, but even Windows does not do subpixel antialiasing in its new apps (UWP/WinUI) anymore. On Linux, GTK4 does not do subpixel antialiasing anymore.

The reason is mostly that it is too hard to make it work under transformations and compositing, while higher resolution screens are a better solution for anyone who cares enough.

Re:Pixel is a unit of length and area by calibas on Wednesday April 23, 2025 @02:32PM (#43775240)
> even Windows does not do subpixel antialiasing in its new apps (UWP/WinUI) anymore

This is a little misleading, as the new versions of Edge and Windows Terminal do use subpixel antialiasing.

What Microsoft did was remove the feature on a system level, and leave implementation up to individual apps.

Re:Pixel is a unit of length and area by verall on Wednesday April 23, 2025 @03:05PM (#43775583)
Is this why font rendering on my Win11 laptop looks like shit on my 1440p external monitor?

Laptop screen is 4k with 200% scaling.

Seriously the font rendering in certain areas (i.e. windows notification panel) is actually dogshit. If I turn off the 200% scaling on the laptop screen then reboot it looks correct again.

Re:Pixel is a unit of length and area by wat10000 on Wednesday April 23, 2025 @10:39AM (#43772757)
I don't think that's true anymore. Modern high-resolution displays have pixels small enough that they don't really benefit from sub-pixel rendering, and logical pixels have become decoupled from physical pixels to the point of making sub-pixel rendering a lot more difficult.
Re:Pixel is a unit of length and area by 7jjjjjjj on Wednesday April 23, 2025 @11:24AM (#43773236)
It's still true for everyone who doesn't have a high DPI monitor. We're talking about a feature that doubles the price of the display for little practical value. It's not universal, and won't be for a long time.
Re:Pixel is a unit of length and area by nayuki on Wednesday April 23, 2025 @12:37PM (#43773949)
Agreed. The fact that a pixel is an infinitely small point sample - and not a square with area - is something that Monty explained in his demo too: https://youtu.be/cIQ9IXSUzuM?t=484
Re:Pixel is a unit of length and area by kazinator on Wednesday April 23, 2025 @04:13PM (#43776185)
[delayed]
Re:Pixel is a unit of length and area by Dylan16807 on Wednesday April 23, 2025 @07:53PM (#43777957)
> A 100x100 thumbnail that was reduced from a 1000x1000 image might have pixels which are derived from 100 samples of the original image (e.g. a simple average of a 10x10 pixel block). Or other possibilities.

And if a downsampling algorithm tries to approximate a point sample, it'll give you a massively increased chance of ugly moire patterns.

The audio equivalent is that you drop 3/4 of your samples and it reflects the higher frequencies down into the lower ones and hurts the quality. You need to do a low-pass filter first. And "point samples from a source where no frequencies exist above X, also you need to change X before doing certain operations" is very different and significantly more complicated than "point samples". Point samples are one leaky abstraction among many leaky abstractions, not the truth. Especially when an image has a hard edge with frequencies approaching infinity.

Re:Pixel is a unit of length and area by kazinator on Wednesday April 23, 2025 @08:04PM (#43778014)
But from the pixels alone, you don't know whether the moire is an artifact of sampling of something that was free of moire, or whether an existing image of moire was sampled and reproduced.
Re:Pixel is a unit of length and area by Dylan16807 on Wednesday April 23, 2025 @08:47PM (#43778268)
I have the source image. I know the downscaled version looks awful and wrong. I know the naive point algorithm was the wrong one to use.

Someone with just the small version wouldn't know if it's supposed to look like that, but we're not asking them.

Unless they can infer it's a picture of normal objects, in which case they can declare it's moire and incorrect.

Re:Pixel is a unit of length and area by dahart on Wednesday April 23, 2025 @12:58PM (#43774158)
Eh, calling it infinitely small is at least as misleading as calling it a square. While they are both mostly correct, neither Monty’s explanation nor Alvy-Rays are all that good. Pixels are samples taken at a specific point, but pixel values do represent area one way or another. Often they are not squares, but on the other hand LCD pixels are pretty square-ish. Camera pixels are integrals over the sensor area, which captures an integral over a solid angle. Pixels don’t have a standard shape, it depends on what capture or display device we’re talking about, but no physical capture or display devices have infinitely small elements.
Re:Pixel is a unit of length and area by wtallis on Wednesday April 23, 2025 @01:47PM (#43774720)
Camera pixels represent an area, but pixels coming out of a 3D game engine usually represent a point sample. Hand-drawn 2d pixel art is explicitly treating pixels as squares. All of these are valid uses that must coexist on the same computer.
Re:Pixel is a unit of length and area by dahart on Thursday April 24, 2025 @02:24AM (#43779820)
> Camera pixels represent area, but pixels coming out of a 3D game engine usually represent a point sample

Having worked on both game engines and CG films, I think that’s misleading. Point sample is kind of an overloaded term in practice, but I still don’t think your statement is accurate. Many games, especially modern games are modeling pixels with area; explicitly integrating over regions for both visibility and shading calculations. In fact I would say games are generally treating pixels as squares not point samples. That’s what DirectX and Vulkan and OpenGL do. That’s what people typically do with ray tracing APIs in games as well. Even a point sample can still have an associated area, and games always display pixels that have area. The fact that you can’t display a point sample without using area should be reason enough to avoid describing pixels that way.

Re:Pixel is a unit of length and area by wtallis on Thursday April 24, 2025 @04:26PM (#43787111)
Conservative rasterization is definitely an area-based square pixel technique. I'm not sure how prevalent it is now that it's been available for a few years. Multisample antialiasing patterns are also constructed with square pixels in mind, but it's relatively rare in games these days due in part to the popularity of deferred rendering.

I'm not sure how raytracing as done by video games can be construed a more area-based than point sample based. It seems to me like ray-hit testing is done in a point-sample manner, and games aren't doing multiple rays per pixel. (They're usually doing less than one ray per pixel, and averaging over several frames that sampled from different locations within a square pixel, then running it all through a blurring denoiser filter and upscaling, but the result is so bad for anything other than a static scene that I don't think it should be used to support any argument.)

Re:Pixel is a unit of length and area by dahart on Thursday April 24, 2025 @09:54PM (#43789429)
> I’m not sure how ray tracing done by video games can be construed a more area-based than point sample based. It seems to me like ray-hit testing is done in a point-sample manner, and games aren’t doing multiple rays per pixel.

Ignoring the denoiser, games are quite commonly using Box Filter for pixels. That’s the square in square pixels. The point sampling is in service of an integral whose shape is a square, and that’s the problem Alvy Ray is talking about.

That point sampling is distinctly different from the “point sample” that represents the pixel value itself, so let’s not conflate them. The averaging over an area that is square shaped, whether it’s multiple samples or over time, is the reason that the pixel shape is summarized as square, not as a point.

Re:Pixel is a unit of length and area by kjkjadksj on Wednesday April 23, 2025 @01:04PM (#43774219)
Some early variants of sony a7 had fewer but larger pixels to improve light gathering at high iso.
Re:Pixel is a unit of length and area by jansan on Wednesday April 23, 2025 @12:58PM (#43774160)
Signal processing engineers against the rest of the world.