I notice that your renderer doesn't even attempt to render text on the GPU and instead just blits glyphs from an atlas texture rendered with Freetype on the CPU: https://github.com/Const-me/Vrmac/blob/master/Vrmac/Draw/Sha...
In contrast, piet-gpu (the subject of the original blog post) has high enough path rendering quality (and performance) to render glyphs purely on the GPU. This makes it clear you didn't even perform a cursory investigation of the project before making a comment to dump on it and promote your own library.
Correct.
> has high enough path rendering quality (and performance) to render glyphs purely on the GPU
Do you have screenshots showing quality, and performance measures showing speed? Ideally from Raspberry Pi 4?
> This makes it clear you didn't even perform a cursory investigation of the project
I did, and mentioned in the docs, here’s a quote: “I didn’t want to experiment with GPU-based splines. AFAIK the research is not there just yet.” Verifiable because version control: https://github.com/Const-me/Vrmac/blob/bbe83b9722dcb080f1aed...
For text, I think bitmaps are better than splines. I can see how splines are cool from a naïve programmer’s perspective, but practically speaking they are not good enough for the job.
Vector fonts are not resolution independent because hinting. Fonts include a bytecode of compiled programs who do that. GPUs are massively parallel vector chips, not a good fit to interpret byte code of a traditional programming language. This means whatever splines you gonna upload to GPU will only contain a single size of the font, trying to reuse for different resolution will cause artifacts.
Glyphs are small and contain lots of curves. Lots of data to store, and lots of math to render, for comparatively small count of output pixels. Copying bitmaps is very fast, modern GPUs, even low-power mobile and embedded ones, are designed to output ridiculous volume of textured triangles per second. Font face and size are more or less consistent within a given document/page/screen. Apart from synthetic tests, glyphs are reused a lot, and there’re not too many of them.
When I started the project, the very first support of compute shaders on Pi 4 was just introduced in the Mesa upstream repo. Was not yet in the official OS images. Bugs are very likely in versions 1.0 of anything at all.
Finally, even if Pi 4 had awesome support for compute shaders back them, the raw compute power of the GPU is not that impressive. Here in my Windows PC, my GPU is 30 times faster than CPU in terms of raw FP32 performance. With that kind of performance gap, you can probably make GPU splines work fast enough, after spending enough time on development. Meanwhile, on Pi 4 there’s no difference, the quad-core CPU has raw performance pretty close to the raw performance of the GPU. To lesser extent same applies to low-end PCs: I only have a fast GPU because I’m a graphics programmer, many people are happy with their Intel UHD graphics, these are not necessarily faster than CPUs.
> I did, and mentioned in the docs, here’s a quote: “I didn’t want to experiment with GPU-based splines. AFAIK the research is not there just yet.”
Not what I said. I said that you didn't investigate the project discussed in the original blog post before declaring, in your words, that "the quality is not good" and comparing it to your own library.
Vrmacs and piet-gpu are two totally different types of renderer. Vrmacs draws paths by decomposing them into triangles, rendering them with the GPU rasterizer, and antialiasing edges using screen-space derivatives in the fragment shader. This approach works great for large paths, or paths without too much detail per pixel, but it isn't really able to render small text, or paths with a lot of detail per pixel, with the necessary quality. (Given this and the other factors you mentioned in your reply, rendering text on the CPU with Freetype is a perfectly reasonable engineering choice and I am not criticizing it in the slightest.)
In comparison, piet-gpu decomposes paths into line segments, clips them to pixel boundaries, and analytically computes pixel coverage values using the shoelace formula/Green's theorem, all in compute shaders. This is more similar to what Freetype itself does, and it is perfectly capable of rendering high-quality small text on the GPU, in way that Vrmacs isn't without shelling out to Freetype.
Again, to be clear, I'm not criticizing any of the design choices that went into Vrmacs; it looks like it occupies a sweet spot similar to NanoVG or Dear ImGui, where it can take good advantage of the GPU for performance while still being simple and portable. My only point here is that you performed insufficient investigation of piet-gpu before confidently making an uninformed claim about it and putting it in a somewhat nonsensical comparison with your own project.
Oh, you were asking why I said so? Because I have clicked the “notes document” link in the article, the OP used the same tiger test image as me, and that document has a couple of screenshots. And these were the only screenshots I have found. Compare them to screenshots of the same vector image rendered by my library, and you’ll see why I noted about the quality.
> Vrmacs draws paths by decomposing them into triangles, rendering them with the GPU rasterizer, and antialiasing edges using screen-space derivatives in the fragment shader.
More or less, but (a) not always, thin lines are different. (b) that’s a high-level overview but there’re many important details on the lower levels. For instance, “screen-space derivatives of what?” is an interesting question, critically important for correct and uniform stroke widths. The meshes I’m building are rotation-agnostic, and to some extent (but not completely) they are resolution-agnostic too.
> and it is perfectly capable of rendering high-quality small text on the GPU
It is, but the performance overhead is massive, compared to GPU rasterizer rendering these triangles. For real-world vector graphics that doesn’t have too much stuff per pixel that complexity is not needed because triangle meshes are good enough already.
> it looks like it occupies a sweet spot similar to NanoVG
They’re similarities, I have copy-pasted a few text-related things from my fork of NanoVG: https://github.com/Const-me/nanovg/ However, Vrmac delivers much higher quality of 2D vector graphics (VAA, circular arcs, thin strokes, etc), is much faster (meshes are typically reused across frames, I use more than 1 CPU core, and the performance-critical pieces are in C++ manually vectorized with NEON or SSE), and is more compatible (GL support on Windows or OSX is not good, you want D3D or Metal respectively).
> Update (7 May): I did a test to see which threads in the fragment shader get scheduled to the same SIMD group, and there’s not enough coherence to make this workable. In the image below, all pixels are replaced by their mean in the SIMD group (active thread mask + simd_sum)
I cloned the piet-gpu repository and was able to render a very nice image of the Ghostscript tiger: https://imgur.com/a/swyW0gl
The problematic elements are thin black lines. On your image the lines are aliased, visible for the lines which are close to horizontal but not quite. And for curved thin lines, results in visually non-uniform thickness along the line.
One of the stated research goals [1] of piet-gpu is to innovate quality beyond what is expected of renderers today, including conflation artifacts, resampling filters, careful adjustment of gamma, and other things. I admit the current codebase is not there yet, but I am excited about the possibilities in the approach, much more so than pushing the rasterization pipeline as you are doing.
[1]: https://github.com/linebender/piet-gpu/blob/master/doc/visio...
Looking forward, I’d rather expect practical 2D renderers using the tech made for modern games. Mesh shaders, raytracing, deep learning, and even smaller features like sparse textures. These are the areas where hardware vendors are putting their transistors and research budgets.
None of the features you mentioned is impossible with rasterizers. Hardware MSAA mostly takes care about conflation artifacts, gamma is doable with higher-precision render targets (e.g. Windows requires FP32 support since D3D 11.0).
Also in terms of “two draw calls”, does that include drawing text as part of your transparent pass, or are you assuming that your window contents are already rendered to textures?
The library only does grayscale AA for vector graphics.
Subpixel rendering implemented for text but comes with limitations: only works when text is not transformed (or transformed in specific ways, like rotated 180°, or horizontally flipped), and you need to pass background color behind the text to the API. Will only look good if the text is on solid color background, or on slow-changing gradient.
Sub pixel AA is hard for arbitrary backgrounds. I’m not sure many GPUs support the required blend states, and workarounds are slow.
> does that include drawing text as part of your transparent pass
Yes, that includes text and bitmaps. Here’s two HLSL shaders who do that, the GPU abstraction library I’ve picked recompiles the HLSL into GLSL or others on the fly: https://github.com/Const-me/Vrmac/blob/master/Vrmac/Draw/Sha... https://github.com/Const-me/Vrmac/blob/master/Vrmac/Draw/Sha... These shaders are compiled multiple times with different set of preprocessor macros, but I did test with all of them enabled at once.
I see that it's not feasible for a lot of complex 3D graphics, but 2D is (probably) a lot less taxing for modern GPUs?
What is more practical is some form of adaptive supersampling: a lot of pixels are filled by only one path and don't require supersampling. There's also some more heuristics that can be used: one that I want to try out in piet-gpu is to exploit the fact that in 2D graphics, most pixels are only covered by at most two paths. So as a base line we can track only two values per pixel plus a coverage mask, then in rare occasions of three or more shapes overlapping, fall back to full supersampling. This should keep the cost amplification more under control.
What you described is called super-sampling. Supersampling is indeed not terribly hard to implement, the problem is performance overhead. Many parts of graphics pipeline scale linearly with count of pixels. If you render at 16x16 upscaled resolution, that gonna result in 256x more pixel shader invocations, and 256x the fill rate.
There’s a good middle ground called MSAA https://en.wikipedia.org/wiki/Multisample_anti-aliasing In practice, 16x MSAA often delivers very good results for both 3D and 2D. In case of 2D, even low-end PC GPUs are fast enough with 8x or 16x MSAA level.
Initial versions of my library used that method.
The problem was, Raspberry Pi 4 GPU is way slower than PC GPUs. The performance with 4x or 2x MSAA was too low for 1920x1080 resolution, even just for 2D. Maybe the problem is actual hardware, maybe it’s a performance bug in Linux kernel or GPU drivers, I have no idea. I didn’t want to mess with the kernel, I wanted a library that works fast on officially supported 32-bit Debian Linux. That’s why I bothered to implement my own method for antialiasing.
I think it is - as far as I know most modern GPUd implement MSAA at the hardware level, and that's why even a mobile GPU can handle. 8x MSAA at 1080p.
I don't know anything about the Raspberry Pi GPU, but maybe you'd have better results switching to FXAA or SMAA there (by which I mean faster, not visually better).
I’ve thought about that, but decided I want to prioritize the quality. 16x MSAA was almost perfect in my tests, 8x MSAA was still good. With levels lower than that, the output quality was noticeably worse than Direct2D, which was my baseline.
And another thing. I have found out stroked lines much thinner than 1px after transforms need special handling, otherwise they don’t look good: too aliased, too thick, and/or causing temporal artifacts while zooming continuously. Some real-world vector art, like that ghostscript’s tiger from Wikipedia, has quite a lot of such lines.
But, font and path rendering, for example in web browsers and/or with PDF or SVG - these things can benefit in both efficiency and in quality from using analytic antialiasing methods. 2D vector rendering is a place where doing something harder has real payoffs.
Just a fun aside - not all supersampling algorithms are equally good. If you use the wrong filter, it's can be very surprising to discover that there are ways you can take a million samples per pixel or more and never succeed in getting rid of aliasing artifacts. (An example is if you just average the samples, aka use a Box Filter.) I have a 2D digital art project to render mathematical functions that can have arbitrarily high frequencies. I spent money making large format prints of them, so I care a lot about getting rid of aliasing problems. I've ended up with a Gaussian filter, which is a tad blurrier than experts tend to like, because everything else ends up giving me visible aliasing somewhere.
If you use the fillRectangle method https://github.com/Const-me/Vrmac/blob/1.2/Vrmac/Draw/iDrawC... to draw them I think you should get solid fill. That particular API doesn’t use AA for that shape. Modern GPU hardware with their rasterization rules https://docs.microsoft.com/en-us/windows/win32/direct3d11/d3... is good at that use case, otherwise 3D meshes would contain holes between triangles.
If you render them as 2 distinct paths, filled, not stroked, and anti-aliased, you indeed gonna see a thin line between them. Currently, my AA method shrinks filled paths by about 0.5 pixels. For stroked paths it’s the opposite BTW, the output is inflated by half of the stroke width (the mid.point of the strokes correspond to the source geometry).
You can merge boxes into a single path with 2 figures https://github.com/Const-me/Vrmac/blob/1.2/Vrmac/Draw/Path/P..., in this case C++ code of the library should collapse the redundant inner edge and the output will be identical to a single box, i.e. solid fill. Will also render slightly faster because less triangles in the mesh.
I have used different approach: https://github.com/Const-me/Vrmac#vector-graphics-engine
My version is cross-platform, tested with Direct3D 12 and GLES 3.1.
My version does not view GPUs as SIMD CPU, it actually views them as a GPU.
When rendering a square without anti-aliasing, the library will render 2 triangles. When rendering a filled square with anti-aliasing, the library will render about 10 triangles, large opaque square in the center, and thin border about 1 pixel thick around it for AA.
It uses hardware Z buffer with early Z rejection to save pixel shaders and fill rate. It uses screen-space derivatives in the pixel shader for anti-aliasing. It renders arbitrarily complex 2D scenes with only two draw calls, one front to back with opaque stuff, another back to front with translucent stuff. It does not have quality issues with stroked lines much thinner than 1px.