Well fuck me, I had no idea. I figured that 'simple 2D vectors' would be beyond piss-easy for modern GPUs. I'd never considered that it wasn't the actual math space that was accelerated, rather the fast memory mapping of everything on presumably comparatively simple geometries. You've just turned my view of the world upside down :(
they are, but on nvidia only... https://developer.nvidia.com/gpu-accelerated-path-rendering
But signed distance fields work well and easily on GPUs.
Here's a recent article and discussion about it:
Vector graphics on GPU (gasiulis.name)
235 points by mpweiher 4 months ago | hide | past | favorite | 95 comments
(OK those links are for NVidia, but if they can do it I guess others can too ?) (also see https://www.researchgate.net/publication/262357352_GPU-accel... )
I've seen some font rendering work that has already embraced them for high perf rasterization.
Still, may be hard to get flash equivalency.
It's definitely not a simple solution, but might enable you to do runtime rasterisation with a good framerate on the GPU rather than pre-rasterising all the vector art.
When Adobe hacked on "HTML5 support" to Animate, they did exactly the same thing the OP did. It renders every shape in the FLA to a sprite sheet and then draws it to a canvas tag. If you have knowledge over what will be drawn ahead of time, this is the most reasonable thing you can do.
Even before HTML5 support, the AS3 Starling framework that let you "use the GPU" would pre-render all your vector assets to bitmap textures at runtime. And that was a framework built for Flex developers; if you were accustomed to building things on the timeline, you rendered on CPU, because that's where all of Flash's very particular rendering logic has to live.
Ruffle gets around this by tesselating every shape into a GPU-friendly triangle mesh. This lets us render stuff that's technically vectors on GPU. But as you can imagine, this creates its own problems:
- Flash has very specific stroke-scaling rules. If a stroke is smaller than 1px[0], it will be rounded up to 1px. This is how Flash's "hairline stroke" feature works: it actually asks for a 1/20px[1] stroke, and that gets rounded up to whatever the current scaling factor for that shape is. When your stroke is a polygon mesh, you can't vary the stroke to match Flash without retesselating, so hairline strokes on shapes that stretch don't animate correctly.
- Likewise, any stretch of a stroke that changes the aspect ratio also distorts the stroke, since its baked into the tesselated mesh. There's a minigolf game that does this to hairlines and it will basically never look right in Ruffle.
- Tesselated vectors lose their smoothness, so we have to sort of guess what scale the art is drawn at and add enough detail polygons for things to render correctly. Most of the time we get it right. However, there are some movies that do crazy things like store all the art at microscopic scale and blow it up. This provides a compression benefit, because it quantizes the points on the art with little visual difference on Flash Player. On Ruffle, however, the art becomes very low-poly.
- All the tesselation work takes a significant amount of time. There are certain movies (notably, a lot of Homestuck) that would hang the browser because of how much ridiculously complicated vector art was being processed before the movie even loads. We had to actually limit how much art could tesselate per frame, and expose that to movies as bytesLoaded, which is why Ruffle movies have preloaders even though we don't support streaming download.
There's another approach to drawing Beziers on GPU: drawing the hull of control points with a procedural texture that calculates the underlying polynomial. This is especially simple for quadratic curves (the ones with one control point), which is what all Flash timeline art[2] is.
However, strokes are more complicated. You'd think we could just take the hull of the stroke and draw that as a fill, but you can't. This is because the offset of a Bezier curve is not a Bezier curve. Drawing the stroke in a pixel shader would make sense, except you still need to define a polygon mesh around your stroke with a reasonable texture coordinate system to make the math work. And the polygonal outlines of Bezier curves can get really funky; there's no obvious way to quickly say "here's a curve, now give me the polygon that encloses a 5px stroke around it". Remember how tesselation takes so long that it would hang Ruffle?
[0] I'm not sure if this is virtual or device pixels.
[1] Flash specifies vectors in fixed-point units called twips. That's why the zoom factor on Flash movies was locked to 2000%.
[2] Flash can draw cubics - the two-control-point curves you think of when you think Bezier - but only in response to an AS3 graphics command. We haven't implemented this in Ruffle yet.