They've made a follow up to that called MeshSplatting. https://meshsplatting.github.io/

The Dreams game engine (only for PS4) does this and is able to achieve scenes that no other game engine can. Here's a technical talk about how they did it: https://www.youtube.com/watch?v=u9KNtnCZDMI

Pretty sure they use SDFs, not triangle or gaussian splatting

If I recall their talk correctly, models are unions of SDFs, which get sampled into a point cloud, and each point gets splatted as another SDF primitive called a "fleck".

Ah thanks for the correction. The splat terminology they also use confused me.

I think SDFs is the data underneath, but baked into SDF voxel tiles at runtime and not always evaluating SDF shapes. The SDF tiles still get splatted.

At one point it used triangles for the large splats, and still compute shaders for the small ones due to quad overdraw. I'm not sure if the final version stuck with that or went all compute, but quad complexity is also why nanite does compute shaders for small triangles and traditional rasterization for the larger ones.

In my experience it isn't really about quad overdraw/helper lanes, but rather about primitive assembly. On my RTX 4090, vertex shaders max out at about 20M tris/ms, while mesh shaders, which bypass primitive assembly, can reach 40M tris/ms.

But it's a bit difficult to find the exact reason since profilers won't really tell you a concrete bottleneck when drawing vast amounts of tiny tris.

The Sony Dreams beta in 2018, was truly revolutionary and even today the world hasn't caught up with a lot of its breakthroughs. I'd been deeply involved in the design of high-end, pro digital media authoring tools for two decades and when I saw that beta it blew my fucking mind.

Media Molecule (LittleBigPlanet) spent over 8 years creating Dreams, which would never have happened without Sony's funding, but the tragedy was this brilliant high-end authoring tool being forced onto a games console interface and controller. Sadly, most of my peers in the high-end 2D, 3D, compositing and animation tools design community totally missed learning from Dreams. Even now, there are still innovative UX approaches in Dreams that could be adapted into tools like Blender.

I think that Gaussian splatting is generally better, as the Gaussians are smoother and easier to optimize, if you can use them.

Things like triangle splatting are useful if you have to integrate into an engine that only support triangles. But with modern graphics pipelines, you can implement Gaussians quite efficiently, including newer techniques like Gaussian point splatting that can scale up seamlessly to enormous scenes:

* Video: https://www.youtube.com/watch?v=2AWv_nIfp-U

* Paper: https://jorisar.nl/gaussian_point_splatting/gaussian_point_s...

* Repo: https://github.com/JorisAR/gaussian-point-splatting

Honestly there should be many more types of splatting imo...

There are various papers on variants of Gaussian splatting which replace the Gaussians with some other explicit geometric representation that can be rendered in real time. A triangle soup is just one example. I recall one which uses some sort of tetrahedral foam.

[delayed]

Isn't nanite in unreal engine exactly that?

To be clear, I'm not proficient in it's use, just watched a few videos about it's theory when it was initially announced

No, nanite is just having different levels of detail for objects, and which LOD gets selected is done automatically by the engine. So users can use ultra-high-def models (including 3d scans of real-world objects), and the engine automatically generates and uses lower polygon versions for when the object is far away or whatever.

Strange, when I looked into it that seemed to be just a desirable side effect with the rendering being done via the triangles - which was also supposedly the reason why you could directly use scanned models, despite them having millions of triangles in their model

But again, I've really only super superficial understanding of this, I just understood it like that back then

Yeah, there were other splats before Gaussian splats, so lots of cool possibilities depending on the style you're after. However, hard triangles have hard edges, and then you might be anti-aliasing those edges in some way. Gaussian splats can just be a triangle or two where the pixel shader feathers towards the edges.

Already done in 2023: https://mobile-nerf.github.io/

runs on a phone: https://storage.googleapis.com/jax3d-public/projects/mobilen...