Re:Compressing Gaussian Splats
by karaterobot on Wednesday December 06, 2023 @11:41AM
(#38546208)
The article links to a page with this video embedded in it, which I found very helpful. I'm not in this field and this was the first time I'd heard of Gaussian splatting; the original article sort of assumes you're not a rando coming to the page from Hacker News, that you are sold on the idea already, and use it so much that you need to worry about optimization.
Polygons are great because you can make a solid, closed surface. But, they are a pain because you have to maintain the connectivity between the vertices. Adding or removing polygons in a mesh is serious work. Especially doing so on a GPU. Most LOD schemes do heavy precomputation at data compilation time and have the GPU just view a sliding window into static results.
Splats however are loose points. It’s harder to get a solid appearance from them. But, you can add/remove/move each point independently.
Splats don’t really use neural techniques. But, they are well set up for neural techniques to work with them. Generating the splats from photos uses some sort of linear algebra optimization/back-propagation. The Gaussian aspect is required to enable that to work because gaussians are differentiable. Hopefully that means GS should fit in well with recent advancements in differentiable rendering.
Re:Compressing Gaussian Splats
by reactordev on Wednesday December 06, 2023 @09:12AM
(#38544113)
https://huggingface.co/blog/gaussian-splatting
Think of this as a LiDAR visualization on steroids... A point cloud of information that can be used to render "The Blob".
But the blobs can change color and opacity depending on direction you are viewing them (spherical harmonics) which allows to encode transparency and specular reflections.
E.g. what's the benefit of using splats vs traditional polygons? is it somehow easier for the neural network to create these from the 2D photos? or what's the magic behind this?