ArthurBrussee

You're welcome and thank you for trying it out!

Hard at work to make performance better - the "main" kernels are at least as fast as gSplat, so now need to remove other overheads.

That, and make splatting train more efficiently in general, lots of compute is wasted on small steps.

Ps: the web version takes a minute to warm up and is generally slower, do try a native version if you haven't yet!

That was in some way the original motivation for the project!

I think if you are reconstructing your own data the algorithm better just work, without input, ideally.

But, imagine you could add in generated videos. Lay down a camera path, tell it what to generate, and add it to the reconstruciton. A brush stroke one might say ;)

Yes :) But it will take a while!

The input to this are two things - images, and camare poses. The camera poses tell you where each camera was in 3D space (and some of its properties).

The training takes this information, to make a 3D model out it, visually matching all your photos.

COLMAP can still be quite expensive & a hassle sadly, order half hour, as opposed to seconds. There are modern alternatives like https://lpanaf.github.io/eccv24_glomap/, or even deep learning based systems like https://github.com/naver/dust3r

This is definitely still a big blocker to adoption. The goal is to get to a more all-in-one system. The splatting optimization can also help align cameras, if they don't start out entirely random, so any system to quickly provide a good "initial guess" will help here. At least for mobile devices, initialization from ARCore / ARKit poses should be enough.

Keep an eye out :)

Not a dumb question! This first version is still mainly targetted at people who are in this area and generate some excitement, I do hope to make this more accesible though!

The inputs are 1. images 2. with a pose. The usual way to get poses for your images is https://github.com/colmap/colmap.

The output is a 3D model. Specifically a "Gaussian Splat", which is a sort of fuzzy point cloud. There are some tools out there to view & edit these (besides Brush), eg. https://playcanvas.com/supersplat/editor.

Yeah this is the next big challenge. There's some ideas what to do, but one step at a time!

Devices & operating systems! Windows/Mac/Linux, AMD / nvidia / built-in GPUs, Android/iOs, or running in a browser context.

My bad, I really bungled the original tweet the title is from :)

I'm a bit out of the loop on async runtimes. I know Tokio is of course the big on, but, that seemed much to heavy to just run some tasks, and isn't very WASM compatbile afaik.

Otherwise there's smol, and maybe others? Would love to hear what a good web WASM compatible async framework is nowadays!

Love to hear it!! Most viewers take some shortcuts, like only sorting every so often, it's good to hear the difference is noticable :)

Training a splat requires a lot less setup with this, but does still require running COLMAP(https://github.com/colmap/colmap) first, which is still a big barrier... one thing at a time!

Its really the latest incarnation in the field of Photogrammetry https://en.wikipedia.org/wiki/Photogrammetry - aka, converting 2D images / video to 3D data.

Imagine one of those house tours on Zoopla on steroids, or street view but smoother.

I think the title is taken from my original tweet which I really bungled, my bad :) Hopefully the readme does a better job!

Wonder if Rust should pursue this as a feature as well - might be hard to promise this as a fully fledged feature when Cranelift comes into the picture though. Still it could be possible to ship a linker that can handle cross-compiling like Zig and compile C code with LLVM irregardless of the Rust backend.

Either way, hope Zig and Rust continue to inspire each other a bit!

Is it me or are those results somewhat underwhelming if anything? Dedicated hardware for a 2x speedup at best, tossup for most results, and only competes in some categories. Not to be just a NVIDIA fan here, surely there is value in dedicated training hardware, but just surprising that benefit isn't bigger!

If every single transaction in the world was Bitcoin it would still be a _ridicilous_ amount of energy by several orders of magnitude. It is totally, fundamentally, broken

So much of a game engine is how it's rendering works. It interacts with everything else. I don't think it's an unreasonable thing for a game engine to want to implement themselves.

Why are we going through all this trouble to make something that in the best case is not worse than existing IDEs?

I can't help but feel that a visualization window + a cacheResult(data, fileName) function would be superior to what Jupyter can ever hope to be

The more you think about salary based on location the less it makes sense. I got an offer for FB London that is as half of offer for bay area. Where does that $$$ go? Maybe some tax, but then a bay area office is also more expensive. I'd be doing the exact same work.

Coming from a freelance perspective, the only equation should be 'how much value do I provide company X, how much of that do they give back'

Hope so! Never understood* why big tech are so far outside SF / San Jose and have such long commutes for their employees.

*$$$ I guess

I unironically can't wait to read release notes about newer versions of my food

I wonder what percentage the FAANG like companies donate. On the one hand, it's probably quite low, on the other hand, they do run a lot of projects themselves & donate labours to others. Still that's hardly sustainable for the rest of OSS

What a world... Thanks for working on Zig, can't wait to see what comes of it. Anything to get some languages back that bring some joy into programming back!

Article is a little snarky but I really agree with the main point: we've accepted terrible tools and comlile times. It's been years since real Innovation happened in this space - regressions if anything. Fingers crossed for Rust improving their compiler, C++ getting their Modules, C# speeding up Roslyn... Or blow makes Jai's million LOC/s really happen.

That's great but I hate how they mark suggestions with a star and add an extra comment. It should just work, not add UI clutter and noise

Jai is trying to fit that C space too. Similair ideas to Zig, but more aimed at the games industry. Curious to see which ends up gaining more traction (if either do, fingers crossed)