Sacco215

I've done some work in this area and here are my two cents:

1. Convolution-based architectures are terrible: I've trained Convolution based architectures and they were almost never scalable. Lately I've switched to transformer based AE and they are soo much better! We even managed to get Chinchilla-style scaling laws out of Transformer AE.

2. VAEs are terrible for downstream tasks: We've tried training Video diffusion models out of MAE and VAE (same architecture) and the MAE is hands down better.

3. This whole field is not science. There is no rigorous way of defining what a "good latent" really is. End-to-end methods (such as PixNerd) are the future, since they eliminate the need to hand-design and optimize the interface between separate components. That being said I've never seen a neural-field based video model and I've done some limited experiments on it with underwhelming results

I've been bikepaking for two years now and I wish I had this tool

I've used komoot in the past but I was never satisfied with the paths it suggested and had to rely on local guides

I've checked with some of the past trips I id and it guesses correctly all of the best paths