News for hackers. Stuff that matters.
Re:Lumiere: A Space-Time Diffusion Model for Realistic Video... by danielbln on Wednesday January 24, 2024 @07:09AM (#39116348)
I'm excited about these text 2 video models, what I'm not excited about is that it's Google publishing this. That means, no code, nothing deployed to try and most likely we will never ever hear about this ever again, or maybe it quietly hits Vertex AI in 2 years (like Imagen) and no-one will care.

Also, ever since the Gemini marketing video shenanigans, I don't really feel like trusting whatever Google's research says they have, if I can't test it myself.

Re:Lumiere: A Space-Time Diffusion Model for Realistic Video... by sho_hn on Wednesday January 24, 2024 @07:22AM (#39116436)
> Also, ever since the Gemini marketing video shenanigans, I don't really feel like trusting whatever Google's research says they have, if I can't test it myself.

The video was released by Google product marketing for a launch to customers, not research.

I'm still somewhat confused by this one. I understand the community has decided to be harsh on Google for that video to draw a line - fair, truth in advertising, etc. -, but at the same time, we all had an understanding of where that tech is at currently and the pace it progresses at. Did anyone watching it really assume it was realtime? Can we not differentiate between technical publications and marketing anymore? Do we have to vilify everyone in an R&D department for the sins of the product marketing wing?

Re:Lumiere: A Space-Time Diffusion Model for Realistic Video... by whywhywhywhy on Wednesday January 24, 2024 @07:49AM (#39116638)
We’re all harsh on it because we all had to see it being posted around by naive people as being amazing when it’s completely faked and misses half of the prompting, all of the latency.

It was completely dishonest. Considering how trash Googles actual AI products are they deserve to be dragged even more over that video.

Re:Lumiere: A Space-Time Diffusion Model for Realistic Video... by IshKebab on Wednesday January 24, 2024 @03:34PM (#39122300)
How is it completely faked? The video didn't give me the impression that the results were calculated instantly, or that no prompts were required.
Re:Lumiere: A Space-Time Diffusion Model for Realistic Video... by ImprobableTruth on Wednesday January 24, 2024 @07:32AM (#39116501)
The issue isn't that it's not real-time/sped-up, it's that it doesn't actually take video as input, but multiple hand picked stills.
Re:Lumiere: A Space-Time Diffusion Model for Realistic Video... by sjwhevvvvvsj on Wednesday January 24, 2024 @11:07AM (#39118999)
“PoC or GTFO” as they say.
Re:Lumiere: A Space-Time Diffusion Model for Realistic Video... by addandsubtract on Wednesday January 24, 2024 @11:35AM (#39119341)
PapersWithCode or GTFO.

[0] https://paperswithcode.com/

Re:Lumiere: A Space-Time Diffusion Model for Realistic Video... by baldgeek on Wednesday January 24, 2024 @12:05PM (#39119727)
2 clicks from the Posted Link: "Read Paper", then "Code, Data and Media" tab will get you the dataset used (https://paperswithcode.com/dataset/ucf101)
Re:Lumiere: A Space-Time Diffusion Model for Realistic Video... by sjwhevvvvvsj on Wednesday January 24, 2024 @01:35PM (#39120964)
Well in the Ai/ML era maybe “models or gtfo” is better. Training data is just common crawl for half these LMs.
Re:Lumiere: A Space-Time Diffusion Model for Realistic Video... by gs17 on Thursday January 25, 2024 @01:39PM (#39133034)
That's not the dataset used for training. From the paper:

>We train our T2V model on a dataset containing 30M videos along with their text caption. [...] We evaluate our model on a collection of 113 text prompts describing diverse objects and scenes. The prompt list consists of 18 prompts assembled by us and 95 prompts used by prior works (Singer et al., 2022; Ho et al., 2022a; Blattmann et al., 2023b) (see App. B). Additionally, we employ a zero-shot evaluation protocol on the UCF101 dataset >