News for hackers. Stuff that matters.

A comment on Make-A-Video: AI system that generates videos from text. Parent comment

Re:Make-A-Video: AI system that generates videos from text by tmjdev on Thursday September 29, 2022 @11:39AM (#33022212)
I am curious if there has been any research on temporal attention in humans. I'm not sure how you'd quantify it. But in myself I know that I'm constantly predicting where something will be or what it will look like based on how it did a second ago. It's probably the root of reflexes.
[ Parent | on HN ]
Re:Make-A-Video: AI system that generates videos from text by mlsu on Thursday September 29, 2022 @03:18PM (#33025705)
Your comment reminded me of this video [1]

They put an eye tracker on someone and captured their motion when walking in some rough terrain. You can sort of see that the person is focusing on the most likely place their foot will go next.

[1] https://www.youtube.com/watch?v=ph6uUHq3a-g

I think that we will discover that there is a more efficient way to encode temporal relationships, which appears to be "just throw transformers at it." My guess is that it will be in a more conceptual latent space that this attention will be applied.

[ Parent | on HN ]