News for hackers. Stuff that matters.

A comment on Make-A-Video: AI system that generates videos from text. Parent comment

Re:Make-A-Video: AI system that generates videos from text by liuliu on Thursday September 29, 2022 @10:45AM (#33021431)
Their video training set is still small (10M + 10M). A lot of the interpolation artifacts seems come from the model haven't acquired enough real-world understanding of "natural" movements (looking at the horse running and the sail-boat examples). I suspect scaling this up to 10x would have much less artifacts.

Reading the paper, it seems to be the "right" approach (separating temporal / spatial for both convolution and attention). Thus, I am optimistic what remains is to scale it up.

[ Parent | on HN ]
Re:Make-A-Video: AI system that generates videos from text by MichaelCollins on Thursday September 29, 2022 @11:37AM (#33022179)
The horse leg motion in these examples is really poor. But to be fair, horse legs are very difficult for human animators too, generally necessitating the use of reference photos to get it right. The first time somebody got a series of photographs showing a horse in motion caused quite a stir: https://en.wikipedia.org/wiki/The_Horse_in_Motion
[ Parent | on HN ]