News for hackers. Stuff that matters.
Re:Make-A-Video: AI system that generates videos from text by VoodooJuJu on Thursday September 29, 2022 @10:13AM (#33020888)
A lot of people saying it's over for traditional movie-making - lmao. I look at these and see nothing but uncanny valley artifacts, and I don't think it will improve much from here.

It's like self-driving cars. They use almost very effective statistical models, certainly better than our previous models, but they never seem to shake off that "almost" and become truly effective.

Re:Make-A-Video: AI system that generates videos from text by criddell on Thursday September 29, 2022 @10:17AM (#33020950)
> I don't think it will improve much from here

That's a bold prediction. Why do you think that?

The first thing I thought was the exact opposite. This isn't very good, but it's only version 1. Motion pictures are less than 150 years old. In another 150 years I bet virtual filmmaking will progress a lot.

Re:Make-A-Video: AI system that generates videos from text by javajosh on Thursday September 29, 2022 @10:42AM (#33021370)
I don't agree with the prediction but I will say that at least in America, we've been overly optimistic about almost everything since world war II. Is particularly painful when you look at the imagination of science fiction versus the reality of science in the realm of say space travel, especially interstellar travel. This imagination effect gets the public behind your new invention in part because it seems to be just the beginning the start of something new. The tip of the iceberg. But sometimes it's really just a tiny bit of ice.

For the record, I'm actually rather bullish on self-driving cars. There's nothing physically impossible about solving the problem, but I'm not surprised it's harder than it sounds. But I don't see humans being fundamentally prevented from solving the problem in the same way that humans are fundamentally prevented from ever engaging in everyday space travel.

Re:Make-A-Video: AI system that generates videos from text by mikepurvis on Thursday September 29, 2022 @10:45AM (#33021427)
Absolutely, particularly if you see this as simply another tool in the belt, like how Jurassic Park's thirty year old special effects absolutely stand the test of time because of them being a convincing mix of early CGI and puppetry.

It's not hard to imagine that this kind of thing could end up doing a lot of the heavy lifting for things like background scenes in the future, opening up the kind of stuff we saw in The Mandalorian, Game of Thrones, and the LOTR film trilogy to increasingly lower and lower budget productions.

Re:Make-A-Video: AI system that generates videos from text by MichaelCollins on Thursday September 29, 2022 @11:23AM (#33021968)
The first Jurassic Park holds up unusually well. They used CGI for things they couldn't convincingly do any other way, and used practical effects for almost everything else. And for the most part they hid the CGI well, they took a lot of care to mix it up with close ups of practical effects, the way they framed the shots to direct audience attention, etc.

I think in a lot of modern stuff they go too far. They now use CGI for things that could easily be practical effects, but they go with CGI because it's simply cheaper or because they want to A/B test different colors of wall paneling behind a character in post production, or who knows what. The end result is apparently good enough to make a billion dollars, but I can't stand it. Movies don't feel authentic anymore. It's hard to describe rationally, but the word 'soulless' sums up how movies made the new way make me feel. Even the scenes which are wholly practical/real get degraded; the excessive CGI and compositing used in the rest of the movie cast a miasma of unrealness across the entire movie.

https://64.media.tumblr.com/248d25e2185a58bf827d329490480fb9...

Re:Make-A-Video: AI system that generates videos from text by mikepurvis on Thursday September 29, 2022 @05:05PM (#33027205)
Absolutely.

I remember listening to the DVD commentary track for X2 (from 2003), and loving that scene near the beginning where Nightcrawler is hiding out in a cathedral but losing control of his teleportation powers because he's being remotely controlled/possessed by the villain of the story. Bryan Singer talks on the track about how great it was to fling the character (can't remember if it it was actually Alan Cumming or a stuntman) from the rafters, and how much better and more weighty it looked to have a "real" shot and just have to remove wires/harnessing vs filming it as a blank canvas and having to insert the character digitally.

Re:Make-A-Video: AI system that generates videos from text by whoooooo123 on Thursday September 29, 2022 @12:14PM (#33022735)
I can't stand it either. CGI excess has ruined movies.

This video does a good job of explaining why, I think: https://www.youtube.com/watch?v=DY-zg8Oo8p4

Re:Make-A-Video: AI system that generates videos from text by mikepurvis on Thursday September 29, 2022 @05:18PM (#33027347)
Haha, why did I know this was going to be a Critical Drinker video.

For real though, I think the best CGI has always been when it's a light touch, providing only slight enhancements to an otherwise mostly-practical scene. And that's also when it's most invisible— so while superhero movies are obvious CGI-fests and can be clearly said to have been "ruined" by it, I think the most interesting modern CGI use is in lower-budget productions like TV shows, enabling the insertion of fantasy- or period-themed backdrops that would never be possible if you had to actually come up with all those props and extras in real life.

And I think to the extent that this is already happening, it's a lot harder to track because it's so much more subtle than the bombastic in-your-face effects of a Marvel movie showdown.

It would be interesting to see a revival of a show like Drive [0] as that was pretty ambitious and expensive for its time, but might be a lot more possible to do now.

[0]: https://en.wikipedia.org/wiki/Drive_(2007_TV_series)

Re:Make-A-Video: AI system that generates videos from text by vsareto on Thursday September 29, 2022 @11:11AM (#33021803)
Most of the AI generated images are good enough to be first drafts or sketches, and with some editing, they can be coherent images. This makes AI generators a good tool. Not sure the same is true for video though. It feels like you'll need reliable still image creation before you can get to a video generator that's useful since editing video takes a bit more than a still image.
Re:Make-A-Video: AI system that generates videos from text by mortenjorck on Thursday September 29, 2022 @11:54AM (#33022427)
The key to predicting technological advancement, that is so often misunderstood by technologists, is the S-curve.

Laypeople tend to think everything is linear, technologists tend to think everything is exponential; more often than not, reality tends to be sigmoidal. Technologies have exponential takeoffs followed by logarithmic plateaus. We're clearly well into the exponential phase of deep-learning ML, but it's only a matter of time before this approach hits its logarithmic phase.

Of course, the hard part of sigmoidal prediction is determining where in the curve we are. Does the current paradigm have an even steeper part ahead of it? Maybe. And yet, we could just as easily be right in the middle of the function, with a leveling off coming as the state of the art gives way to incremental improvements.

Re:Make-A-Video: AI system that generates videos from text by dgritsko on Thursday September 29, 2022 @10:19AM (#33020991)
I agree, it feels like this technology is progressing by leaps and bounds almost by the day.
Re:Make-A-Video: AI system that generates videos from text by paulgb on Thursday September 29, 2022 @10:26AM (#33021089)
Case in point, just compare the results of Stable Diffusion / Dall-E / midjourney to papers from 5-10 years ago (here's a random one I found http://proceedings.mlr.press/v48/reed16.pdf). Remarkable progress in that time, and as I understand it, a good amount (but certainly not all) of the improvement comes “free” from being able to scale up to more weights.
Re:Make-A-Video: AI system that generates videos from text by antegamisou on Thursday September 29, 2022 @10:23AM (#33021054)
> A lot of people saying it's over for traditional movie-making - lmao.

The groups supporting such absurd claims largely boil down to:

    - Money-driven researchers in the applied AI field. Those are the people that spam popular ML conferences with barely novel contributions other than some minor tweaks on code from their previous papers. 

    - People unable to critically think and evaluate the significant limitations of SOTA methods occasionally marketed as AGI breakthroughs. 
Last is virtually the entire HN userbase and the one that needs to be taken the least serious.

The first category is much more troubling however, since they can significantly influence research directions due to the broken citation system in academia (more citations --> higher quality contributions).

Re:Make-A-Video: AI system that generates videos from text by anigbrowl on Thursday September 29, 2022 @02:13PM (#33024887)
On your second point, a few years ago the SOTA was Google's Deep Dream.

I agree with your deeper criticism, though preferential attachment/ranking is very much How Humans Do Things. You could do a much improved citation system by expanding the time dimension and looking at papers that were unpopular at first and then attracted wide interest later.

Of course, academics also have a tendency to over-cite (because they don't want to be rejected for inadequate literature review), so there are incentives to cite a bunch of research whose premises or conclusions you hope to overturn.

Re:Make-A-Video: AI system that generates videos from text by bm-rf on Thursday September 29, 2022 @10:28AM (#33021150)
CogVideo was released only 4 months ago, you can see some samples on this page https://models.aminer.cn/cogvideo/. Statistical models can be used more efficiently (see KerasCV, https://news.ycombinator.com/item?id=33005585), and computing power can increase.

I don't think AI will be able to create a movie anytime soon, but I think it will become "good enough" to serve as inspiration for creatives, or to replace simple stock footage (Much like SD and DALLE-2 is now).

Re:Make-A-Video: AI system that generates videos from text by iamsaitam on Thursday September 29, 2022 @10:49AM (#33021483)
It will definitely improve from here onward. On the other hand, I agree that it's a ridiculous thing to say that traditional movie-making is over. There's so much involved in making a movie, so much happenstance from the actors performances in a specific environment. You will never be able to get this in AI.. you might get some sort of mimicry, but the comparison is futile.. a movie isn't just a sequence of changing pixels.
Re:Make-A-Video: AI system that generates videos from text by wokwokwok on Thursday September 29, 2022 @11:05AM (#33021722)
I think it’s more complex than that.

What you’re going to see is a race to the bottom, the same as with claymation films and 3d.

It will suddenly require a lot less (expensive, highly trained) people to make the same films.

You’ll still need expensive highly trained people to do it, but with different skills and a lot less of them can do a lot more a lot more quickly.

…and that means that some studios will make bad, low grade films… and some studios will make amazing films using hybrid techniques (like 3d printed faces for claymation).

…but overall, the people funding movies will expect to get more for less, and that will mean a downsizing of the number of people employed currently in certain roles.

Traditional film making over? Hm… it’s complicated. Is it over if the entire industry changes, but people are still making films? Or is that just the “traditional” part of it which is over?

It’s definitely going to change the industry.

People will still definitely film things.

…but, I wager, less people will be doing highly payed skilled manual work, which will replaced by a few people doing a different type of AI assisted work.

…and we’ll see some really amazing indy films, of small highly technical teams producing content with very little physical filming.

Re:Make-A-Video: AI system that generates videos from text by anigbrowl on Thursday September 29, 2022 @02:15PM (#33024907)
Accurate, no notes.
Re:Make-A-Video: AI system that generates videos from text by uup on Thursday September 29, 2022 @10:27AM (#33021111)
> I look at these and see nothing but uncanny valley artifacts

Ok? It's a nascent technology. Look at the original DALL-E blog post from last year [1]. Now compare it to DALL-E 2 and Stable Diffusion.

[1]: https://openai.com/blog/dall-e/

Re:Make-A-Video: AI system that generates videos from text by mrandish on Thursday September 29, 2022 @10:51AM (#33021526)
> A lot of people saying it's over for traditional movie-making

While these early versions are primitive, as a traditional filmmaker I think in a couple years these technologies will creatively empower visual storytellers in exciting new ways. The key will be developing interfaces which allow us to engage, direct and constrain the AI to help us achieve specific goals.

Re:Make-A-Video: AI system that generates videos from text by bckr on Thursday September 29, 2022 @10:56AM (#33021609)
> these technologies will creatively empower visual storytellers in exciting new ways

Exactly. Every step forward in creative technology is additional leverage for the artist with a vision.

Re:Make-A-Video: AI system that generates videos from text by bottlepalm on Thursday September 29, 2022 @10:20AM (#33021000)
It’s been like a month since people were saying this tech would be too hard to apply to video.
Re:Make-A-Video: AI system that generates videos from text by Xelynega on Thursday September 29, 2022 @10:26AM (#33021096)
The leap from "generating a frame" to "generating a video" is not as big as the leap from where we are now to a 'perfect' image synthesis engine(which would be required to get rid of artifacts).

The much more likely scenario IMO is that people get used to the artifacts and notice them less.

Re:Make-A-Video: AI system that generates videos from text by bottlepalm on Thursday September 29, 2022 @10:55AM (#33021587)
There is something with the AI needing to have a ‘sense’ for the world the scene exists in so longer videos can be created that are coherent. Currently we’ve only seen long videos that have no consistency and jump around a lot like an acid trip.
Re:Make-A-Video: AI system that generates videos from text by anigbrowl on Thursday September 29, 2022 @02:27PM (#33025082)
Yep, we're in a real hype cycle.

A good path forward is to fuse these image-element compositing tools with some of the 3d scene inference ones. So you start out with 'giant fish riding in a golf cart, using its tail to steer', then give that as ground truth to a modeling tool that figures out a fish and a wheeled vehicle well enough to reference some canonical examples with detailed shape and structure, the idea of weight etc. Then you build a new model with those and do some physics simulation (while maintaining a localized constraint of unreality that allows the fish to somehow stay in the seat of the golf cart).

Re:Make-A-Video: AI system that generates videos from text by liuliu on Thursday September 29, 2022 @10:45AM (#33021431)
Their video training set is still small (10M + 10M). A lot of the interpolation artifacts seems come from the model haven't acquired enough real-world understanding of "natural" movements (looking at the horse running and the sail-boat examples). I suspect scaling this up to 10x would have much less artifacts.

Reading the paper, it seems to be the "right" approach (separating temporal / spatial for both convolution and attention). Thus, I am optimistic what remains is to scale it up.

Re:Make-A-Video: AI system that generates videos from text by MichaelCollins on Thursday September 29, 2022 @11:37AM (#33022179)
The horse leg motion in these examples is really poor. But to be fair, horse legs are very difficult for human animators too, generally necessitating the use of reference photos to get it right. The first time somebody got a series of photographs showing a horse in motion caused quite a stir: https://en.wikipedia.org/wiki/The_Horse_in_Motion
Re:Make-A-Video: AI system that generates videos from text by britzkopf on Thursday September 29, 2022 @11:02AM (#33021680)
I'm not sure I'd agree with the "it won't improve much from here" sentiment, but I am a little confused by the sharp disagreements with this comment. They seem to contain a tacit assumption that the complexity curve for this problem is smoothe, but I could easily imagine a very sharp elbow in that curve, making any progress come to a seemingly grinding halt.
Re:Make-A-Video: AI system that generates videos from text by kuu on Thursday September 29, 2022 @10:22AM (#33021035)
We already have films that use "similar" technology (for example, adding Leia in the new SW movies), and this is just another step, another improvement into the direction of autogenerated videos. Are we far from generating complete movies? Sure! Are we progressing? Yes too!
Re:Make-A-Video: AI system that generates videos from text by ZetaZero on Thursday September 29, 2022 @10:34AM (#33021244)
Right and wrong.

It's not over for traditional moving-making. It would be decades before the software and hardware could surpass. But it will improve tremendously, just like computers do for nearly everything.

Re:Make-A-Video: AI system that generates videos from text by j45 on Thursday September 29, 2022 @10:39AM (#33021336)
There's always the next curve of improvement by this tech or something else.

It's just the first version and seeing Stable Diffusion come out while openai's tools were coming out were something to remember and think about.

Re:Make-A-Video: AI system that generates videos from text by fullshark on Thursday September 29, 2022 @11:17AM (#33021896)
The only way it will be over is that tiktok/instagram will attract more eyeballs and hurt their bottom line. Ultimately this is for amateurs to create really short form content for the foreseeable future.
Re:Make-A-Video: AI system that generates videos from text by hackerlight on Thursday September 29, 2022 @10:33AM (#33021242)
This tech went from nothing to beating a human artist in an art competition in a few years, and yet you say "I don't think it will improve much". So, I disagree.
Re:Make-A-Video: AI system that generates videos from text by appletrotter on Thursday September 29, 2022 @10:16AM (#33020942)
> I don't think it will improve much from here.

Any more specific reason why you feel this way? Curious

Re:Make-A-Video: AI system that generates videos from text by sterlind on Thursday September 29, 2022 @11:53AM (#33022417)
self-driving cars need safety guarantees. video's allowed to occasionally be crappy, it's not going to kill anyone.

I thought the same way you did about speech-to-text and image search, back in the day. boy was I wrong.

Re:Make-A-Video: AI system that generates videos from text by permo-w on Thursday September 29, 2022 @01:12PM (#33023889)
> A lot of people saying it's over for traditional movie-making

who is saying that?

Re:Make-A-Video: AI system that generates videos from text by anigbrowl on Thursday September 29, 2022 @02:32PM (#33025137)
Me, film-industry vet.

You'll see an excellent mostly- or all-AI feature within 5-10 years. There will be terrible ones before that, maybe 2-3 years. The first really good one will be enjoyed on its own merits, ad the artificiality of it will come to light after it has gained popularity.

Re:Make-A-Video: AI system that generates videos from text by permo-w on Thursday September 29, 2022 @03:20PM (#33025732)
I would imagine you’ll see one or a few that are barely AI-produced as soon as marketing teams get wind of this. “come and see the first ever AI-produced* blockbuster! ⠀ ⠀ ⠀

*produced, assisted, or discussed on set one time during lunchbreak”

Re:Make-A-Video: AI system that generates videos from text by wslh on Thursday September 29, 2022 @10:29AM (#33021168)
If this technology (or its future iterations) is not enough you can always combine this with your movie cuts. This lowers the resources/bar needs for creative people and obviously also for the uncreative ones.
Re:Make-A-Video: AI system that generates videos from text by squidbeak on Thursday September 29, 2022 @10:41AM (#33021368)
This is a good example of the most abject type of Luddism: telling yourself that a fast-developing tech you happen to dislike is already at or near the limit of its potential and that defects you notice now will be there forever.
Re:Make-A-Video: AI system that generates videos from text by dang on Thursday September 29, 2022 @10:57AM (#33021612)
Please make your substantive points without name-calling.

https://news.ycombinator.com/newsguidelines.html

Re:Make-A-Video: AI system that generates videos from text by anigbrowl on Thursday September 29, 2022 @02:35PM (#33025172)
It's not name calling to make a critique of an argument. Luddism is a real term that describes a particular philosophical tendency that places itself in opposition to what it views as a naive technophilia.
Re:Make-A-Video: AI system that generates videos from text by [deleted] on Friday September 30, 2022 @10:07AM (#33035017)
Re:Make-A-Video: AI system that generates videos from text by MichaelCollins on Thursday September 29, 2022 @10:57AM (#33021627)
The Luddites were a labor rights movement reacting to the new shifting power dynamic of centralized production owned by a few, in an era with virtually no safety and worker rights regulations. How would you feel if your career evaporated and you were forced to choose between starvation and sending your children to work in a textile factory where they get maimed by the machines they're told to crawl into and repair? You probably wouldn't find much comfort in the popular retort of "but shirts are really cheap now!"

That Luddites have been successfully maligned as irrationally anti-technology crazy people is a propaganda victory by industrialist factory owners and their friends, the newspaper men.

Re:Make-A-Video: AI system that generates videos from text by mitchdoogle on Thursday September 29, 2022 @11:43AM (#33022263)
Whatever their motivation, their ire is misguided and selfish. Is the world supposed to just sit and around and never innovate or try to become more productive? So people can have a job doing work better done by machines? I don't think so. The Luddites and any such analogs today are focused on the wrong thing - they should not attack new technology or the companies and individuals using them, but rather campaign for better social safety net from the government.
Re:Make-A-Video: AI system that generates videos from text by MichaelCollins on Thursday September 29, 2022 @11:53AM (#33022418)
Maybe new technology should be restricted by default until regulation can catch up with it. I'm not sure, but the 'cheap shirts benefit everybody too much to pump the brakes' argument for unfettered innovation leaves me deeply unimpressed.

IMHO the best argument for unfettered innovation is the impossibility of slowing innovation globally; it can be slowed in one country, but that country can't force all the rest to get with that program and will eventually be overtaken by technologically superior foes.

Re:Make-A-Video: AI system that generates videos from text by anigbrowl on Thursday September 29, 2022 @02:38PM (#33025210)
A valid point, but restrictions are often used to build or fortify fiefdoms.
Re:Make-A-Video: AI system that generates videos from text by trention on Thursday September 29, 2022 @10:46AM (#33021435)
You will find plenty of comments in this very thread that are the most abject type of !Luddism.
Re:Make-A-Video: AI system that generates videos from text by terafo on Thursday September 29, 2022 @11:21AM (#33021943)
You are hilariously wrong if you think it will not improve. Combination of prompt engineering, better and bigger datasets, better architectures, bigger models, artifact-fixing tools and sheer human creativity will seriously challenge traditional movie-making within two years.