News for hackers. Stuff that matters.
Re:Video to Video with Stable Diffusion (Step-by-Step) – Sta... by colordrops on Monday June 12, 2023 @02:32AM (#36289203)
Did you scroll down and see the different methods used, specifically method 5? Not perfect, but getting pretty close.
Re:Video to Video with Stable Diffusion (Step-by-Step) – Sta... by justinclift on Monday June 12, 2023 @02:50AM (#36289329)
It's a lot better than earlier work, but still not really at the level it needs to get to.
Re:Video to Video with Stable Diffusion (Step-by-Step) – Sta... by moonchrome on Monday June 12, 2023 @04:33AM (#36290162)
The last one looks the creepiest - like a dystopian matrix glitch. Could be a cool effect to illustrate that.
Re:Video to Video with Stable Diffusion (Step-by-Step) – Sta... by rpastuszak on Monday June 12, 2023 @06:24AM (#36291175)
The light reflection on the shoulder doesn't make much sense. I think this is less pronounced when soft light is involved. I wonder if sharp light and reflections will take longer to be handled properly. Every frame on the shoulder looks ok on its own, but when animated it feels like it was taken in a different environment than the rest.

I suspect depth maps (SD2 already supports them) could be used to achieve that in the future.

I wonder if a diffusion model could accept an "onion skin" noise, so the transitions between frames would be less jarring. Can someone with more knowledge than me explain what's the most promising approach here?

Re:Video to Video with Stable Diffusion (Step-by-Step) – Sta... by bondarchuk on Monday June 12, 2023 @06:42AM (#36291360)
>The light reflection on the shoulder doesn't make much sense.

People will really grasp at anything to find something to critique in generative AI, huh?

Re:Video to Video with Stable Diffusion (Step-by-Step) – Sta... by SirMaster on Monday June 12, 2023 @09:13AM (#36292888)
Wait, should we not critique?

OK i'll fix that.

It's perfect. Stop all work and development on it now guys, it's perfect. It has no flaws and makes no mistakes. Amazing I guess.

Re:Video to Video with Stable Diffusion (Step-by-Step) – Sta... by rpastuszak on Monday June 12, 2023 @09:29AM (#36293104)
I think @bondarchuk took my comment out of context, quoting just the first line. No big deal, it's just hard to respond to that.

I don't even see an issue with that particular artefact, I think it's an interesting problem from a technical pov, hence literally every other part of my comment.

Re:Video to Video with Stable Diffusion (Step-by-Step) – Sta... by bondarchuk on Monday June 12, 2023 @09:58AM (#36293505)
It just seemed like a bit of a nitpick in the context of the huge temporal consistency issues/fixes, of course you're all free to discuss whatever you want and I agree it's somewhat interesting.
Re:Video to Video with Stable Diffusion (Step-by-Step) – Sta... by rpastuszak on Wednesday June 14, 2023 @02:42AM (#36322283)
Haha, nope, not my intention at all. Although I understand the expectation give the average style of discourse on this site.
Re:Video to Video with Stable Diffusion (Step-by-Step) – Sta... by llm_nerd on Monday June 12, 2023 @08:15AM (#36292224)
The last one is largely the original video with a cartoon filter and basically anime facetune. It is educational and interesting, but it looks the best because it's actually doing the least, though even then you get the temporal artifacts on the hair and so on.

It is improving super rapidly and it will be a pretty seamless soon enough in all likelihood. We are in the intermediate stage right now, and the results are going to be incredible.