Color-Diffusion: using diffusion models to colorize black and white images

Re:Color-Diffusion: using diffusion models to colorize black... by NBJack on Thursday August 03, 2023 @05:17PM (#36992082)

I think the bigger question is would it be stable enough. Many SD like models struggle with consistency across multiple images (i.e. frames) even when content doesn't change much. Would he a cool problem to see tackled.

[ Parent | on HN ]

Re:Color-Diffusion: using diffusion models to colorize black... by erwannmillon on Thursday August 03, 2023 @06:12PM (#36992737)

temporal coherence is def an issue with these types of models, though I haven't tested it out with ColorDiffusion. Assuming you're not doing anything autoregressive (from frame to frame) to do temporal coherence, you can also parallelize the colorization of each frame, which would affect cost.

Tbh most cost effective would be a conditional GAN though

[ Parent | on HN ]

Re:Color-Diffusion: using diffusion models to colorize black... by lajamerr on Thursday August 03, 2023 @06:31PM (#36992966)

Change up the model. That allows it to see previous frames and 1-2 future frames.

Then train the model on movies that are color and then turn them black and white.

That way you can train temporal coherence.

[ Parent | on HN ]