News for hackers. Stuff that matters.

andrusza2

Re:Comixify: Transform video into a comics on Wednesday December 12, 2018 @03:50PM (#18667634)
Hi, I am one of co-authors of the paper.

Thank you for your interest in our demo.

Regarding the points you raise in your comment:

1. Layout

You are right that we use static layout for all comics we generate. Changing that has a high priority on our todo list, we are currently working on more dynamic layout generation that takes into account what we know about the story and which frames are more important than others.

2. Style transfer

The method we use gives the most pleasant results of all obtained with various style transfer methods applied to comics - see our previous work on comparison between them here: https://arxiv.org/abs/1809.01726.

3. Story sense

(>probably not doing anything to detect/convey a better sense of the story<) We use Video Summarization model in our pipeline, however as mentioned above, we use static layouts and therefore this is not always apparent.

Overall, this demo is only a starting point. We are working on adding automatic speech recognition and text generation to introduce dialogs into our comics. And we have many other improvements planned. Stay tuned and if you have any other comments feel free to send us an email to any of the addresses from the paper: [email protected], [email protected], [email protected], [email protected].