News for hackers. Stuff that matters.
Re:Sakuga-42M Dataset: Scaling Up Cartoon Research by anonylizard on Friday May 17, 2024 @11:03AM (#40390708)
Wow, this looks like the beginning of the end for in-between animation.

A large chunk of an anime's budget goes to in-betweening. Essentially human interpolation of graphics between two key frames (Its usually like 6-12 inbetweens per key frame). People hate this job, and it is highly unproductive, so generally outsourced by Japan to other countries.

Western animation decided to abandon it altogether, and move first to flash, then to 3d animation. But in retrospect that was a mistake, as it lost so much of the creative flexibility of 2d animation. Anime today is substantially bigger than western animation as a result. Crunchyroll has 13 million subscribers.

AI will solve the problems with 2d animation. Something like SORA fine-tuned on anime-keyframe data like this. Can probably easily solve in-betweening. Then the 2d animation workflow will dominate 3d. Its so much easier to just draw a beginning and end key-frame, then have the AI fill it in. Than to model and rig and render a entire scene.

Re:Sakuga-42M Dataset: Scaling Up Cartoon Research by CactusOnFire on Friday May 17, 2024 @11:09AM (#40390759)
This is one job I am actually hopeful will be fully automated. I've heard before that the people doing this 'tween' frames make less than minimum wage on contract.

That being said, I'm sure this is going to take a while to fully replace the manual efforts. There will probably be awkward phase in between the outset and the perfect modelling efforts, and I'm sure lower budget shows (or ones looking to cut corners) will be the early adopters.

Re:Sakuga-42M Dataset: Scaling Up Cartoon Research by anonylizard on Friday May 17, 2024 @11:18AM (#40390849)
Anime already cuts corners by using cheap 3d models in place of 2d hand drawn objects.

The more rigid the object, the better this works. 3d cars look better than 2d cars, even in an otherwise 2d show. Mechs look a bit worse. And human characters look horrible.

Yet anime studios still do it. Including for critical highlight scenes like dance scenes (Check out Love live dance scens), because it is so, so hard to draw humans dancing.

So if anime studios are willing to do something, that looks obviously bad, as a widespread practice. There'll be 0 barriers to AI inbetweening adaptation, which would likely look BETTER than human inbetweening within a year of release.

AI anime art has already wiped out the lower-end of patreon artists, and is heavily impacting the mid-tier. Because AI has gotten more technically proficient than the average mid-tier artist. Pretty much only the higher-end can hold their heads above water. Or they have to transition to drawing comics with storylines, instead of just simple images.

Re:Sakuga-42M Dataset: Scaling Up Cartoon Research by GaggiX on Friday May 17, 2024 @11:59AM (#40391273)
>3d cars look better than 2d cars, even in an otherwise 2d show.

This is quite debatable, if you notice that the car is a 3D object, then something is already wrong.

Re:Sakuga-42M Dataset: Scaling Up Cartoon Research by Hamuko on Friday May 17, 2024 @12:15PM (#40391446)
Usually you can tell by the fact that nothing is wrong, since the 3D model is very consistent and on model. A hand-drawn car is usually not that.
Re:Sakuga-42M Dataset: Scaling Up Cartoon Research by GaggiX on Friday May 17, 2024 @12:32PM (#40391636)
The same could be said for a 3D human character, it's very consistent because it's a 3D model but it's horrible to look at.
Re:Sakuga-42M Dataset: Scaling Up Cartoon Research by speedgoose on Friday May 17, 2024 @01:27PM (#40392269)
Do you notice when Bluey characters are animated from 3D or 2D? The software they use allows to do 2D drawings from 3D animated models.

https://www.celaction.com/en/celaction2d/

Re:Sakuga-42M Dataset: Scaling Up Cartoon Research by GaggiX on Friday May 17, 2024 @02:06PM (#40392614)
I have never seen Bluey but from the software you linked it is clear that it looks 2D because of how inconsistent the character looks at different angles, for example when you rotate the character the mouth changes position, the hands jump from one sprite to another, it's cel shading with a lot of 2D element on top, it works with simple animation but for anime I'm not optimistic.
Re:Sakuga-42M Dataset: Scaling Up Cartoon Research by mlyle on Friday May 17, 2024 @12:44PM (#40391757)
His point, I believe: artistically interpreting the motion and shape of humans or objects with larger moving parts makes animation look more on-style.

But for "boring" rigid objects, there's less of this advantage; hence, the consistency benefits often are more important.

Re:Sakuga-42M Dataset: Scaling Up Cartoon Research by numpad0 on Friday May 17, 2024 @12:59PM (#40391923)
unless it's Miyazaki films - in which case most humans are intended to be lifeless rigid objects and every machines are to be lifelike animations (/s)
Re:Sakuga-42M Dataset: Scaling Up Cartoon Research by Hamuko on Friday May 17, 2024 @03:02PM (#40393201)
I've seen 3D animation where the people are still quite fluid and not awful to look at. Not as good as 2D animation but still pretty good and more than watchable.

https://www.youtube.com/watch?v=OO9zNw_uHg4

https://www.youtube.com/watch?v=eCc4md8Cuy8

Re:Sakuga-42M Dataset: Scaling Up Cartoon Research by redwall_hp on Friday May 17, 2024 @03:02PM (#40393207)
It was a major consideration for Initial D, which is pretty much the definitive car anime. Animating movement in tandem with dynamic camera movement is very difficult (also why shows like Love Live use 3DCGI for dance scenes, and why Disney was using 3D elements in films like Beauty and the Beast) and modeling accurate vehicle physics in hand drawn animation is also difficult. It simply wouldn't have been viable without 3D animation.

https://youtube.com/watch?v=YDqKsQu9el4

Re:Sakuga-42M Dataset: Scaling Up Cartoon Research by TJSomething on Friday May 17, 2024 @02:22PM (#40392786)
A lot of the dancing stuff is about the ability to spin the camera around a moving subject to the music, which is quite difficult otherwise.

There's a lot of impressive work in 3D animation that looks quite good. Outside of Bandai Namco's work on idol anime, Studio Orange has made some of the best looking 3D modeled anime lately and a few other studios have been getting into it. I'm more familiar with video game animation, where Arcsys Works has made great strides too, by using animation on threes, manual tweening, stretch and squish bones, and carefully UV mapped textures for crisp color boundaries.

Re:Sakuga-42M Dataset: Scaling Up Cartoon Research by Keyframe on Friday May 17, 2024 @04:59PM (#40394165)
Friend, but anime was never about quality of animation. In fact, it was a prime example how to cut corners to get to animations. That was always the case. It doesn't reflect on the quality of character designs, environments, storytelling, camera action, directing, etc. Motion was not one of them; Never was. Anime is the first place I'd expect to see new ground breaking, just like it was with all the tools from 90's onwards (Toonz, anyone?).
Re:Sakuga-42M Dataset: Scaling Up Cartoon Research by astrange on Saturday May 18, 2024 @12:46AM (#40396505)
Anime is much better at action animation than anyone else, simply because noone else remembers how to do it because they've stopped trying.

I'm not sure if I'll be able to find this, but there's an episode of Steven Universe with an extended reference to a scene from Kill la Kill of a transformation sequence.

It looks maybe 1% as good; not only that, but the character turns into a pure white silhouette for the entire transformation because it doesn't have a character design suitable for being transformed. (Instead it has one designed to make the animators' lives easier.)

Re:Sakuga-42M Dataset: Scaling Up Cartoon Research by BizarroLand on Friday May 17, 2024 @07:34PM (#40395167)
Most of that could be dealt with via proper compositing of the shots and managing the layers/lights. When it's good, you only recognize the 3d computer drawn effect because it's so good that you realize no human could have ever done this.

When it's bad, you recognize that it's janky crap tier 3d animation from a company that either didn't care or was put under such a tight timeline that they simply couldn't care.

Re:Sakuga-42M Dataset: Scaling Up Cartoon Research by guyomes on Friday May 17, 2024 @04:20PM (#40393831)
> I've heard before that the people doing this 'tween' frames make less than minimum wage on contract.

An employee of an animation company describes in a comic book his experience on working with people drawing the in-between frames. They were paid literally with rice bags [1].

[1]: https://en.wikipedia.org/wiki/Pyongyang:_A_Journey_in_North_...

Re:Sakuga-42M Dataset: Scaling Up Cartoon Research by mcphage on Friday May 17, 2024 @11:40AM (#40391079)
I wondered about that—they stated that they processed videos to pull out keyframes. It makes me wonder how much that will or won't help with in-betweening.
Re:Sakuga-42M Dataset: Scaling Up Cartoon Research by astrange on Saturday May 18, 2024 @12:51AM (#40396519)
Keyframes in digital video means a different thing from keyframes in animation; they could be related but usually wouldn't be. You can hide animation keyframes with a number of techniques, for instance by having different "keyframes" for different parts of a scene so the video doesn't show them all at the same instant.

(Rough generalization, but anime is more likely to do this with foreground vs background elements, while Western animation would do this with different characters. In anime the keyframe artists tend to draw every character in the frame, and put more of a personal touch on them, so it's easier to see their individual styles.)

Since this is a "sakuga" dataset, Mitsuo Iso is one of the most famous examples of really natural looking movement here.

https://www.youtube.com/watch?v=NTMJ8dGFUkc

When they're bad at it you get this effect where characters constantly seem to "settle" into a keyframe pose that looks realistic but too static, and then immediately go back into inbetweening that moves a lot but isn't physically possible. I feel like B-grade Disney stuff is the worst here but don't have an example on hand.

Re:Sakuga-42M Dataset: Scaling Up Cartoon Research by zbirkenbuel on Saturday May 18, 2024 @12:42AM (#40396495)
Can't have a conversation about inbetween frames without sharing this[1] Noodle video. Now granted the interpolation there is pretty rigid, and the goal of this dataset will be to train an LLM replicate hand tweening, but I can't help but feel there's going to be a massive uncanny valley.

[1]https://youtu.be/_KRb_qV9P4g?si=vtDvfLU6XYz5QwWV&t=355