sidneyprimas

After much user feedback, we removed the Infinity watermark from the generated videos. Thanks for the feedback. Enjoy!

Interesting! Are you saying you would first want tools to really design your character, and only after start making videos with the character you built? That's interesting.

HeyGen (and our V1 model) literally uses the user on-boarding video in the final output. See here for a demonstration of this (https://toinfinityai.github.io/v2-launch-page/#comparisons). We are not talking about that in this thread. We are trying to solve a quirk of our Diffusion Transformer model (V2 model).

Our V2 model is trained on specific durations of audio (2s, 5s, 10s, etc) as input. So, if give the model a 7s audio clip during inference, it will generate lower quality videos than at 5s or 10s. So, instead, we buffer the audio to the nearest training bucket (10s in this case). We have tried buffering it with a zero array, white noise and just concatenating the input audio (inverted) to the end. The drawback is that the last frame (the one at 7s) has a higher likelihood to fail. We need to solve this.

And, no shade on HeyGen. It's literally what we did before. And their videos look hyper realistic, which is great for B2B content. The drawback is you are always constrained to the hand motions and environment of the on-boarding video, which is more limiting for entertainment content.

Thank you! Just want many people to use it. And, it's super interesting to see what type of content people are making with it.

Well then. Tik Tok, and keep ticking to you too.

Nice! These are really good. I wanted them to continue telling their story.

Can you share an example of this happening? I am curious. We can get static videos if our model doesn't recognize it as a face (e.g. an Apple with a face, or sketches). Here is an example: https://toinfinityai.github.io/v2-launch-page/static/videos/...

I would be curious if you are getting this with more normal images.

Ya, it's only a matter of time until very high quality video models will be open sourced.

That's how I see it as well. Very soon, people will assume most videos are AI generated, and the burden of prove will be on people claiming videos are real. We plan to embed some kind of hash to indicate our video is AI generated, but people will be able to get around this. Google/Apple/Samsung seem to be in the best place to solve this: whenever their devices record a real video, they can generate a hash directly in HW for that video, which can be used to verify that it was actually recorded by that phone.

Also, I think it will cost around $100k to train a model at this quality level within 1-2 years. And, will only go down from there. So, the genie is out of the bag.

This makes me so happy. Thanks for reporting back! Goal is to reduce creepiness over time.

We are considering it. Do you have anything specific you want to use it for?

Agreed! The teeth can be problematic. The good news is we just need to train at higher resolution (right now we are at 320x320px), and that should resolve the teethe issue.

So far, we have purposely trained on low resolution to make sure we get the gross expressions / movements right. The final stage of training with be using higher resolution training data. Fingers crossed.

Ya, this is tricky. Our stance is the people should be able to make funny, parody videos with famous people.

Yes, exactly! We just wanted to make it easy to compare. We also used some inputs from other famous research papers for comparison (EMO and VASA). But all videos we show on our website/blog are our own. We don't host videos from any other model on our website.

Also, Loopy is not available yet (they just published the research paper). But you can try our model today, and see if it lives up to the examples : )

Thanks for the feedback! The good news is that the new V2 model will allow people to create their own actors very easily, and so we won't be restricted to the list. You can try that model out here: https://studio.infinity.ai/

The rest of our website still uses the V1 model. For the V1 model, we had to explicitly onboard actors (by fine-tuning our model for each new actor). So, the V1 actor list was just made based on what users were asking for. If enough users asked for an actor, then we would fine-tune a model for that actor.

And yes, the 7s limit on v1 is also a problem. V2 right now allows for 30s, and will soon allow for over a minute.

Once V2 is done training, we will get it fully integrated into the website. This is a pre-release.

We are not related to Loopy Avatar. We trained our own models. It's a coincidence that they launched yesterday.

In the AI/research community, people often try to use the same examples so that it's easier to compare performance across different models.

It's much easier to use standard CSS packages, and these come with more standard styles. Our team doesn't have much experience building websites, so we just went with the standard styles. We used TailwindCSS.