Running out of 12GB ？

[juntaosun]

The configuration required is too high.

Because the GPU memory is exhausted, the speed is very slow!

The current video generation scheme has too many shortcomings. Almost all frames are generated at once, causing the GPU memory to be exhausted.

If in the future, a frame can be generated, then this frame is saved as an image png sequence, and then all the frame image sequences are merged. This should save a lot of GPU memory.

[feifeiobama]

We have addressed some issues #76, please update the codebase and see how it goes. We also have a big upcoming improvement #75 that should allow use with less than 8GB of GPU memory. Stay tuned for further efficiency improvements.

[juntaosun]

I tried many image-to-video conversions, but the results were not good.
The picture became uncontrollable after 1 second.

Looking at the actual examples, the effect needs to be improved.

test.mp4

[juntaosun]

I suggest to remove the "Image-to-Video" function.
The generated faces and hands are the same as in the "Thriller" movie.

[feifeiobama]

I suggest to remove the "Image-to-Video" function. The generated faces and hands are the same as in the "Thriller" movie.

Thank you for your valuable feedback. We acknowledge these issues (human deformation due to SD3 weight initialization; quality degradation due to insufficient autoregressive training) and are working on a new model checkpoint.