ICLR 2024 Spotlight: AnimateDiff Brings Video Generation to Existing Image Diffusion Models

AnimateDiff, a plug-and-play motion module presented at ICLR 2024 as a spotlight paper, enables any community text-to-image model to generate short animations without model-specific training. The framework, authored by researchers from Alibaba, Tsinghua University, and UC Berkeley, works by appending a lightweight motion layer to frozen diffusion models, learning transferable motion patterns from real-world video data.