Alibaba Introduces Open-Source Model for Video Creation and Editing

Alibaba has unveiled Wan 2.1-VACE (Video All-in-one Creation and Editing), its latest open-source model for video creation and editing. This innovative tool combines multiple video processing functions into a single model, to streamline the video creation process, boosting efficiency and productivity.

As part of Alibaba's video generation large model - the Wan2.1 series - VACE is the first open-source model in the industry to provide a unified solution for various video generation and editing tasks.

Wan2.1-VACE supports video generation with multi-modal inputs spanning text, image, and video while offering creators comprehensive video editing capabilities. These editing features include referencing images or frames, video repainting, modifying selected areas of the video and spatio-temporal extension, all of which enable the flexible combination of various tasks to enhance creativity.

With this advanced tool, users can generate video containing specific interacting subjects based on image samples and bring static images to life by adding natural movement effects. They can also enjoy advanced video repainting functions such as pose transfer, motion control, depth control, and recolorization.

The model also supports adding, modification or deletion to selective specific areas of a video without affecting the surroundings. It also allows for the extension of video boundaries while intelligently filling in content to enrich the visual experience.

As an all-in-one AI model, Wan2.1-VACE delivers unparalleled versatility, enabling users to seamlessly combine multiple functions and unlock innovative potential. Users can turn a static image into video while controlling the movement of objects by specifying the motion trajectory. They can seamlessly replace characters or objects with specified references, animate referenced characters, control poses, and expand a vertical image horizontally to create a horizontal video while adding new elements through referencing.

Innovative Technologies

Wan2.1-VACE leverages several innovative technologies, to take into account the needs of different video editing tasks during construction and design. Its unified interface, called Video Condition Unit (VCU), supports unified processing of multimodal inputs such as text, images, video, and masks.

The model employs a Context Adapter structure that injects various task concepts using formalized representations of temporal and spatial dimensions. This innovative design enables it to flexibly manage a wide range of video synthesis tasks.

Thanks to advancements in model architecture, Wan2.1-VACE can be widely applied in the rapid production of social media short videos, content creation for advertising and marketing, post-production and special effects processing in film and television, and for educational training video generation.

Training video foundation models requires immense computing resources and vast amounts of high-quality training data. Open access helps lower the barrier for more businesses to leverage AI, enabling them to create high-quality visual content tailored to their needs, quickly and cost-effectively.

Alibaba is open-sourcing the Wan2.1-VACE model in two versions; a 14-billion(B)-parameter and a 1.3-billion(B)-parameter. The models are available to download for free on Hugging Face and GitHub, as well as Alibaba Cloud's open-source community, ModelScope.

As one of the earliest major global tech companies to open source its self-developed large-scale AI models, Alibaba open sourced four Wan2.1 models in February 2025 and, last month, a video generation model that supports video creation with start and end frames. To date, the models have attracted over 3.3 million downloads on Hugging Face and ModelScope.

• ข่าวinnovative
• ข่าวo:member
• ข่าวo:loc
• ข่าวen
• ข่าวo:it