Fix/wan animate frame alignment

[zwplus]

Summary

• Align wan-animate time dimension by accounting for the single input_image frame upstream
• Remove the downstream “drop last 4 frames” split
• Update wan-animate example scripts to use 77-frame inputs

What changed

• diffsynth/pipelines/wan_video.py
  • WanVideoUnit_NoiseInitializer: add +1 time step when input_image & animate_pose_video are present
  • WanVideoUnit_InputVideoEmbedder: prepend input_image latent for wan-animate
  • remove WanVideoUnit_AnimateVideoSplit from the pipeline
• Examples:
  • examples/wanvideo/model_training/validate_lora/Wan2.2-Animate-14B.py
  • examples/wanvideo/model_training/validate_full/Wan2.2-Animate-14B.py
  • examples/wanvideo/model_inference/Wan2.2-Animate-14B.py
  • examples/wanvideo/model_inference_low_vram/Wan2.2-Animate-14B.py
  • change num_frames to 77 and slice inputs as [:77]

Rationale / Issue

Fixes the time-dimension mismatch discussed in #1027. The prior drop last 4 frames behavior is a downstream patch; it avoids shape errors but leaves conditioning and target frames misaligned. This change aligns the reference frame and latents upstream so all inputs and targets are consistent.

Related Issues

• First-Frame Color Shift in Wan 2.2 Animate LoRA Fine-Tuning #1105: Reports noticeable color shift in the first frame when training Wan 2.2 Animate LoRA. This PR aligns the reference frame and temporal dimension upstream, which may reduce first‑frame color bias. (First-Frame Color Shift in Wan 2.2 Animate LoRA Fine-Tuning #1105)
• wan2.2 animte 首帧色差 #1197: Reports global color discrepancy between the first frame and later frames. By fixing temporal alignment and removing the downstream frame‑drop patch, this PR may improve that behavior. (wan2.2 animte 首帧色差 #1197)

[gemini-code-assist]

Summary of Changes

Hello, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request addresses a critical time-dimension mismatch in the wan-animate pipeline, which previously led to misaligned conditioning and target frames. By shifting the frame alignment logic upstream and removing a compensatory downstream frame-dropping mechanism, the changes ensure consistency across all inputs. This fix is expected to improve the quality of generated videos, potentially resolving issues like color shifts and discrepancies between initial and subsequent frames.

Highlights

• Time Dimension Alignment: The core change aligns the wan-animate time dimension by accounting for the single input_image frame upstream, resolving a previous mismatch.
• Removed Downstream Frame Drop: The problematic downstream 'drop last 4 frames' split, which previously avoided shape errors but caused misalignment, has been removed.
• Updated Example Scripts: All relevant wan-animate example scripts have been updated to use 77-frame inputs, reflecting the new alignment logic.
• Pipeline Unit Modifications: The WanVideoUnit_NoiseInitializer and WanVideoUnit_InputVideoEmbedder were updated to handle the input_image and animate_pose_video for proper time dimension adjustment and latent prepending.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Changelog

• diffsynth/pipelines/wan_video.py
  • Removed WanVideoUnit_AnimateVideoSplit from the WanVideoPipeline initialization.
  • Updated WanVideoUnit_NoiseInitializer to accept input_image and animate_pose_video and conditionally increment the frame length by one for wan-animate.
  • Modified WanVideoUnit_InputVideoEmbedder to accept input_image and animate_pose_video as input parameters.
  • Implemented logic in WanVideoUnit_InputVideoEmbedder to prepend the input_image latent to input_latents when input_image and animate_pose_video are provided.
  • Deleted the WanVideoUnit_AnimateVideoSplit class definition.
• examples/wanvideo/model_inference/Wan2.2-Animate-14B.py
  • Changed animate_pose_video, animate_face_video, animate_inpaint_video, and animate_mask_video slicing from [:81-4] to [:77].
  • Updated num_frames parameter in pipe calls from 81 to 77.
• examples/wanvideo/model_inference_low_vram/Wan2.2-Animate-14B.py
  • Changed animate_pose_video, animate_face_video, animate_inpaint_video, and animate_mask_video slicing from [:81-4] to [:77].
  • Updated num_frames parameter in pipe calls from 81 to 77.
• examples/wanvideo/model_training/validate_full/Wan2.2-Animate-14B.py
  • Changed animate_pose_video and animate_face_video slicing from [:81-4] to [:77].
  • Updated num_frames parameter in pipe calls from 81 to 77.
• examples/wanvideo/model_training/validate_lora/Wan2.2-Animate-14B.py
  • Changed animate_pose_video and animate_face_video slicing from [:81-4] to [:77].
  • Updated num_frames parameter in pipe calls from 81 to 77.

Activity

• No specific activity (comments, reviews, progress updates) was provided in the context for this pull request.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request refactors the WanVideoPipeline in diffsynth/pipelines/wan_video.py to improve support for "wan-animate" functionality. Key changes include the removal of the WanVideoUnit_AnimateVideoSplit unit and modifications to WanVideoUnit_NoiseInitializer and WanVideoUnit_InputVideoEmbedder. These units now accept input_image and animate_pose_video parameters, with NoiseInitializer adjusting video length and InputVideoEmbedder preprocessing and prepending the latent representation of a single reference input_image when both are present. Additionally, several example scripts (model_inference, model_inference_low_vram, model_training/validate_full, and model_training/validate_lora) were updated to consistently process 77 frames instead of 81 for various animate_video inputs. Review comments point out the unintended addition of a UTF-8 Byte Order Mark (BOM) character to the import torch line in the example scripts, which should be removed, and suggest defining the number 77 as a variable for improved readability and maintainability.

[gemini-code-assist]

The number 77 is used in multiple places within this block. To improve readability and maintainability, it's good practice to define it as a variable before this block and reuse it. This avoids 'magic numbers' and makes it easier to change this value in the future. For example:

num_frames_to_process = 77
animate_pose_video = VideoData(...).raw_data()[:num_frames_to_process]
animate_face_video = VideoData(...).raw_data()[:num_frames_to_process]
video = pipe(
    ...
    num_frames=num_frames_to_process,
    ...
)

[gemini-code-assist]

The number 77 is used in multiple places within this block. To improve readability and maintainability, it's good practice to define it as a variable before this block and reuse it. This avoids 'magic numbers' and makes it easier to change this value in the future. For example:

num_frames_to_process = 77
animate_pose_video = VideoData(...).raw_data()[:num_frames_to_process]
animate_face_video = VideoData(...).raw_data()[:num_frames_to_process]
video = pipe(
    ...
    num_frames=num_frames_to_process,
    ...
)

[gemini-code-assist]

The number 77 is used in multiple places within this block. To improve readability and maintainability, it's good practice to define it as a variable before this block and reuse it. This avoids 'magic numbers' and makes it easier to change this value in the future. For example:

num_frames_to_process = 77
animate_pose_video = VideoData(...).raw_data()[:num_frames_to_process]
animate_face_video = VideoData(...).raw_data()[:num_frames_to_process]
video = pipe(
    ...
    num_frames=num_frames_to_process,
    ...
)

[gemini-code-assist]

The number 77 is used in multiple places within this block. To improve readability and maintainability, it's good practice to define it as a variable before this block and reuse it. This avoids 'magic numbers' and makes it easier to change this value in the future. For example:

num_frames_to_process = 77
animate_pose_video = VideoData(...).raw_data()[:num_frames_to_process]
animate_face_video = VideoData(...).raw_data()[:num_frames_to_process]
video = pipe(
    ...
    num_frames=num_frames_to_process,
    ...
)