Skip to content

fix: WanVAE2.2 encode and decode error#1296

Open
Explorer-Dong wants to merge 3 commits intomodelscope:mainfrom
Explorer-Dong:fix/wan_vae
Open

fix: WanVAE2.2 encode and decode error#1296
Explorer-Dong wants to merge 3 commits intomodelscope:mainfrom
Explorer-Dong:fix/wan_vae

Conversation

@Explorer-Dong
Copy link

我正在 Wan2.2-TI2V-5B 上展开实验,但我发现最近对 WanVAE 的调整 4e9db26 6886f7b 导致原本可以运行的代码失效了。我的修复逻辑如下:

  1. 对于我的第一个 commit:WanVAE2.2 在 encode 阶段的下采样过程中,调用了 Resample38 类的 forward,由于 Resample38 继承自 Resample,且 Resample 的 forward 方法返回三元组 x, feat_cache, feat_idx,所以 Resample38 的前向传播结果需要三元组来收集。WanVAE2.2 的 decode 阶段也调用了 Resample38 类的 forward,所以同理。
  2. 对于我的第二个 commit:经过实验,我发现解码上采样的过程中始终需要 Up_ResidualBlock 类返回三元组,根据最近对 WanVAE 代码的调整,我估计原作者忘记添加了。

@gemini-code-assist
Copy link
Contributor

Summary of Changes

Hello @Explorer-Dong, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

此拉取请求旨在修复 WanVAE2.2 模型中存在的编码和解码错误,这些错误是由于近期对 WanVAE 代码的调整导致 Resample38Up_ResidualBlock 等模块的返回值未被正确处理。通过确保在下采样和上采样过程中,模块的输出三元组(包含特征缓存和索引)被正确解包和传递,此更改恢复了模型的稳定性和功能,特别是对于 Wan2.2-TI2V-5B 实验至关重要。

Highlights

  • 修复 WanVAE2.2 编码阶段错误: 在 WanVAE2.2 的编码(下采样)过程中,更新了 forward 方法中的循环,以正确解包 module 返回的三元组 x, feat_cache, feat_idx,确保特征缓存和索引的正确传递。
  • 修复 WanVAE2.2 解码阶段错误: 在 WanVAE2.2 的解码(上采样)过程中,更新了 forward 方法中的循环和返回语句,以正确解包 module 返回的三元组,并确保 feat_cachefeat_idx 始终作为返回值的一部分。
  • 解决 Wan2.2-TI2V-5B 模型兼容性问题: 这些修复解决了由于 WanVAE 最近调整导致 Wan2.2-TI2V-5B 模型无法正常编码和解码的问题,恢复了模型的预期功能。

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Changelog
  • diffsynth/models/wan_video_vae.py
    • 更新了编码阶段 forward 方法中的循环,以正确处理 module 返回的三元组。
    • 更新了解码阶段 forward 方法中的循环和返回语句,以正确处理 module 返回的三元组,并确保 feat_cachefeat_idx 被返回。
    • 移除了一个调试用的注释掉的 breakpoint()
Activity
  • 此拉取请求由 Explorer-Dong 创建,旨在解决 WanVAE2.2 模型的编码和解码问题。
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

这个拉取请求修复了 WanVAE2.2 在编码和解码过程中出现的错误。这些更改正确地处理了 Resample38.forwardUp_ResidualBlock.forward 的元组返回值,解决了近期 API 变更导致的运行时错误。逻辑是合理的。不过,代码中残留了一个用于调试的注释掉的 breakpoint() 调用,建议将其移除。

x = self.conv2(z)
for i in range(iter_):
self._conv_idx = [0]
# breakpoint()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

这里有一个用于调试的 breakpoint() 调用被注释掉了。建议在合并之前移除这行代码,以保持代码库的整洁。

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

已移除

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant