[Ops][Feature] Enable Qwen4Exp (Qwen3.8-Flash-Next) on Ascend NPU - #15574
[Ops][Feature] Enable Qwen4Exp (Qwen3.8-Flash-Next) on Ascend NPU#15574krell11 wants to merge 2 commits into
Conversation
Add patch_qwen4_exp.py to disable CUDA-only low-latency GEMM hooks and route QSA QKV/RoPE through the Ascend text path. GDN layers reuse the existing QwenGatedDeltaNetAttention patch from patch_qwen3_5.py. QSA/PLE/HyperConnection NPU backends remain follow-up work. Signed-off-by: krell11 <11bobin.kiryusha@gmail.com>
|
👋 Hi! Thank you for contributing to the vLLM Ascend project. The following points will speed up your PR merge:
If CI fails, you can run linting and testing checks locally according Contributing and Testing. Tip 💡 Consider Linking a Related Issue or RFCYour PR title contains the [Feature] tag, indicating a bug fix or new feature. Linking a related issue or RFC in the PR description is strongly encouraged — it gives reviewers helpful context and speeds up the review. You can use any of these keywords:
🙏 Thanks for helping us keep the project well-organized! |
Summary of ChangesHello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request adds the necessary infrastructure to support the Qwen4Exp model on Ascend NPUs. By introducing a new patch module, it successfully bypasses CUDA-specific optimizations and leverages existing Ascend-compatible attention paths, ensuring the model can run on NPU hardware without requiring full duplication of the upstream model implementation. Highlights
New Features🧠 You can now enable Memory (public preview) to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console. Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize the Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counterproductive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here. Footnotes
|
There was a problem hiding this comment.
Code Review
Suggested PR Title:
[Ops][Feature] Enable Qwen4Exp (Qwen3.8-Flash-Next) on Ascend NPUSuggested PR Summary:
### What this PR does / why we need it?
This PR adds an NPU enablement scaffold for Qwen4Exp (Qwen3.8-Flash-Next) to support it on Ascend NPU without duplicating model files. It disables CUDA-only low-latency GEMM hooks and routes QSA QKV projection through the Ascend Qwen3Next RoPE path.
Feedback: In `patch_qwen4_exp.py`, using `.view()` on non-contiguous tensors returned by `split` or `chunk` will raise a runtime error. It is highly recommended to use `.reshape()` instead to safely handle non-contiguous tensors.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Added a unit test `test_patch_qwen4_exp_module_exports_availability_flag` to verify the module's availability flag.Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> Signed-off-by: Kirill Bobin <75633909+krell11@users.noreply.github.com>
What this PR does / why we need it?
This PR adds an NPU enablement scaffold for Qwen4Exp (Qwen3.8-Flash-Next) to support it on Ascend NPU without duplicating model files. It disables CUDA-only low-latency GEMM hooks and routes QSA QKV projection through the Ascend Qwen3Next RoPE path.
Feedback: In
patch_qwen4_exp.py, using.view()on non-contiguous tensors returned bysplitorchunkwill raise a runtime error. It is highly recommended to use.reshape()instead to safely handle non-contiguous tensors.Does this PR introduce any user-facing change?
No.
How was this patch tested?
Added a unit test
test_patch_qwen4_exp_module_exports_availability_flagto verify the module's availability flag.