Skip to content

[Ops][Feature] Enable Qwen4Exp (Qwen3.8-Flash-Next) on Ascend NPU - #15574

Open
krell11 wants to merge 2 commits into
vllm-project:mainfrom
krell11:main
Open

[Ops][Feature] Enable Qwen4Exp (Qwen3.8-Flash-Next) on Ascend NPU#15574
krell11 wants to merge 2 commits into
vllm-project:mainfrom
krell11:main

Conversation

@krell11

@krell11 krell11 commented Sep 2, 2026

Copy link
Copy Markdown

What this PR does / why we need it?

This PR adds an NPU enablement scaffold for Qwen4Exp (Qwen3.8-Flash-Next) to support it on Ascend NPU without duplicating model files. It disables CUDA-only low-latency GEMM hooks and routes QSA QKV projection through the Ascend Qwen3Next RoPE path.

Feedback: In patch_qwen4_exp.py, using .view() on non-contiguous tensors returned by split or chunk will raise a runtime error. It is highly recommended to use .reshape() instead to safely handle non-contiguous tensors.

Does this PR introduce any user-facing change?

No.

How was this patch tested?

Added a unit test test_patch_qwen4_exp_module_exports_availability_flag to verify the module's availability flag.

Add patch_qwen4_exp.py to disable CUDA-only low-latency GEMM hooks and route QSA QKV/RoPE through the Ascend text path. GDN layers reuse the existing QwenGatedDeltaNetAttention patch from patch_qwen3_5.py. QSA/PLE/HyperConnection NPU backends remain follow-up work.

Signed-off-by: krell11 <11bobin.kiryusha@gmail.com>
@github-actions

github-actions Bot commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

👋 Hi! Thank you for contributing to the vLLM Ascend project. The following points will speed up your PR merge:‌‌

  • A PR should do only one thing, smaller PRs enable faster reviews.
  • Every PR should include unit tests and end-to-end tests ‌to ensure it works and is not broken by other future PRs.
  • Write the commit message by fulfilling the PR description to help reviewer and future developers understand.

If CI fails, you can run linting and testing checks locally according Contributing and Testing.


Tip

💡 Consider Linking a Related Issue or RFC

Your PR title contains the [Feature] tag, indicating a bug fix or new feature.

Linking a related issue or RFC in the PR description is strongly encouraged — it gives reviewers helpful context and speeds up the review. You can use any of these keywords:

  • Fixes #<issue_number>
  • Closes #<issue_number>
  • Resolves #<issue_number>
  • Refs #<rfc_or_issue_number> (for RFCs)

🙏 Thanks for helping us keep the project well-organized!

@gemini-code-assist

Copy link
Copy Markdown
Contributor

Summary of Changes

Hello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request adds the necessary infrastructure to support the Qwen4Exp model on Ascend NPUs. By introducing a new patch module, it successfully bypasses CUDA-specific optimizations and leverages existing Ascend-compatible attention paths, ensuring the model can run on NPU hardware without requiring full duplication of the upstream model implementation.

Highlights

  • NPU Support Scaffold: Introduced a new patch file, patch_qwen4_exp.py, to enable Qwen4Exp (Qwen3.8-Flash-Next) support on Ascend NPUs.
  • CUDA Hook Handling: Implemented a no-op for CUDA-only low-latency GEMM hooks to ensure compatibility on NPU hardware.
  • Attention Path Routing: Routed QSA QKV and RoPE operations through the existing Ascend-optimized Qwen3Next path.
New Features

🧠 You can now enable Memory (public preview) to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize the Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counterproductive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

Suggested PR Title:

[Ops][Feature] Enable Qwen4Exp (Qwen3.8-Flash-Next) on Ascend NPU

Suggested PR Summary:

### What this PR does / why we need it?
This PR adds an NPU enablement scaffold for Qwen4Exp (Qwen3.8-Flash-Next) to support it on Ascend NPU without duplicating model files. It disables CUDA-only low-latency GEMM hooks and routes QSA QKV projection through the Ascend Qwen3Next RoPE path.

Feedback: In `patch_qwen4_exp.py`, using `.view()` on non-contiguous tensors returned by `split` or `chunk` will raise a runtime error. It is highly recommended to use `.reshape()` instead to safely handle non-contiguous tensors.

### Does this PR introduce _any_ user-facing change?
No.

### How was this patch tested?
Added a unit test `test_patch_qwen4_exp_module_exports_availability_flag` to verify the module's availability flag.

Comment thread vllm_ascend/patch/worker/patch_qwen4_exp.py
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Signed-off-by: Kirill Bobin <75633909+krell11@users.noreply.github.com>
@krell11 krell11 changed the title [Feature][Model] Add Qwen4Exp NPU scaffold patch for Qwen3.8-Flash-Next [Ops][Feature] Enable Qwen4Exp (Qwen3.8-Flash-Next) on Ascend NPU Sep 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant