[Bug] multi_node_tp_pp + GLM-5.2 + MTP: synthesized command fails at startup with NotImplementedError

[aminer-404]

Summary

On https://recipes.vllm.ai/zai-org/GLM-5.2?strategy=multi_node_tp_pp (H100 · Multi-Node TP + Pipeline Parallel · FP8), the synthesized launch command combines --pipeline-parallel-size 2 with --speculative-config '{"method":"mtp","num_speculative_tokens":5}'. This combination cannot start on any vLLM version — it fails during config validation with:

NotImplementedError: Pipeline parallelism is not supported for this model.
Supported models implement the `SupportsPP` interface.

The error is misleading: the main model (GlmMoeDsaForCausalLM) does support PP (it declares SupportsPP and its forward accepts intermediate_tensors). The failure comes from the MTP draft model: with method="mtp", SpeculativeConfig.hf_config_override rewrites the draft architecture to DeepSeekMTPModel, and DeepSeekMTP (vllm/model_executor/models/deepseek_mtp.py) neither declares SupportsPP nor implements make_empty_intermediate_tensors / a PP-capable forward. SpeculativeConfig._verify_args calls draft_model_config.verify_with_parallel_config(draft_parallel_config), and create_draft_parallel_config forwards the target's pipeline_parallel_size to the draft, so the PP gate is hit on the draft class.

Reproduction

Exact command from the recipe page (2 nodes, head node tab):

vllm serve zai-org/GLM-5.2-FP8 \
  --tensor-parallel-size 8 \
  --pipeline-parallel-size 2 \
  --nnodes 2 \
  --node-rank 0 \
  --master-addr $HEAD_IP \
  --kv-cache-dtype fp8 \
  --tool-call-parser glm47 \
  --enable-auto-tool-choice \
  --reasoning-parser glm45 \
  --speculative-config '{"method":"mtp","num_speculative_tokens":5}'

Minimal config-level repro (no GPUs needed beyond one node for ModelConfig):

from vllm.config import ModelConfig, ParallelConfig, SpeculativeConfig

mc = ModelConfig(model=<GLM-5.2 path>, runner="generate", trust_remote_code=True)
pc = ParallelConfig(pipeline_parallel_size=2, tensor_parallel_size=8,
                    distributed_executor_backend="ray")
sc = SpeculativeConfig(target_model_config=mc, target_parallel_config=pc,
                       model=<GLM-5.2 path>, method="mtp", num_speculative_tokens=5)
# → NotImplementedError: Pipeline parallelism is not supported for this model.

Verified across versions

I diffed the entire failing path between releases — it is byte-equivalent, so the recipe command fails on all of:

• v0.23.0 (the version the GLM-5.2 recipe pins: min_vllm_version: 0.23.0, docker v0.23.0)
• v0.24.0
• latest main (as of 2026-08-20): the draft class moved to DeepseekV32MTPModel (vllm/models/deepseek_v32/nvidia/mtp.py), which still has no SupportsPP / make_empty_intermediate_tensors. Both MTP classes gained an intermediate_tensors forward parameter upstream, but the body ignores it — PP for MTP drafts is not implemented.

Also note upstream main explicitly rejects dspark + PP at runtime (vllm/v1/worker/gpu/model_runner.py: f"{method} with pipeline parallel is not supported." for eagle3/dflash/dspark), which suggests spec-decode methods with separate draft models are generally not expected to work under PP yet.

Suggested fix

The recipe synthesizer combines compatible_strategies: [multi_node_tp_pp, ...] with the opt-in spec_decoding feature without checking mutual exclusibility. Options:

1. When the selected strategy involves --pipeline-parallel-size > 1 and the model's spec-decoding method uses a separate draft model (mtp, dspark, eagle-family), drop --speculative-config from the synthesized command (or grey out the feature with a tooltip explaining why).
2. Add a per-model strategy_overrides entry for GLM-5.2 excluding multi_node_tp_pp / pd_cluster when spec_decoding is enabled (finer-grained than removing the strategy entirely, since the main model alone does support PP).

Environment details: reproduced on a 2-node A100 cluster with GLM-5.2 (zai-org checkpoints, config glm_moe_dsa); config-level repro does not depend on GPU arch.

Found while debugging a production deployment; happy to provide more traces if useful. AI assistance was used in the investigation.