[MM Encoder] Apply DP ViT for Qwen3-VL model series

[ywang96]

Purpose

Add DP ViT for Qwen3-VL models - this PR should be merged only after #24727 is merged.

Part of #22743

Test Plan

Running on Qwen3-VL-30B-A3B-Instruct with 4xL40s (PCI-E) with the following changes to the vision_language.py

    sampling_params = SamplingParams(
-        temperature=0.2, max_tokens=64, stop_token_ids=req_data.stop_token_ids
+        temperature=0.0, max_tokens=1, stop_token_ids=req_data.stop_token_ids # measure prefill perf
    )

    engine_args = EngineArgs(
        model=model_name,
-       max_model_len=4096,
-       max_num_seqs=5,
        mm_processor_kwargs={
            "min_pixels": 28 * 28,
            "max_pixels": 1280 * 28 * 28,
            "fps": 1,
        },
        limit_mm_per_prompt={modality: 1},
+       tensor_parallel_size=4,
+       mm_encoder_tp_mode="data", # vs "weights"
+       enable_prefix_caching=False,
+       mm_processor_cache_gb=0,
    )

Test Result

Running 500 image prompts with mm_encoder_tp_mode="weights"

python3 examples/offline_inference/vision_language.py -m qwen3_vl_moe --modality image --num-prompts 500 --seed 0

Processed prompts: 100%|█████████████████████████████████████████████████████████| 1000/1000 [02:30<00:00,  6.63it/s, est. speed input: 6491.45 toks/s, output: 6.63 toks/s]

Running 500 images prompts with mm_encoder_tp_mode="data"

python3 examples/offline_inference/vision_language.py -m qwen3_vl_moe --modality image --num-prompts 500 --seed 0

Processed prompts: 100%|██████████████████████████████████████| 1000/1000 [01:12<00:00, 13.82it/s, est. speed input: 13533.77 toks/s, output: 13.82 toks/s]

Results are bit biased since we would not typically expect one to run high TP on chips without NVLink - but this is what I have available.

---

Essential Elements of an Effective PR Description Checklist

• The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
• The test plan, such as providing test command.
• The test results, such as pasting the results comparison before and after, or e2e results
• (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
• (Optional) Release notes update. If your change is user facing, please update the release notes draft in the Google Doc.

[ywang96]

@tjtanaa Could you take a look too? Thanks

[Isotr0py]

LGTM!

[DarkLight1337]

Have you run lm-eval to ensure the correctness?

[ywang96]

Have you run lm-eval to ensure the correctness?

@DarkLight1337 Yea that's what I'm planning to do next - though we don't have official results but should be fine as long as the two results match

[tjtanaa]

@ywang96 LTGM as well. Can't wait for the model release.

[ywang96]

I won't merge this PR until I verify the correctness tomorrow.

[ywang96]

Additional results on 4xH200

vllm bench serve  \
--endpoint-type openai-chat \
--model Qwen-SGlang/Qwen3-VL-30B-A3B-Instruct   \
--tokenizer /tmp-nvme/models/Qwen-SGlang/Qwen3-VL-30B-A3B-Instruct \
--endpoint /v1/chat/completions   \
--dataset-name hf   \
--dataset-path lmarena-ai/VisionArena-Chat   \
--hf-split train   \
--num-prompts 1000 --request-rate 3

TP

============ Serving Benchmark Result ============
Successful requests:                     1000      
Request rate configured (RPS):           3.00      
Benchmark duration (s):                  334.79    
Total input tokens:                      94327     
Total generated tokens:                  121241    
Request throughput (req/s):              2.99      
Output token throughput (tok/s):         362.14    
Total Token throughput (tok/s):          643.88    
---------------Time to First Token----------------
Mean TTFT (ms):                          284.71    
Median TTFT (ms):                        203.83    
P99 TTFT (ms):                           2600.75   
-----Time per Output Token (excl. 1st token)------
Mean TPOT (ms):                          19.60     
Median TPOT (ms):                        18.02     
P99 TPOT (ms):                           47.18     
---------------Inter-token Latency----------------
Mean ITL (ms):                           19.90     
Median ITL (ms):                         13.24     
P99 ITL (ms):                            153.90    
==================================================

DP

============ Serving Benchmark Result ============
Successful requests:                     1000      
Request rate configured (RPS):           3.00      
Benchmark duration (s):                  334.76    
Total input tokens:                      94327     
Total generated tokens:                  121609    
Request throughput (req/s):              2.99      
Output token throughput (tok/s):         363.28    
Total Token throughput (tok/s):          645.05    
---------------Time to First Token----------------
Mean TTFT (ms):                          191.12    
Median TTFT (ms):                        166.74    
P99 TTFT (ms):                           1195.92   
-----Time per Output Token (excl. 1st token)------
Mean TPOT (ms):                          15.98     
Median TPOT (ms):                        15.07     
P99 TPOT (ms):                           39.88     
---------------Inter-token Latency----------------
Mean ITL (ms):                           16.23     
Median ITL (ms):                         11.37     
P99 ITL (ms):                            119.14    
==================================================