[Model] Support Qwen3-VL Model Series

[ywang96]

Purpose

This PR adds model support for the upcoming Qwen3-VL models, including both dense and MoE variants.

Originally authored by @wulipc @JJJYmmm - much thanks to Qwen Team to upstream the model support!

Reference HF implementation - huggingface/transformers#40795

To run the model, the latest transformers version or 4.57.0+ is required

uv pip install git+https://github.com/huggingface/transformers.git

Co-authored by @Isotr0py

Follow-ups:

• DP ViT [MM Encoder] Apply DP ViT for Qwen3-VL model series #24955
• Support Triton interleaved MRoPE [Kernel][Performance] Add Triton kernel for Qwen3-VL interleaved MRoPE #25055
• EP [Bugfix] Fix Qwen3-VL-MoE weight loading for EP #25300
• batch pos_embed interpolate [MM][Perf] Minor Optimization on Qwen3-VL fast_pos_embed_interpolate #25337 [Perf] Further optimization for Qwen3-VL fast_pos_embed_interpolate #25347
• YaRN with MRope [feat] Support MRoPE + YaRN #25384

Test Plan

To be added after model release

Test Result

---

Essential Elements of an Effective PR Description Checklist

• The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
• The test plan, such as providing test command.
• The test results, such as pasting the results comparison before and after, or e2e results
• (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
• (Optional) Release notes update. If your change is user facing, please update the release notes draft in the Google Doc.

[gemini-code-assist]

Code Review

This pull request adds support for the Qwen3-VL model series. The changes include adding new model definitions, updating registries, and modifying rotary embedding logic to support multimodal inputs. My review focuses on correctness and maintainability. I've identified a few critical issues related to potential bugs in the implementation and code duplication that should be addressed. Specifically, there's a potential UnboundLocalError in an example file, a typo in a model name in the test registry, a likely bug in the interleaved RoPE implementation, and an in-place list modification that could cause side effects.

[DarkLight1337]

Maybe we should allow overriding this for each model architecture

[ywang96]

Good point - let me confirm with @wulipc on this

[wulipc]

Qwen-VL models impose no limit on the maximum number of video frames—only max_se_length constrains it. We’ve set a sufficiently large value, and you can also remove the max_frames_per_video restriction entirely.

[DarkLight1337]

Bump regarding this

[Isotr0py]

The processor test can pass locally now. I think the only thing left is checking example's availability. (I have to go to bed now. Will check it tomorrow)

Running 3 items in this shard: tests/models/multimodal/processing/test_common.py::test_processing_correctness[1.0-32-0.3-/home/mozf/LLM/Qwen3-VL-4B-Instruct/], tests/models/multimodal/processing/test_common.py::test_processing_correctness[1.0-32-0.5-/home/mozf/LLM/Qwen3-VL-4B-Instruct/], tests/models/multimodal/processing/test_common.py::test_processing_correctness[1.0-32-1.0-/home/mozf/LLM/Qwen3-VL-4B-Instruct/]

tests/models/multimodal/processing/test_common.py::test_processing_correctness[1.0-32-0.3-/home/mozf/LLM/Qwen3-VL-4B-Instruct/] INFO 09-13 02:39:12 [__init__.py:742] Resolved architecture: Qwen3VLForConditionalGeneration
`torch_dtype` is deprecated! Use `dtype` instead!
INFO 09-13 02:39:12 [__init__.py:1815] Using max model len 32768
PASSED
tests/models/multimodal/processing/test_common.py::test_processing_correctness[1.0-32-0.5-/home/mozf/LLM/Qwen3-VL-4B-Instruct/] INFO 09-13 02:39:19 [__init__.py:742] Resolved architecture: Qwen3VLForConditionalGeneration
INFO 09-13 02:39:19 [__init__.py:1815] Using max model len 32768
PASSED
tests/models/multimodal/processing/test_common.py::test_processing_correctness[1.0-32-1.0-/home/mozf/LLM/Qwen3-VL-4B-Instruct/] INFO 09-13 02:39:22 [__init__.py:742] Resolved architecture: Qwen3VLForConditionalGeneration
INFO 09-13 02:39:22 [__init__.py:1815] Using max model len 32768
PASSED

================================================================================== warnings summary ===================================================================================
.venv/lib/python3.12/site-packages/schemathesis/generation/coverage.py:305
  /home/mozf/develop-projects/vllm/.venv/lib/python3.12/site-packages/schemathesis/generation/coverage.py:305: DeprecationWarning: jsonschema.exceptions.RefResolutionError is deprecated as of version 4.18.0. If you wish to catch potential reference resolution errors, directly catch referencing.exceptions.Unresolvable.
    ref_error: type[Exception] = jsonschema.RefResolutionError,

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
==================================================================== 3 passed, 210 deselected, 1 warning in 19.46s ====================================================================

[ywang96]

The processor test can pass locally now. I think the only thing left is checking example's availability. (I have to go to bed now. Will check it tomorrow)

Running 3 items in this shard: tests/models/multimodal/processing/test_common.py::test_processing_correctness[1.0-32-0.3-/home/mozf/LLM/Qwen3-VL-4B-Instruct/], tests/models/multimodal/processing/test_common.py::test_processing_correctness[1.0-32-0.5-/home/mozf/LLM/Qwen3-VL-4B-Instruct/], tests/models/multimodal/processing/test_common.py::test_processing_correctness[1.0-32-1.0-/home/mozf/LLM/Qwen3-VL-4B-Instruct/]

tests/models/multimodal/processing/test_common.py::test_processing_correctness[1.0-32-0.3-/home/mozf/LLM/Qwen3-VL-4B-Instruct/] INFO 09-13 02:39:12 [__init__.py:742] Resolved architecture: Qwen3VLForConditionalGeneration
`torch_dtype` is deprecated! Use `dtype` instead!
INFO 09-13 02:39:12 [__init__.py:1815] Using max model len 32768
PASSED
tests/models/multimodal/processing/test_common.py::test_processing_correctness[1.0-32-0.5-/home/mozf/LLM/Qwen3-VL-4B-Instruct/] INFO 09-13 02:39:19 [__init__.py:742] Resolved architecture: Qwen3VLForConditionalGeneration
INFO 09-13 02:39:19 [__init__.py:1815] Using max model len 32768
PASSED
tests/models/multimodal/processing/test_common.py::test_processing_correctness[1.0-32-1.0-/home/mozf/LLM/Qwen3-VL-4B-Instruct/] INFO 09-13 02:39:22 [__init__.py:742] Resolved architecture: Qwen3VLForConditionalGeneration
INFO 09-13 02:39:22 [__init__.py:1815] Using max model len 32768
PASSED

================================================================================== warnings summary ===================================================================================
.venv/lib/python3.12/site-packages/schemathesis/generation/coverage.py:305
  /home/mozf/develop-projects/vllm/.venv/lib/python3.12/site-packages/schemathesis/generation/coverage.py:305: DeprecationWarning: jsonschema.exceptions.RefResolutionError is deprecated as of version 4.18.0. If you wish to catch potential reference resolution errors, directly catch referencing.exceptions.Unresolvable.
    ref_error: type[Exception] = jsonschema.RefResolutionError,

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
==================================================================== 3 passed, 210 deselected, 1 warning in 19.46s ====================================================================

@Isotr0py I just pushed a change and test the two examples - should be good now

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @ywang96.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork