[RFC]: vLLM-Omni 2026 Q1 Roadmap

[hsliuustc0106]

Motivation.

This issue outlines the development roadmap for vllm-omni in Q1 2026. Our primary focus for this quarter includes architectural refactoring to support dynamic execution graphs, expanding support for diffusion and omni models, and introducing disaggregated serving capabilities. Any contribution and discussion is welcome :)

Please attach your design doc using this template in your RFC :)

Proposed Change.

🛠 Core Architecture & Infrastructure

Refactoring the core execution pipeline to support more dynamic and efficient workflows.

P0:

• Entrypoint:

  • Stage Config refactoring: simplify and optimize stage configurations to improve clarity, maintainablity, and extensibility. [Config Refactor][1/2] Model Pipeline Configuration System #1115
  • Single-Stage CLI: support single stage cli for serving models in a single-stage mode. [Feature] Support Stage Based Deployment CLI #939
  • Stage DP: implement DP(coordinator) at the stage level to coordinate DP execution. [RFC]: Omni Coordinator #984 Add coordinator module and corresponding unit test #1465
  • Entrypoint refactoring to align with vLLM upstream [RFC]: vLLM-Omni Entrypoint Refactoring #967 [Entrypoint][Refactor] vLLM-Omni Entrypoint Refactoring #1908

• Hardware Abstraction: Implement a plugin system to support diverse hardware backends (NPU, TPU, XPU, Metal, etc.). @faaany [RFC]: Hardware Abstraction Layer & Plugin System for Unified Backend Support #702 [Hardware] Support platforms and plugin system #774
  For NPU related issues, please refer to [RFC]: vLLM-Omni NPU 2026 Q1 Roadmap #886 for details.
  For XPU related issues, please refer to [RFC]: vLLM-Omni XPU 2026 Q1 Roadmap #1127 for details.

🚀 Disaggregated Serving & Distributed Systems

Enhancing serving capabilities for large-scale and multi-node deployments.

P0:

• Multi-Node Serving: Support for multi-node setups using the Mooncake Connector. [Feature] vLLM-Omni RDMA connector #1019
• Full EPDG Disaggregation: Complete disaggregation of the Prefill, Decode, and Generate stages. [Feature] Support Prefill-Decode disaggregation via vLLM KV transfer #1303 [Feature] Split #1303 Part 1: PD disaggregation scaffolding #1863 [Feature] Split #1303 Part 2: Qwen PD integration #1912 Feat/pd disagg v1908 #1999
• Model Support: Add support for Bagel and HunyuanImage3.0 models in disaggregated setups. Support Bagel Model #726 [Model] Add Hunyuan Image3 AR Support #759

🎨 Feature: Diffusion Pipeline #814

Major feature additions to improve the flexibility and performance of image/video generation.

⚡️ Feature: Omni(AR+DiT) Pipeline

Enhancing the multimodal interaction capabilities.

P0:

• Performance & Execution:

  • Support Quantization. [Core] Unified quantization framework #1764
  • Support CUDA Graph execution for each stage. [Feature] Support Qwen3 Omni talker cudagraph #669 feat(qwen3-tts): Add CUDA Graph support for speech tokenizer decoder #1205
  • Support Async Chunked Computation across stages. [Feature]Support async computation and communication across stages by chunks #727 async chunk based communication between stages for overlapped execution #742

• Memory Management:

  • CPU offloading for KV Cache. [RFC]: Support KV Cache CPU Offloading #1150

• Streaming I/O:

  • Support multimodal streaming input (aligning with vllm upstream). [Feature] add session based streaming input support to v1 vllm#28973 [Feature]: Streaming Input for Qwen3-TTS #1766 [Feature]: Streaming Input for Qwen3-TTS #1766
  • Support audio streaming output. [Qwen3TTS][Feat] Streaming output #1438

🤖 Reinforcement Learning & Model Support

Expanding the ecosystem of supported models and training feedback loops.

• RL Integration: Support for the veRL + veOmni + vLLM-Omni stack. [RFC]: support customized scheduler & pipelines for diffusion model with arbitrary returns #686 [RFC] Support Qwen-Image Flow-GRPO Training based on vLLM-Omni verl-project/verl#4639 [RFC]: Reinforcement learning support on vllm-omni #778 [Misc] Support WorkerWrapperBase and CustomPipeline for Diffusion Worker #764

Targeted Model Optimizations

• Qwen3-Omni [RFC]: Qwen3-Omni deployment #409 [Feature]Support async computation and communication across stages by chunks #727 [Refactor] Refactor async chunk and fix the shape mismatch issue #1151 [Feature]async scheduling to overlap chunk IO and compute #951 [RFC]: Add async chunk processing test cases for Qwen3-Omni-30B-A3B-Instruct #1016 [skip ci][Doc] add design docs for async chunk in qwen3-omni #962
• Qwen3-tts [RFC]: Qwen3-TTS Production Ready - February Milestone #938
• Hunyuan-Image 3.0 [Model] Add Hunyuan Image3 AR Support #759 [Model] SupportHunyuanImage3 Diffusion Model in vllm-omni #1085 [FP8] enable hunyuan-image-3 diffusion model with fp8 online quant #1935 [Feat] Enable expert parallel for diffusion MoE layers #1323
• Bagel Support Bagel Model #726 [RFC]: Bagel deployment #936
• Qwen-Image family [Test] L4 complete diffusion feature test for Qwen-Image-Edit models #1682
• WAN 2.2 [feature] add VAE patch parallel to Wan2.2 #1350 [Feat] support teacache for WAN2.2 #1365

📊 Benchmarks, Metrics & Logging

Improving observability and establishing performance baselines.
P0:

• Benchmarks: Establish standard benchmarks for both Diffusion and Omni pipelines.
  • Diffusion [Benchmark] DiT Model Benchmark under Mixed Workloads #529 [Benchmark] [Diffusion] [Enhancement] Random dataset #1657
  • Omni [Benchmark] Add vLLM-Omni Omni model online benchmark #780
• Metrics Refactor:
  • Expose granular per-request information. [Feature] Opt metrics structure #891
  • Add system-level metrics. 🙋
• Profiling: PyTorch Profiler integration (align with vLLM). [RFC]: Support torch profiler support for diffusion #650 [Feature]: Profiler Trace Size Optimization #651 [Core] Implement Diffusion Profiler Support #709
• Logging: Remove print statements and optimize log verbosity. @Bounty-hunter

P1:

• Metrics Refactor: Support returning metrics via logs and Prometheus integration. 🙋
• Profiling: NVIDIA Nsight Systems integration (align with vLLM). [Profiler] Add Nsight Systems support for serving #1098

🧪 CI/CD & Quality Assurance

please check our design doc

• Multi-tiered tests [RFC]: vllm-omni CI/CD plan #400
• Diverse hardware backend [CI]init intel ci dispatch in buildkite folder #1721

Feedback Period.

No response

CC List.

@ywang96 @Gaohan123 @tzhouam @ZJY0516 @DarkLight1337 @Isotr0py @SamitHuang @david6666666

Any Other Things.

No response

Before submitting a new issue...

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

[kechengliu97]

Please assign this to me @hsliuustc0106 :

Targeted Model Optimizations for Qwen3-Omni

[fhfuih]

Work on #427 and later will look into #475 as well