OPOD-Qwen3-Omni-30B-A3B

OPOD (On-Policy Omni Distillation) consolidates separate text, image, and audio teachers into a single omni-modal student. This model is the 30B-A3B student from the paper OPOD: On-Policy Omni Distillation, post-trained from Qwen/Qwen3-Omni-30B-A3B-Instruct.

It reaches an overall average of 70.8 across twelve text / vision / audio / omni-modal benchmarks (+3.8 over the base model), and beats the base model and pooled-data GRPO on all 12 benchmarks. The teachers are discarded after training — inference uses one model, with no teacher-ensemble latency or memory.

• 📄 Paper: arXiv:2607.20918
• 💻 Code: github.com/VincentZhao2002/OPOD
• 🤗 Other scales: OPOD-Qwen2.5-Omni-7B · OPOD-Qwen2.5-Omni-3B

Method

Post-training on pooled multimodal data often fails to preserve the strengths of modality specialists, and standard on-policy distillation does not extend cleanly to several teachers: their guidance conflicts on a shared backbone, and symmetric distribution matching imposes a teacher ceiling. OPOD addresses this with three components:

Each student rollout is routed by its input modality to the matching teacher (each obtained by applying GRPO to that modality's data from the same base model).

Results

Accuracy (%) on the paper's twelve benchmarks. Bold = best in the block; the three specialist teachers are shown as references (they are not deployable as one model).

Highlights:

• +3.8 over base, +3.5 over pooled GRPO, +2.9 over native OPD, +2.2 over ExOPD, and +2.1 over the strongest specialist teacher.
• Wins over base and GRPO on all 12 benchmarks; ranks 1st or 2nd on 11 of 12 even when the specialist teachers are included in the comparison.
• Beats a per-benchmark oracle over the three teachers (70.3) by 0.5 points with a single model.
• Improves every modality group over base: text +7.0, vision +1.4, audio +2.2, omni +0.8.

Evaluation protocol. Accuracy, temperature 0.7, top-p 0.7, max prompt / response length 8,192 tokens, identical for all methods. Training prompts are disjoint from every benchmark (exact and normalized-string overlaps removed).

Usage

This model keeps the Qwen3-Omni architecture and processor, so it is a drop-in replacement for the base model.

Prompt format (important)

The model was RL-trained to reason inside <think></think> and put the final answer inside <answer></answer>. Use a system prompt of the same form, otherwise accuracy will drop and answers become harder to parse. The prompts used in the paper are in eval/prompts/system_prompts.py.

For image / audio / omni multiple-choice questions:

Please think about this question as if you were a human pondering deeply, carefully
considering BOTH the visual and the audio information before answering, engaging in an
internal dialogue using expressions such as let me think, wait, hmm, oh I see, or let's
break it down, including self-reflection or verification in the reasoning process,
providing the detailed reasoning between the <think> </think> tags, and finally giving
only the single option letter (e.g., A, B, C, D, etc.) as the final answer within the
<answer> </answer> tags.

For text problems:

You are a careful problem solver. Think step by step inside <think> </think> tags, then
output the final answer in the exact format requested at the end of the user prompt (do
not add any extra wrappers around the final answer).

transformers

import soundfile as sf
from transformers import Qwen3OmniMoeForConditionalGeneration, Qwen3OmniMoeProcessor

model_id = "Tung111/OPOD-Qwen3-Omni-30B-A3B"
model = Qwen3OmniMoeForConditionalGeneration.from_pretrained(
    model_id, dtype="auto", device_map="auto", attn_implementation="flash_attention_2",
)
processor = Qwen3OmniMoeProcessor.from_pretrained(model_id)

SYSTEM = (
    "You are a careful problem solver. Think step by step inside <think> </think> tags, "
    "then output the final answer in the exact format requested at the end of the user "
    "prompt (do not add any extra wrappers around the final answer)."
)

conversation = [
    {"role": "system", "content": [{"type": "text", "text": SYSTEM}]},
    {"role": "user", "content": [
        {"type": "image", "image": "https://example.com/chart.png"},
        {"type": "text", "text": "What is the highest value in the chart? "
                                 "Put the final answer in <answer></answer>."},
    ]},
]

text = processor.apply_chat_template(conversation, add_generation_prompt=True, tokenize=False)
inputs = processor(text=text, return_tensors="pt").to(model.device)

ids = model.generate(**inputs, max_new_tokens=8192, temperature=0.7, top_p=0.7, do_sample=True)
print(processor.batch_decode(ids[:, inputs.input_ids.shape[1]:], skip_special_tokens=True)[0])

Use Qwen3OmniMoeProcessor's multimodal loading utilities (process_mm_info) for audio and video inputs, exactly as with the base model.

vLLM

vllm serve Tung111/OPOD-Qwen3-Omni-30B-A3B \
  --tensor-parallel-size 4 --max-model-len 16384 \
  --allowed-local-media-path / --trust-remote-code

Then query the OpenAI-compatible endpoint with the system prompts above.

Training details

Limitations

• Post-training targets the thinker (text/reasoning) pathway. The talker / speech-generation components are inherited unchanged from the base model and were not optimized or evaluated.
• Optimized for verifiable reasoning with the <think> / <answer> format; free-form chat, long-form generation, and speech output are outside the evaluated scope.
• Single run per configuration with a fixed seed; no multi-seed variance is reported.
• Inherits the base model's biases and knowledge cutoff. Not evaluated for safety alignment.

Citation

@article{zhao2026opod,
  title   = {OPOD: On-Policy Omni Distillation},
  author  = {Zhao, Tong and Hu, Yuyang and Zhu, Yutao and Li, Reed and
             Liang, Haijin and Shi, Haibo and Lu, Yu and Dou, Zhicheng},
  journal = {arXiv preprint arXiv:2607.20918},
  year    = {2026},
  url     = {https://arxiv.org/abs/2607.20918}
}

License

Derived from Qwen3-Omni-30B-A3B-Instruct and released under the same terms as the base model. See the base model license.