dQwen3-1.7B-Base
A masked diffusion language model adapted from Qwen3-1.7B. The backbone is full attention, and every layer is made bidirectional. It is the control model in the paper's matched comparison against the hybrid dQwen3.5-2B. This is a base model, with no instruction tuning.
Paper: dQwen3.5: Hybrid-Attention Diffusion Language Models
Code: https://github.com/AntonXue/dQwen
The dQwen3.5 family
| model | total params | backbone |
|---|---|---|
| dQwen3.5-0.8B-Base | 0.75B | hybrid |
| dQwen3.5-2B-Base | 1.88B | hybrid |
| dQwen3.5-4B-Base | 4.21B | hybrid |
| dQwen3.5-9B-Base | 8.95B | hybrid |
| dQwen3-1.7B-Base (this repo) | 1.72B | full attention (control) |
Quickstart
Needs a CUDA GPU and transformers>=5.13 (tested with torch 2.7.1+cu128, flash-linear-attention 0.5.1).
import torch
from transformers import AutoModel
model = AutoModel.from_pretrained("UT-IFML/dQwen3-1.7B-Base", trust_remote_code=True, dtype=torch.bfloat16).cuda().eval()
stop_strings = ["\n\n", "\ndef", "\nclass", "<|endoftext|>"]
print(model.generate("def fibonacci(n):\n", gen_length=512, stop_strings=stop_strings).text)
generate decodes the whole canvas at once, committing positions above a confidence
threshold (tau=0.9); pass block_length=32 for left-to-right block decoding, or
tau=None, steps_per_block=k for a fixed budget. The 50B-token checkpoint from the
paper is revision="step25000-swa".
Citation
@article{xue2026dqwen,
title = {dQwen3.5: Hybrid-Attention Diffusion Language Models},
author = {Xue, Anton and Rout, Litu and Akella, Aditya and Klivans, Adam and Sanghavi, Sujay and Shakkottai, Sanjay},
journal = {arXiv preprint arXiv:2609.20751},
year = {2026}
}