Fast-dLLM-v2-7B-dTRPO-Lite
dTRPO preference-finetuned Fast-dLLM-v2-7B.
The dTRPO estimator selects positions using target-token probabilities within 32-token blocks. Policy and frozen reference use the same noisy inputs and selected positions. The reference is forwarded in training mode without parameter updates.
Training parameters
| Parameter | Value |
|---|---|
| Base model | Efficient-Large-Model/Fast_dLLM_v2_7B |
| Learning rate | 5e-7 |
| DPO beta | 0.01 |
| # GPU | 4 |
| Per-device batch size | 2 |
| Gradient accumulation | 64 |
| Monte Carlo noise samples | 4 |
| Noise microbatch size | 1 |
| Dataset streaming | False |
| Training epochs | 1 |
| Final optimizer step | 441 |
| Maximum training sequence length | 2048 |
| Loss | sigmoid DPO |
| Warmup ratio | 0.1 |
| LR scheduler | cosine_with_min_lr |
| Minimum LR rate | 0.1 |
| Maximum gradient norm | 0.2 |
| BF16 training | True |
Evaluation results
| Dataset | Split / Setting | Result |
|---|---|---|
| GSM8K | CoT · test | 88.02% |
| MATH | CoT · full test (5,000) | 62.33% |
| GPQA | Diamond | 31.82% |
| LiveCodeBench | release_v6 · pass@1 | 17.06% |
| MBPP+ | EvalPlus v0.2.0 · pass@1 | 57.50% |
| HumanEval+ | EvalPlus v0.1.10 · pass@1 | 69.11% |
| IFEval | Strict · prompt-level | 70.49% |
Loading and generation
The evaluation environment uses PyTorch 2.10.0 (CUDA 12.8) and Transformers 4.57.6. A CUDA GPU and an appropriate PyTorch installation are required.
import torch
from transformers.dynamic_module_utils import get_class_from_dynamic_module
repo = "wxzhang/Fast-dLLM-v2-7B-dTRPO-Lite"
# Loads custom Python code from this repository; review it before execution.
Generator = get_class_from_dynamic_module("generate.FastdLLMGenerator", repo)
generator = Generator(repo, device="cuda", dtype=torch.bfloat16)
prompt = generator.tokenizer.apply_chat_template(
[
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": (
"Question: What is 6 times 7?\n"
r"Please reason step by step, and put your final answer within \boxed{}."
)},
],
tokenize=False,
add_generation_prompt=True,
)
outputs = generator.batch_generate(
[prompt], batch_size=1, max_new_tokens=2048,
threshold=0.9, temperature=0.0, repetition_penalty=1.1,
use_prefix_cache=True, use_block_cache=True,
)
print(outputs[0])
generate() accepts exactly one prompt. Use batch_generate() for multiple
prompts; it sorts requests by token length and maintains per-sample cache state.
Use this diffusion decoder rather than the standard autoregressive
transformers.generate() path. Different batch sizes may produce different
outputs; follow the benchmark settings above when reproducing the scores.
For loading weights without the decoding wrapper:
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained(
"wxzhang/Fast-dLLM-v2-7B-dTRPO-Lite",
trust_remote_code=True, dtype=torch.bfloat16,
)
tokenizer = AutoTokenizer.from_pretrained(
"wxzhang/Fast-dLLM-v2-7B-dTRPO-Lite", fix_mistral_regex=True,
)
Attribution and included files
Base model: Fast-dLLM-v2-7B,
based on Qwen2.5-7B-Instruct. The upstream base model is marked Apache-2.0.
The accompanying post-training project code carries the FAIR Noncommercial
Research License, preserved in LICENSE; upstream notices are retained in
BASE_MODEL_README.md. This package does not remove upstream restrictions.
This research checkpoint has not been validated for safety-critical deployment. The repository includes model weights, tokenizer, custom inference code and documentation, but no optimizer states, trainer state, private logs or credentials.