CycleGRPO-4B
CycleGRPO-4B is post-trained from zhouyik/Qwen3-VL-4B-SAMTok with caption ↔ grounding cycle-consistent reinforcement learning: a caption is rewarded by how well the model can ground it back to the region it describes (cycle IoU) plus GT-free regularizers — no reference-caption supervision in the RL stage. It produces descriptions with interleaved segmentation masks for the corresponding parts of the answer, decoded through the SAMTok mask tokenizer.
Code: github.com/devinxzhang/CycleGRPO
Quickstart
CycleGRPO-4B is a Qwen3-VL-4B that emits SAMTok mask tokens (<|mt_...|>). Plain
text generation works with 🤗 Transformers directly; turning the mask tokens into
segmentation masks needs the VQ-SAM2 decoder from the CycleGRPO repo
(projects.transformers.vq_sam2), so clone and install it first:
pip install "transformers>=4.57"
git clone https://github.com/devinxzhang/CycleGRPO.git
cd CycleGRPO # run from the repo root so `projects.transformers.vq_sam2` imports
pip install -e .
Generate (text + mask tokens)
import torch
from transformers import Qwen3VLForConditionalGeneration, AutoProcessor
model_id = "XinNUS/CycleGRPO-4B"
model = Qwen3VLForConditionalGeneration.from_pretrained(
model_id, dtype="auto", device_map="auto"
).eval()
processor = AutoProcessor.from_pretrained(model_id)
messages = [{
"role": "user",
"content": [
{"type": "image", "image": "figs/totoro.jpg"},
{"type": "text", "text": "Describe the image with interleaved segmentation "
"masks for the corresponding parts of the answer."},
],
}]
inputs = processor.apply_chat_template(
messages, tokenize=True, add_generation_prompt=True,
return_dict=True, return_tensors="pt",
).to(model.device)
out = model.generate(**inputs, max_new_tokens=512, do_sample=False)
text = processor.batch_decode(out[:, inputs["input_ids"].shape[1]:], skip_special_tokens=True)[0]
print(text) # answer text interleaved with <|mt_start|><|mt_XXXX|><|mt_YYYY|><|mt_end|> mask tokens
Decode mask tokens → segmentation masks
The <|mt_...|> tokens are decoded to masks by the VQ-SAM2 mask tokenizer. Use the
reference implementation in the CycleGRPO repo rather than re-deriving it — see
evaluation/groundingsuite/qwen3vl_groundingsuite_infer.py (or
evaluation/dlc_bench/inference.py), which build the decoder and run the decode loop:
from projects.transformers.vq_sam2 import VQ_SAM2, VQ_SAM2Config, SAM2Config
# Those scripts also contain the `DirectResize` preprocessor, the mt-token parsing
# (extract_mt_token_ids / fix_mt_format), and the `VQ_SAM2.forward_with_codes(...)`
# decode step (codebook size 256, depth 2). Reuse them directly.
The decoder weights — mask_tokenizer_256x2.pth and sam2.1_hiera_large.pt — come
from the base model Qwen3-VL-4B-SAMTok.
License
Released under Apache-2.0. Derived from Qwen3-VL-4B-SAMTok; use is also subject to the base model's license and terms.