Sherry 35B-A3B 0.1 GRPO Preview

Merged GRPO-QLoRA adapter for Qwen/Qwen3.6-35B-A3B, produced from the 200-step checkpoint (checkpoint-200).

This is a preview release. The available evaluation runs were not used as a quality benchmark because their short smoke generations reached the token limit, and the longer comparison run was stopped before publication. Do not interpret the training reward in the metadata as a validated capability score.

Training and merge details

• Training method: GRPO with QLoRA/LoRA.
• Adapter source: qwen36-reasoning-effort-grpo-0.1-qlora-200-12k/checkpoint-200.
• Training steps: 200.
• LoRA rank: 32.
• LoRA alpha: 64.
• LoRA dropout: 0.0.
• Target modules: q_proj, k_proj, v_proj, o_proj, shared_expert.gate_proj, shared_expert.up_proj, shared_expert.down_proj.
• Trainable token IDs: 248077, 248078, 248079.
• Base precision during merge: bfloat16.
• Published weights: safetensors, split into 15 shards.
• Merge operation: PEFT merge_and_unload().

The original adapter artifacts are preserved separately from this merged release. The published safetensors use the canonical Qwen 3.6 packed-MoE layout required by current inference runtimes.

Reasoning effort interface

The tokenizer includes three project-specific special tokens:

• <|reasoning_effort_low|> (token ID 248077)
• <|reasoning_effort_medium|> (token ID 248078)
• <|reasoning_effort_high|> (token ID 248079)

The published chat template accepts reasoning_effort="low", "medium", or "high" and emits the corresponding token before the thinking block. No xhigh token is provided.

Example:

from transformers import AutoModelForImageTextToText, AutoTokenizer

model_id = "valendra/sherry-35b-a3b-0.1-grpo-preview"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForImageTextToText.from_pretrained(
    model_id,
    device_map="auto",
    dtype="auto",
)

inputs = tokenizer.apply_chat_template(
    [{"role": "user", "content": "Solve: 12 * 13"}],
    add_generation_prompt=True,
    enable_thinking=True,
    reasoning_effort="medium",
    return_tensors="pt",
).to(model.device)

outputs = model.generate(inputs, max_new_tokens=2048)
print(tokenizer.decode(outputs[0, inputs.shape[-1]:], skip_special_tokens=False))

For non-thinking answers, call the template with enable_thinking=False. The model is multimodal-compatible through the Qwen3.6 architecture, but this release was trained and smoke-tested primarily through its text-generation path.

Limitations

• Preview release; independent quality evaluation is incomplete.
• The logged training reward is not an accuracy or benchmark score.
• Generation length and quality may vary by reasoning-effort level and prompt.
• Use the base model's license and usage requirements.

Intended use

Research and experimentation with controllable reasoning effort. Validate outputs for your use case before deployment.