Qwen3.5-4B-Opus-Distilled-Heretic-Thinking-Multistage-SFT-v1.0

Qwen3.5-4B-Opus-Distilled-Heretic-Thinking-Multistage-SFT-v1.0 is a reasoning-capable 4B-parameter language model built on top of Qwen/Qwen3.5-4B. The model was trained using a multi-stage supervised fine-tuning (SFT) pipeline on approximately 6,000 coding and STEM-focused Opus reasoning traces, along with additional high-quality reasoning traces, to improve long-form reasoning, coding, mathematical problem solving, scientific analysis, and instruction-following capabilities.

[!NOTE] This model is an experimental release and may generate unexpected behaviors or reasoning artifacts in certain scenarios.

Key Highlights

• Qwen 3.5 Foundation: Built directly on top of Qwen/Qwen3.5-4B.
• Multi-Stage SFT Training: Trained using a multi-stage supervised fine-tuning pipeline to progressively improve reasoning performance.
• Opus Reasoning Distillation: Distilled from approximately 6,000 coding and STEM-focused Opus reasoning traces.
• General Reasoning Enhancement: Further trained on additional high-quality reasoning traces to strengthen long-form reasoning, instruction following, and multi-step analytical capabilities.
• Research-Focused Release: Designed for reasoning research, experimentation, and evaluation.
• Efficient 4B Deployment: Suitable for local inference and research environments.

Quick Start with Transformers

pip install transformers
pip install accelerate

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

model = AutoModelForCausalLM.from_pretrained(
    "prithivMLmods/Qwen3.5-4B-Opus-Distilled-Heretic-Thinking-Multistage-SFT-v1.0",
    torch_dtype="auto",
    device_map="auto"
)

tokenizer = AutoTokenizer.from_pretrained(
    "prithivMLmods/Qwen3.5-4B-Opus-Distilled-Heretic-Thinking-Multistage-SFT-v1.0"
)

messages = [
    {
        "role": "user",
        "content": "Explain how a transformer model processes text."
    }
]

inputs = tokenizer.apply_chat_template(
    messages,
    tokenize=True,
    add_generation_prompt=True,
    return_tensors="pt"
).to(model.device)

outputs = model.generate(
    inputs,
    max_new_tokens=512
)

print(
    tokenizer.decode(
        outputs[0][inputs.shape[-1]:],
        skip_special_tokens=True
    )
)

Training Details

Intended Use

• Reasoning Research: Studying reasoning distillation and multi-stage supervised fine-tuning techniques.
• Coding Assistance: Improving code understanding, generation, and debugging through distilled reasoning.
• STEM Reasoning: Solving mathematical, scientific, and engineering problems with structured reasoning.
• Instruction Following: Evaluating and improving multi-step instruction-following capabilities.
• Local Deployment: Running efficient 4B reasoning models in research and experimentation environments.

Limitations

• Experimental Model: Behavior may differ from the base model in certain scenarios.
• Reasoning Artifacts: Complex reasoning chains may occasionally produce incorrect intermediate steps or conclusions.
• Training Biases: Performance reflects the characteristics and coverage of the reasoning data used during training.

Acknowledgements

• Qwen/Qwen3.5-4B: Base model used for this project.

• TRL - Transformers Reinforcement Learning: TRL is a full stack library providing tools to train transformer language models with methods including Supervised Fine-Tuning (SFT), Group Relative Policy Optimization (GRPO), Direct Preference Optimization (DPO), Reward Modeling, and more.

• Transformers: Transformers provides state-of-the-art machine learning models for text, computer vision, audio, video, and multimodal tasks, supporting both inference and training.