Qwen3-4B-OutsideTheBox-RL
This repository provides the outcome-based reinforcement learning checkpoint developed for the paper Thinking Outside the Box: Can Language Models Rely on External Guidance Selectively?
Model Description
The model is initialized from Qwen3-4B-OutsideTheBox-SFT and further trained with direct task-outcome rewards on workflow-conditioned mathematical reasoning prompts. Its RL data mixes prompts containing misleading workflows with prompts containing helpful workflows to improve selective reliance on external guidance.
Intended Use
This checkpoint is intended for research on mathematical reasoning, external workflow utilization, and robustness to incorrect guidance. It corresponds to the outcome-based RL stage reported in the paper.
Usage
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "ElvisWang111/Qwen3-4B-OutsideTheBox-RL"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
model_id,
torch_dtype="auto",
device_map="auto",
)
Training Lineage
- Base model:
Qwen/Qwen3-4B - Initialization:
ElvisWang111/Qwen3-4B-OutsideTheBox-SFT - Training stage: outcome-based reinforcement learning
- Domain: workflow-conditioned mathematical reasoning