r2egym_8b_rope_65k-step17
RL-trained Qwen3-8B on R2EGym tasks (65k context with YaRN rope scaling, 17 steps).
Training Details
| Parameter | Value |
|---|---|
| Base model | laion/GLM-4_7-r2egym_sandboxes-maxeps-131k-lc (Qwen3-8B SFT, long-context) |
| Dataset | R2EGym GPT5/Codex solved tasks (1,785 tasks) |
| Algorithm | RLOO-N (Leave-One-Out with neutral masking) |
| Learning rate | 3.0e-5 |
| Train batch size | 32 |
| Samples per prompt | 8 |
| Max episodes | 64 |
| Max generate length | 8,192 tokens |
| Max input tokens | 57,344 |
| Max model length | 65,536 |
| Rope scaling | YaRN (factor=4.0, original_max_pos_emb=32,768) |
| KL loss | Disabled |
| Reward shaping | Enabled (pass_ratio) |
| Staleness steps | 16 |
| Policy nodes | 2 (8 GPUs, FSDP2) |
| Inference engines | 20 (TP=1) |
| Training steps | 17 |
| Framework | BenSkyRL + Harbor |
Usage
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained("laion/r2egym_8b_rope_65k-step17")
tokenizer = AutoTokenizer.from_pretrained("laion/r2egym_8b_rope_65k-step17")