ALFWorld-Qwen3-8B-GiGPO
A Qwen/Qwen3-8B policy trained with GRPO on ALFWorld text-based embodied household tasks, with the GiGPO estimator: episode-level group advantages plus discounted step-level group advantages (gamma 0.95, step weight 1.0, mode mean_std_norm).
The repository root holds the final policy, at training step 400.
Trained 400/400 steps over 117 hours on 4 GPUs, finishing 2026-08-13. Training-rollout success rate averaged 86.8% over the last 50 steps, against 76.0% for the outcome-GRPO arm (W&B 0f5r5p0u, Alfworld_Qwen3-8B_outcome) over the same window. Per task type at step 400: pick_and_place 1.00, pick_heat_then_place 1.00, pick_cool_then_place 1.00, pick_clean_then_place 0.92, pick_two_obj_and_place 0.83.
Two caveats on that comparison. The outcome arm ran with max_prompt_length=2048 against this run's 4096 (it predates commit d3fa8424); neither run ever clipped a prompt, but filter_overlong_prompts=True may have dropped different samples. And gamma differs (1.0 vs 0.95), which is inherent to the estimator rather than a confound. n=1 per arm. The outcome arm has no checkpoints or rollouts on this host — it was trained on huawei-job-001 and survives here only as W&B history.
Length caps, recorded so the prompt-cap difference is not re-derived later: neither arm ever clipped a prompt. Max prompt_length/max over all steps is 1641 tokens for GiGPO (cap 4096) and 1772 for outcome (cap 2048), and prompt_length/clip_ratio is 0.0000 at every step of both. The standalone_eval alfworld_qwen3 preset evaluates at a 2048 prompt cap, which is therefore non-binding for both arms and does not favour either. Response truncation against the shared 2048 response cap is confined to early training — mean response_length/clip_ratio is 1.6% (GiGPO) and 1.4% (outcome) over steps 1-50, peaking near 17%, then 0.1% or below for the rest of both runs.
That truncated early regime does not contaminate this pair's GiGPO advantage, because on these arms the advantage is not an early one. Training-rollout success, GiGPO minus outcome, by window: 1-25 +1.7, 26-50 +1.2, 51-75 +5.0, 76-100 +9.6, 101-150 +5.6, 151-200 +3.0, 201-300 +0.1, 301-350 +5.9, 351-399 +10.9 points. The gap is ~1 point through steps 1-50 where truncation peaks, only opens from step ~76 onward once clipping has fallen to ~0.02%, closes to nothing over 201-300, and reopens late. Any claim that GiGPO helps early would need the truncation caveat; the claim supported here is that it helps late.
Usage
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained("wckwan/ALFWorld-Qwen3-8B-GiGPO")
tokenizer = AutoTokenizer.from_pretrained("wckwan/ALFWorld-Qwen3-8B-GiGPO")
# An intermediate checkpoint
model_step = AutoModelForCausalLM.from_pretrained("wckwan/ALFWorld-Qwen3-8B-GiGPO", subfolder="step_20")
Checkpoints
step_20/step_40/step_60/step_80/step_100/step_120/step_140/step_160/step_180/step_200/step_220/step_240/step_260/step_280/step_300/step_320/step_340/step_360/step_380/
Raw FSDP checkpoints
fsdp/global_step_<N>/ holds the unmerged verl FSDP checkpoint (sharded fp32
model state, optimizer state and extra state) for step(s) 400. These are for
resuming training, not for inference — use the merged exports above to load a
policy.