Affine SN120 candidate p28 (HiAlpha-GRPO on p24 / t6)
Reason-v4 GRPO checkpoint trained in the /mining/ralph loop on an 8×H100
PCIe box. Pushed for a live SN120 duel against the sitting king
vera6/affine-5g4yy75zuz-t6 @ 8e3f1695e058837ed80fec3238ff439fdc2d0f0e
(reign 36). Architecture is stock Qwen3_5MoeForConditionalGeneration
(hidden 2048, 40 layers, 256 experts / 8 active). No custom modeling code,
no auto_map, no *.py.
Lineage
- Live king
vera6/affine-5g4yy75zuz-t6@8e3f1695e058837ed80fec3238ff439fdc2d0f0e - p24 — HiAlpha-GRPO on that king (
cand_p24-grpo-king-affine-5g4yy75zu). Local v4 screen vs t6: n=400, margin −0.00009, z −0.09, med |z| 139, B-pass 0.46 (tie / BELOW_BAR). - p28 — same recipe on p24 (this repo). Merged LoRA → bf16 safetensors
via
/mining/sim/merge_lora2.py(nonzero delta + 333 visual tensors).
Experiment path: /mining/ralph/runs/p28-grpo-p24-grpo-king-affine-5g4
Method
Teacher-anchored Reason v4 GRPO (train_reason_grpo.py).
Per-sample reward:
a_i = lpC(y_i | z_A) − lpC(y_i | ∅) # k=3 teacher refs
Reason = τ · log((1/k) · Σ exp(a_i/τ)) # τ=0.03
reward = Reason + length_shape(|z|) # penalize |z|≥220 only
Winner-only tail-boost 2.0 on the best group member. Ranked quantity is the
thought z (action y is not the score). Teacher is the frozen Affine
teacher zai-org/GLM-4.5-Air-FP8, two local vLLM TP=2 endpoints.
Data
- Pool:
/mining/ralph/data/grpo.jsonl - Kept 871 / 1200 rows (char budget 12288)
- Affine public turn corpus D (SWE-style coding turns), same chat contract
the validator uses (
THOUGHT+ last closed bash fence)
Hyperparameters
| knob | value |
|---|---|
| method | HiAlpha-GRPO (LoRA) |
| lr | 5e-6 |
| LoRA r / α / dropout | 16 / 128 / 0.05 |
| target modules | q,k,v,o,gate,up,down _proj |
| group size G | 4 |
| steps | 200 |
| max new tokens (z sample) | 64 |
| max seq | 6144 |
| KL coef | 0.0 |
| tail-boost | 2.0 winner-only |
| length shape | penalty on |z|≥220, no length bonus |
| τ / k_refs | 0.03 / 3 |
| dtype | bfloat16 |
Train wall: 4524 s (~75 min). Last-20 mean reward 0.044. Trainable 8.36M /
34.7B (0.024%). GPUs 4–7 for LoRA (device_map=auto); teachers on 0–1 and 2–3.
Local sim vs live king (Reason v4, k=3, τ=0.03)
Screened 2026-08-19 against the same king SHA still sitting at push time
(vera6/affine-5g4yy75zuz-t6 @ 8e3f1695…). gate_screen.py / fast_screen
n=160:
| cand | king | |
|---|---|---|
| mean Reason | 0.01184 | 0.00995 |
| med |z| | 144 | 145 |
| B-pass | 0.46 | 0.49 |
- margin +0.00189 · SE 0.00149 · z +1.27 · n=160
- win_frac 0.40 · median Δ 0 · p90 Δ +0.00943 · top-5% contrib +0.00288
- gates: thought length PASS (≥80), causality B PASS (≥0.30)
- live crown bar = max(2·SE, δ=0.002) ≈ 0.0030 on this slice
- local verdict: BELOW_BAR (n=160 is a rejector vs live n_turns=1300)
p28 was the closest this-loop candidate vs t6. Live duel is n=1300 k=3; this n=160 slice does not prove a crown.
Serving
Same family the eval pod loads: Qwen3.5-MoE, canonical sharded safetensors
(model-00001-of-00002 + model-00002-of-00002 + model.safetensors.index.json
model-visual.safetensors). No--trust-remote-code. Local screen served this merge with vLLM TP=2.
Hardware
8× NVIDIA H100 80GB PCIe. Merge on CPU (device_map=cpu).