Kaveri ST-GRPO 0.5B

Kaveri ST-GRPO 0.5B is an experimental coding post-training checkpoint derived from Qwen/Qwen2-0.5B-Instruct.

It uses stock TRL GRPO with an experimental Signed Tensor GRPO (ST-GRPO) reward-shaping layer. Reward and punishment are maintained as separate signals. Third-order tensors encode interactions among triples of sampled solutions.

ST-GRPO

For each prompt, the trainer samples 8 rollouts, giving 56 unordered triples.

Positive tensor channels:

• verified quality
• quality-weighted structural/strategy diversity
• complementary official-test coverage

Punishment tensor channels:

• verifier-derived penalty
• collapse of bad structurally similar solutions
• common official-test failure
• lexical repetition/degeneration

R_i = a*z(r_i+) - b*z(r_i-) + lp*z(C_i+) - ln*z(C_i-)

The signed scalar is returned to TRL, which performs normal group-relative advantage computation and policy optimization.

Training setup

• Base model: Qwen/Qwen2-0.5B-Instruct
• Dataset: open-r1/codeforces, verifiable-prompts
• Language: C++17
• Trainer: TRL GRPOTrainer
• Generations per prompt: 8
• Third-order triples per prompt: 56
• Generation batch size: 16
• Optimizer steps: 150
• Max completion length: 256
• LoRA rank: 8
• LoRA alpha: 16
• LoRA targets: q_proj, v_proj
• Positive weight: 1.0
• Punishment weight: 0.85
• Tensor positive weight: 0.35
• Tensor punishment weight: 0.45
• Verifier: local g++ compile + official tests

Files

The repository root contains the merged standalone model. The original LoRA adapter is preserved in lora_adapter/.

Status

Experimental research checkpoint. Evaluate independently before drawing benchmark conclusions.

mmlu

43.94%