Grug 67B A2B BFCL Recovery
Research artifact. This model has not been properly tested or evaluated and is not necessarily secure. Use at your own risk in production settings.
These weights were produced by prior DPO run .304, which performed two optimizer updates; hf/step-1 is its zero-indexed final export label. A separate extended native MarinSkyRL DPO experiment (original run .357, continuation .365) used this export as its initializer. The initializer was the highest-scoring completed checkpoint in that extended experiment. The published model received zero updates from the extended run. All original weight shards are preserved; the publication manifest was refreshed from the export's actual files.
The architecture is Grug-67B-A2B, with approximately 67 billion total and 2 billion active parameters. The export contains 39 BF16 safetensors shards and 502 indexed tensors. Its inherited manifest carries producer identity sha256:7232ea204416bd1576dadc659caec3bbad3877fcdfd27b991d9e39f4f48e1335, which does not describe the re-exported weights. Levanter copied this reference metadata without regenerating the manifest. Publication checks the original .304 receipt's shard sizes, configuration and tensor headers, verifies that shards predate that receipt, then records fresh full-file hashes and checks them against the Hub. Original .304 full-weight hashes were not recorded, so this is not a historical byte-for-byte comparison.
Training provenance
The model descends from the Antidoom/RLVR1 step-12 checkpoint through recovered warm-start checkpoint .26 and the two-update .304 DPO run. The .304 recipe used batch size 16 preference pairs, learning rate 4e-6, and DPO beta 0.1. The available preference cache contained 2,352 pairs; the two updates consumed 32 pairs. The native extended experiment initialized policy and fixed reference from this exported model with a fresh optimizer.
Limited evaluation
On the fixed 123-task BFCL-Parity holdout using Pi 0.87.0 and Daytona, the initializer scored 34/123 (27.64%) in the extended experiment's pretraining evaluation. All 123 tasks were attempted and scored. No later completed checkpoint from that experiment exceeded this score. A separate earlier evaluation of the same export scored 35/123 (28.46%); these are separate stochastic evaluations, not pooled results.
This checkpoint was selected for recovery and inspection; it did not pass the campaign's 30% accuracy bar. Later completed checkpoints scored lower in the extended experiment's evaluation: 10/123 after four updates and 0/123 after eight and 32 updates. The extended experiment was cancelled. These limited evaluations do not establish general coding ability, safety, or production suitability.
Runtime and data
Use a runtime with GrugMoe support, such as the Marin-compatible Grug vLLM or MarinSkyRL stack. Compatibility with an unmodified stock Transformers installation was not verified for this release. Tokenizer files and the chat template are included without modification.
This is a model-only release. Preference data, agent traces, and captured harness scaffold are not included in this public repository. Dataset repositories remain private.
Public campaign evidence and limitations. BFCL-Parity is the 123-task Harbor adaptation, and Pi is its agent harness; these are not standard BFCL leaderboard scores.
License
The model materials distributed here use OpenMDW 1.1. See the complete LICENSE.