Dr. Doom
Research artifact. This model has not been comprehensively tested or evaluated and is not validated for production use. The limited de-looping and correctness evaluations below do not establish safety, general model quality, or long-context performance. The inherited 262K context length is a configuration value; its performance remains unvalidated.
Dr. Doom is a BF16, full-weight FTPO fine-tune of Grug-67B-A2B-Datakit-SFT-262K-2026.09.21, targeting exact repetitive “doom loop” continuations. The released checkpoint is mixed-long-smoke v2, update 2, from the 1–2 October 2026 campaign.
Measured result
| Evaluation | Original SFT | Dr. Doom |
|---|---|---|
| Math loops · 512 prompts · 16K output limit | 139/512 (27.15%) | 12/512 (2.34%) |
| Math correct answers · same cohort | 232/512 (45.31%) | 283/512 (55.27%) |
| Cross-domain loops · 100 prompts · 32K output limit | 43 confirmed + 1 unassessed (43–44%) | 19/100 (19%) |
| Cross-domain verifier passes | 19 passes; 92 graded, 7 unavailable, 1 error | 26 passes; 94 graded, 6 unavailable, 0 errors |
Prompt hashes match within each comparison. These are single evaluations across different runtime and weight-loading contexts; they do not isolate each stage's causal contribution. Cross-domain verifier passes are counts, not a uniform benchmark accuracy: missing/error verification is not a verified failure. Per-domain rates, thresholds, and grading availability are in the domain table and evaluation evidence.
The detector requires at least eight exact copies of a token pattern continuing to the end of the final generated segment. It examines the full response window, with periods up to 2,048 tokens at 16K and 4,096 at 32K. It misses irregular/semantic repetition and can flag intentional repetition. Unassessed tool histories contribute to conservative loop bounds.
The campaign goal was strict <5% loops for each domain with fewer than 50 examples and <2% for each larger domain. That goal was not reached. Instruction following, coding, and tool-assisted math remain problematic; tiny-domain estimates are noisy. The selected export has no full 512-prompt math evaluation at 32K. Its final 64-prompt in-training math pilot at 32K measured 8 loops and 40 correct answers.
Mixed-round update 10 had the lowest single math result, 9/512 loops (1.76%), but lower correctness, 268/512 (52.34%). Its in-training evaluation measured 13/512 loops. The selected checkpoint offers a reasonable observed tradeoff; the campaign does not establish a statistically definitive ranking.
RL method
The campaign used online final-token preference optimization (FTPO) in MarinSkyRL, adapting the Antidoom reference. Greedy online rollouts provide exact token IDs and raw top-32 token log probabilities. When an exact repetitive suffix is detected, FTPO selects the first repetition boundary, treats the generated token as rejected, and trains alternative tokens to exceed it by a logit margin. Vocabulary-logit MSE penalties against a frozen reference constrain the change. Per-batch frequency balancing reduces domination by common rejected/chosen tokens. Empty target batches skip the optimizer update.
The final stage initialized both the policy and frozen reference from mixed-domain round update 5, with a fresh optimizer. It used math, instruction-following and coding sampling weights 2:1:1, batch size 64, one greedy response per prompt, a 2,048-token prompt budget, and up to 32,768 response tokens. The loop detector covered the whole 32K response, with period cap 4,096 and at least eight repeats. AdamW ran two updates at LR 2e-7, no weight decay, and gradient-norm clipping at 0.5.
FTPO defaults were margin 2.0, non-target MSE coefficient 0.4, target MSE coefficient 0.05, target-logit deviation tolerance 0.5, minimum relative candidate probability 0.01, and at most 20 chosen alternatives. Native task verification supplied the separately reported correctness metrics. Reward advantages, an extra KL objective, and entropy loss were disabled. This is an online full-weight adaptation of Antidoom; it is not a claim of reproducing the original study.
Training used 40 H100 GPUs on Iris/CoreWeave: 32 for colocated policy/reference actors (TP=1, PP=2, CP=1, EP=16), plus 8 rollout GPUs. The pinned final MarinSkyRL runtime was a090abe57113a375e8dbee05ea0cf7e17daa2316. Exact launch envelopes, earlier lineage stages, and source entrypoints are included in configs/ and source/.
The released model's lineage is: original SFT → standard math FTPO through update 35 → full-window branch through update 45 → fresh-reference math round update 5 → mixed-domain round update 5 → long-response FTPO update 2. Step numbers reset in fresh rounds; the release is not merely two updates from the original SFT checkpoint.
Campaign records
- Marin experiment issue: campaign report and merged PRs; lessons-learned discussion post.
- Comprehensive experiment tracker: all 17 RL attempts, ablations, failures, retained measurements, W&B links, and supporting jobs.
- Machine-readable tracker and config manifest.
- Lessons learned.
- Public gists with exact winning-lineage launch configurations.
- Public experiment tracker and lessons-learned post.
- W&B project, selected training run, saved-model math evaluation, and cross-domain evaluation.
Release manifest, weight hash verification, and storage cleanup record document the preserved release.
Model and use
Architecture: GrugMoeForCausalLM (grug_moe), 67,078,882,816 total parameters, approximately 2B active non-embedding parameters/token, 256 routed experts with top-4 routing and a shared expert. This release contains 39 BF16 safetensors shards; it is a full model, not an adapter. Architecture and tokenizer come from base revision b8c07f7df1df65525abbfdbcd1572318ba11c42f.
Use a runtime with Grug MoE support. This campaign exercised the pinned MarinSkyRL/vLLM stack; stock Transformers model loading and production serving of this HF release have not been validated. Preserve the supplied chat template and generation configuration, including both stop IDs: 128001 (<|end_of_text|>) and 128009 (<|eot_id|>). Thinking is enabled by default; chat_template_kwargs={"enable_thinking": false} selects nonthinking mode where supported.
Training-source provenance and the inherited SFT mixture are documented in the base model card. The original research-artifact limitations continue to apply. The RL campaign's short-input and limited-domain evaluations do not validate the inherited 262,144-token configured context, multilingual coverage, safety, or general instruction-following quality.
License
Model materials are released under the inherited OpenMDW License Agreement, version 1.1.