Grug 67B A2B Datakit SFT 262K — 2026.09.11

Research artifact. This model has not been properly tested or evaluated and is not validated for production use. Testing consists of smoke tests and exploratory probes. The 262K context length is a configuration value; long-context performance remains untested.

This repository contains a BF16 Grug mixture-of-experts checkpoint after supervised fine-tuning (SFT). Its chat template defines thinking and nonthinking modes and XML tool-call formatting. A few examples of these behaviors were smoke-tested.

Model and provenance

Step 159,000 is the native checkpoint counter, not the number of SFT-only updates. The run's data-mixture label is datakit80_pretrain20; a complete training-data inventory is not provided here.

The current inference metadata is pinned at revision 929b18ebfb6c78328527f670f569edb6be86f53f. The weights are unchanged from revision b03c6c3f56458fb67a5f58290dd569d9622b2c5b. File hashes were checked after transfer to Hugging Face. This checks transfer integrity; numerical equivalence to the native checkpoint has not been established.

Chat and generation settings

Omitting enable_thinking defaults to thinking. To select nonthinking mode, include this field in the JSON body of a vLLM chat request:

{"chat_template_kwargs": {"enable_thinking": false}}

For local prompt formatting, pass enable_thinking=False to the tokenizer:

from transformers import AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained(
    "open-athena/Grug-67B-A2B-Datakit-SFT-262K-2026.09.11",
    revision="929b18ebfb6c78328527f670f569edb6be86f53f",
)
messages = [{"role": "user", "content": "What is 17 times 23?"}]
prompt_ids = tokenizer.apply_chat_template(
    messages, tokenize=True, return_dict=False,
    add_generation_prompt=True, enable_thinking=False,
)

The generation configuration stops on both <|end_of_text|> (128001) and <|eot_id|> (128009). The tokenizer's EOS remains <|end_of_text|> (128001). Preserve the full generation stop list when configuring another runtime.

Thinking output uses <|start_think|> (128002) and <|end_think|> (128003). Decode with skip_special_tokens=False when these boundaries are needed for reasoning extraction. Tools use JSON inside <tool_call>...</tool_call>; tool arguments may be objects or JSON strings in the input conversation.

chat_template.jinja adds the thinking default, JSON-string argument handling, and text-part normalization to the training template. The export-time audit records template and tokenizer checks. Its model_generation_tested: false field predates the smoke test described below.

Serving

This architecture requires a runtime with Grug MoE support. The tested runtime was Marin's vLLM fork at f0d7cc7f5874, using one node with eight H100 GPUs. Stock Transformers model loading has not been validated for this release.

The earlier smoke test used the original tokenizer revision. This command selects the metadata fix, validated for token parity, with the same 8K serving limit:

vllm serve open-athena/Grug-67B-A2B-Datakit-SFT-262K-2026.09.11 \
  --revision 929b18ebfb6c78328527f670f569edb6be86f53f \
  --dtype bfloat16 --tensor-parallel-size 1 --data-parallel-size 8 \
  --enable-expert-parallel --max-model-len 8192 --max-num-seqs 8 \
  --gpu-memory-utilization 0.8 --enforce-eager --generation-config auto \
  --chat-template-content-format string

The original smoke test used revision b03c6c3f56458fb67a5f58290dd569d9622b2c5b and client-side XML tool parsing. The current template normalizes text-part messages to the training string format and preserves assistant text beside tool calls. Template validation checks token parity; it does not measure model quality. Servers using an older revision should set --chat-template-content-format string or upgrade and reload the tokenizer.

Evaluation and limitations

This model is a research artifact and has not been properly tested or evaluated. The smoke test provides no reliable estimate of benchmark performance, safety, or suitability for deployment.

Six requests ran on 8×H100 on September 11, 2026, with an 8K serving limit:

• Default and explicit thinking produced exactly the same response text for the tested prompt; nonthinking omitted thinking delimiters. Each answered the same arithmetic question correctly.
• Multi-turn recall returned the requested code word but failed the instruction to return only that word.
• One XML tool-call round trip produced valid arguments and a final answer consistent with the supplied tool result.
• All six responses stopped on EOT 128009 without request-level stop overrides.

Exploratory tool-use probes subsequently found premature completion, repeated calls, and fabricated tool-output text. Long-context performance remains unvalidated. This card reports no benchmark scores.

License

The model materials in this repository are released under the OpenMDW License Agreement, version 1.1. The license text is copied unchanged from the official OpenMDW repository.