GLM-5.2-SIQ-Fruit-bf16
The plain-BF16, CPU-runnable twin of GLM-5.2-SIQ-Fruit. It contains the same QNOISE-annealed 5.04B-parameter checkpoint—0.46B parameters active per token—but replaces the custom SIQ expert representation with ordinary BF16 tensors.
This repository loads with stock Transformers and needs no GPU. It is the reference path for model behavior and CPU integration; it does not exercise SIQ/Trellis dequantization, sparse DSA attention, low-precision KV caches, or MTP speculative decoding.
Measured CPU run
Intel Core i7-14700K, 20 Torch threads, BF16, Transformers 5.14.1; 128-token greedy decode after a 16-token warmup:
| measurement | result |
|---|---|
| decode | 33.12 tok/s |
| warm-cache load | 2.51 s |
| resident memory after load | 16.01 GiB RSS |
| peak resident memory | 16.76 GiB RSS |
An earlier card estimated “about 10 GB RAM.” That was not an end-to-end process measurement and has been withdrawn.
Usage
import torch
from transformers import AutoConfig, AutoModelForCausalLM, AutoTokenizer
model_id = "malaiwah/GLM-5.2-SIQ-Fruit-bf16"
torch.set_num_threads(20)
config = AutoConfig.from_pretrained(model_id)
assert config.rope_theta == config.rope_parameters["rope_theta"] == 500000.0
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
model_id,
config=config,
dtype=torch.bfloat16,
).eval()
inputs = tokenizer("Once upon a time", return_tensors="pt")
output = model.generate(**inputs, max_new_tokens=40, do_sample=False)
print(tokenizer.decode(output[0], skip_special_tokens=True))
Expected output begins with a TinyStories-style continuation about a little girl named Lily; exact wording can vary with library kernels and version.
What loads
- Exercised: dense MLA attention, GLM tokenizer, and the real 256-expert top-8 MoE weights.
- Ignored by stock Transformers: DSA indexer tensors and the MTP draft layer. They are reported as unexpected keys. This is intentional for this compatibility path; 4.57B of the 5.04B parameters load.
- Production path: use the SIQ release with a compatible b12x/SparkInfer + vLLM build to exercise Trellis experts, sparse MLA, fp8/nvfp4 KV, and MTP.
Release correction
Revision b97f91d222906142e46827a010a5caf5a9e35928 encoded the trained RoPE
theta incorrectly: it retained 8,000,000 in the nested field and omitted the
legacy top-level field. The 2026-08-07 correction writes 500,000 to both
locations and regenerates MANIFEST.sha256. Weight shards did not change.
The fail-closed assertion in the usage example protects pinned or cached stale
configurations.
The export also carries the same trainer-to-serving RoPE channel permutation
and MTP eh_proj input-half conversion as the SIQ artifact; those
transformations preserve the trained function before stock Transformers drops
the unsupported indexer/MTP modules.
Limitations
- The model is a serving proxy and CI fixture, not a general assistant.
- CPU generation uses dense attention and therefore does not validate the DSA indexer or long-context sparse-attention path.
- The measured speed and memory numbers are for the exact hardware/software setup above; they are not hardware-independent guarantees.
Reproducibility and integrity
MANIFEST.sha256 authenticates all serving files except the card and Git
attributes. The exporter invocation uses FRUIT_BF16=1; source checkpoint,
training recipe, gauntlet, and review evidence are documented in
proxy-fruit and the
base model card.
License
MIT; see LICENSE. Earlier model-card metadata incorrectly reported Apache-2.0; the packaged license file has always been MIT.