ZDTaichu5.0-9B GGUF

Community GGUF quantizations of TaichuAI/ZDTaichu5.0-9B.

About ZDTaichu5.0-9B

ZDTaichu5.0-9B is a multimodal foundation model for visual understanding, spatial reasoning, agentic tool use, and embodied-AI workloads. The upstream model combines a Qwen3.5-9B language backbone with the C-RADIOv4-H vision encoder and accepts text, one or more images, and video. The upstream card describes any-resolution visual input and up to 128K tokens of context; the GGUF metadata preserves the converted model context configuration, while practical context length depends on the runtime and available memory.

This release is for llama.cpp. The language model is provided as quantized GGUF files and the visual encoder / projector is provided separately as mmproj-ZDTaichu5.0-9B-BF16.gguf. Use the same mmproj file with each language-model quant when passing images or video. The official upstream model card and Transformers/vLLM instructions are available here.

Files

Fidelity measurements

The table reports next-token fidelity against the converted BF16 language-model GGUF on the hold-out wiki.valid.raw corpus, using one 2048-token chunk at context length 2048. The qwen4exp llama-perplexity runtime was used with GPU0 only. Each row is the mean reported for that evaluated chunk; this is a compact fidelity smoke measurement, not a task or visual-understanding benchmark.

Lower Mean KLD, ΔPPL, and RMS Δp, and higher Top-1 agreement, indicate closer next-token behavior to BF16. The lower tiers are included for constrained hardware and should be treated as experimental; IQ1_M and especially Q1_0 are not recommended when capability retention matters. Task-specific and multimodal behavior can differ from this text-only measurement. The compact machine-readable source is reproducibility/quality-summary.tsv, with commands and hashes in reproducibility/manifest.md.

Quick start

Text-only smoke test (the measured sweet-spot candidate):

./llama-cli \
  -m ZDTaichu5.0-9B-IQ4_NL.gguf \
  --no-mmproj \
  --chat-template-file chat_template.jinja \
  --jinja \
  --reasoning off \
  -p 'Answer briefly in English: What is GGUF and why is it useful for running language models locally?' \
  -n 128 -c 4096 -ngl 99

Multimodal input uses the BF16 projector alongside the language model:

./llama-cli \
  -m ZDTaichu5.0-9B-Q4_K_M.gguf \
  --mmproj mmproj-ZDTaichu5.0-9B-BF16.gguf \
  --image image.jpg \
  --chat-template-file chat_template.jinja \
  --jinja \
  --reasoning off \
  -p 'Describe the visible objects and their spatial relationships in this image.' \
  -n 128 -c 4096 -ngl 99

Adjust -ngl, -c, batch sizes, and projector offload options for the target hardware. Image/video support depends on a llama.cpp build with the mtmd runtime enabled.

Reproducibility and validation

The conversion source is locked to upstream revision a22afd15a3f85659f103caa659ec4aa9500a998e. Quantization was performed directly from the converted BF16 language-model GGUF, never from another quantized file. The model-specific imatrix combines WikiText-2 train text and the included English/Chinese spatial, JSON/tool, code, and agent calibration text. wiki.valid.raw and wiki.test.raw were held out from imatrix creation. All nineteen text variants passed deterministic load/generate smoke tests. Q4_K_M also passed a multimodal smoke with the BF16 mmproj and an image; the runtime reported text, vision, and video modalities and generated a grounded description.

The compact commands, hashes, calibration split information, converter/runtime commits, and validation profile are in reproducibility/manifest.md. Artifact checksums are in SHA256SUMS.txt. Raw conversion, quantization, smoke-test, and evaluation logs are kept locally under reports/ and are not part of this public package.

License and attribution

The upstream weights are distributed under the NVIDIA Open Model License Agreement. The Qwen3.5 language backbone and C-RADIOv4-H components have additional notices; see the included NOTICE, THIRD_PARTY_LICENSES.md, and LICENSE. Redistribution should retain those files and their attribution requirements.

These are community GGUF quantizations, not an official ZDTaichu5.0-9B release or endorsement.