Ornith 1.5 35B A3B NVFP4 512K — GGUF

This repository contains a GGUF conversion of Ornith AI's Ornith-1.5-35B-A3B-NVFP4 checkpoint for use with compatible GGML/llama.cpp-based inference engines.

The purpose of this conversion is to preserve the model's native NVFP4 mixed-precision representation in GGUF rather than requantizing the model into another quantization format.

The model's context length has been extended to 524288 tokens (512K) using YaRN with a ROPE scaling factor of 2, allowing it to handle significantly longer sequences than the original checkpoint.

Model

• Base model: ornith-ai/Ornith-1.5-35B-A3B-NVFP4

• Architecture: Qwen3.5 MoE

• Parameters: 35B total / ~3B active

• Quantization: NVFP4

• Format: GGUF

• Context length: 524288 tokens (512K)

• Intended runtime: llama.cpp and other runtimes supporting GGUF NVFP4 tensors

Conversion

The original Hugging Face Safetensors checkpoint was converted using the convert_hf_to_gguf.py tooling from llama.cpp.

The conversion preserves the source checkpoint's NVFP4 tensors rather than converting the model to FP16, BF16, or FP32 and subsequently requantizing it.

Context length extension to 524288 tokens was applied using YaRN with a ROPE scaling factor of 2.

Ornith-1.5-35B-A3B-NVFP4
             │
             │ Hugging Face Safetensors
             ▼
convert_hf_to_gguf.py
             │
             │ Native GGUF NVFP4 representation
             ▼
Ornith-1.5-35B-A3B-NVFP4.gguf
             │
             │ YaRN ROPE scaling factor 2
             ▼
Ornith-1.5-35B-A3B-NVFP4-512K.gguf
             │
             ▼
          llama.cpp

Testing

The following settings have been tested on an RTX 5090 running llama.cpp:

ctx-size:           524288
flash-attn:         on
temp:               0.6
top-p:              0.95
top-k:              20
image-min-tokens:   1024
reasoning-preserve: enabled
cache-type-k:       q8_0
cache-type-v:       q8_0