Ornith-1.0-35B-TQ3_4S

Ornith-1.0-35B-TQ3_4S is a compact TurboQuant GGUF build of deepreinforce-ai/Ornith-1.0-35B.

Required Runtime

This model uses the custom TQ3_4S tensor type and requires turbo-tan/llama.cpp-tq3.

Stock llama.cpp builds without TurboQuant support cannot load this model.

This is the standard 35B model, not an MTP release. Do not add draft-MTP speculative-decoding flags.

Ornith-1.0-35B is a Qwen-VL multimodal model. Vision is optional — load the mmproj projector to enable image input (see Vision (Multimodal)).

Files

• Ornith-1.0-35B-TQ3_4S.gguf - main model, 13.00 GiB
• mmproj-f16.gguf - vision projector (mmproj), F16, 0.84 GiB
• thumbnail.png - model card banner
• benchmark.png - benchmark comparison card
• benchmark_notes.md - compact benchmark provenance and comparison

Build the Required Runtime

git clone https://github.com/turbo-tan/llama.cpp-tq3
cd llama.cpp-tq3

cmake -S . -B build \
  -DCMAKE_BUILD_TYPE=Release \
  -DGGML_CUDA=ON \
  -DGGML_CUDA_FA_ALL_QUANTS=OFF \
  -DGGML_CUDA_GRAPHS=ON

cmake --build build --target llama-server -j

For an RTX 3090, -DCMAKE_CUDA_ARCHITECTURES=86 may be added explicitly. Use the architecture matching your GPU on other systems.

Recommended Runtime

Validated on an NVIDIA GeForce RTX 3090 Founders Edition with 24 GiB VRAM:

./build/bin/llama-server \
  -m Ornith-1.0-35B-TQ3_4S.gguf \
  --alias Ornith-1.0-35B-TQ3_4S \
  --host 127.0.0.1 --port 8080 \
  -c 32768 -np 1 -ngl 99 -fa on \
  -ctk q8_0 -ctv tq3_0 \
  --reasoning off --jinja

Runtime notes:

• -fa on enables flash attention at runtime.
• The validated CUDA build uses GGML_CUDA_FA_ALL_QUANTS=OFF.
• -ngl 99 fully offloads the model on supported GPUs. Avoid partial offload when comparing the published speed.
• Reduce context from 32768 if the available VRAM is lower than 24 GiB.

Quick Smoke Test

curl -s http://127.0.0.1:8080/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{"model":"Ornith-1.0-35B-TQ3_4S","messages":[{"role":"user","content":"Write ONLY the word ok."}],"max_tokens":16,"temperature":0.0}'

Expected assistant content:

ok

Vision (Multimodal)

Ornith-1.0-35B is a Qwen-VL multimodal model. To enable image input, add the vision projector with --mmproj:

./build/bin/llama-server \
  -m Ornith-1.0-35B-TQ3_4S.gguf \
  --mmproj mmproj-f16.gguf \
  --alias Ornith-1.0-35B-TQ3_4S \
  --host 127.0.0.1 --port 8080 \
  -c 32768 -np 1 -ngl 99 -fa on \
  -ctk q8_0 -ctv tq3_0 \
  --reasoning off --jinja

Send an image using the OpenAI vision schema (image_url with a base64 data URI):

curl -s http://127.0.0.1:8080/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{"model":"Ornith-1.0-35B-TQ3_4S","messages":[{"role":"user","content":[
        {"type":"text","text":"Describe this image."},
        {"type":"image_url","image_url":{"url":"data:image/png;base64,<BASE64_IMAGE>"}}
      ]}],"max_tokens":256,"temperature":0.0}'

Vision notes:

• The mmproj is only needed for image input; text-only use does not require it.
• Qwen-VL grounding works best with at least 1024 image tokens. If accuracy is poor on detailed images, add --image-min-tokens 1024.

Benchmark Summary

Local BenchLoop and Hard86 comparison on an RTX 3090 FE using the TurboTan runtime and the launch settings above:

The field score uses 0.85 * task_score + 0.15 * size_factor, with size normalized to the smallest displayed 35B model. Hard86 is weighted at 2x, EasyCode at 0.5x, and the remaining benchmark categories at 1x.

Validation Notes

• Benchmark results are local measurements, not claims from the parent model repository.

License

Use is subject to the base model license and the licenses of turbo-tan/llama.cpp-tq3.