Ornith-1.5-35B-A3B-Spark-Abliterated-VL-MTP-NVFP4

This model provides a vLLM recipe that runs on DGX spark and similar powered machines (Blackwell+ with ~44GB+ VRAM) with an Abliterated Ornith 1.5 35B which has Vision Tower and MTP, in NVFP4.

Merged model:

• Language core (LM + lm_head + embed + norm): from pottokao/Ornith-1.5-35B-A3B-abliterated-NVFP4-DFlash (abliterated), renamed model.* -> model.language_model.*
• Vision tower + merger: from ornith-ai/Ornith-1.5-35B-A3B-NVFP4 (model.visual.*, 333 tensors)
• MTP (multi-token prediction): from ornith-ai/Ornith-1.5-35B-A3B-NVFP4 (mtp.*, 785 tensors, unquantized per quantization_config.ignore)
• Config/tokenizer/processor: from ornith-ai (multimodal Qwen3_5MoeForConditionalGeneration)
• DFlash draft: preserved from pottokao under dflash_draft/ for optional --speculative-config method=dflash

Total tensors: 94393 (same layout as ornith-ai, 3 shards).

Use — tested on 1x DGX Spark, single-stream 70+ tok/s, 16-stream ~400 tok/s

OMP_NUM_THREADS=4 MAX_JOBS=4 vllm serve \
    jnjn/Ornith-1.5-35B-A3B-Spark-Abliterated-VL-MTP-NVFP4 \
    --load-format fastsafetensors \
    --port 8000 \
    --tool-call-parser qwen3_coder \
    --reasoning-parser qwen3 \
    --enable-auto-tool-choice \
    --served-model-name ornith-1.5-35b-a3b ornith qwen-3.x-moe qwen3 qwen \
    --gpu_memory_utilization 0.45 \
    --max_model_len 262144 \
    --max-num-seqs 32 \
    --max-num-batched-tokens 8192 \
    --trust-remote-code \
    --async-scheduling \
    --mamba-cache-mode align \
    --mamba-block-size 8 \
    --enable-prefix-caching \
    --speculative-config '{"method":"mtp","num_speculative_tokens":3}'