Qwen3.8-27B-Uncensored for NInfer-4090

This repository contains a single-file, mixed-precision .ninfer conversion of orcarouter/Qwen3.8-27B-Uncensored, built for UDPSendToFailed/ninfer-4090 on a 24 GB RTX 4090 (sm_89).

This is a deployment-format conversion only. It does not add training, change the upstream behavior, or independently establish the upstream model's "uncensored" or "abliterated" characteristics. The artifact is not a Safetensors or GGUF checkpoint and is not intended for Transformers, llama.cpp, or unrelated NInfer forks.

Quick facts

The conversion report records NInfer revision 6d3fd16. The verification below used the later source revision e4233d6, whose cached-token reporting change is merged upstream in UDPSendToFailed/ninfer-4090#5.

What is inside the .ninfer bundle

The file is a self-contained runtime bundle: mixed-precision model weights plus six embedded frontend resources (tokenizer.json, tokenizer and generation configs, chat_template.jinja, and image/video processor configs). Q4G64_F16S, for example, means 4-bit weights quantized in groups of 64 with FP16 scales; the Q5/Q6 variants use the same group size, while W8G32_F16S uses 8-bit weights in groups of 32.

NInfer materializes only the components selected at startup. In the verified runs, resident weights were 15.92 GiB for baseline text, 16.19 GiB with Vision, and 16.67 GiB with MTP. The 16.96 GiB artifact size therefore is not the same thing as per-mode VRAM usage; KV state, workspaces, CUDA Graphs, and concurrency add further runtime memory.

Download

hf download pyros-vault/Qwen3.8-27B-Uncensored-NInfer `
  --local-dir .\Qwen3.8-27B-Uncensored-NInfer

Verify the artifact:

Get-FileHash '.\Qwen3.8-27B-Uncensored-NInfer\Qwen3.8-27B-Uncensored.ninfer' -Algorithm SHA256

Serving with the intended NInfer runtime

Build a current native sm_89 version of UDPSendToFailed/ninfer-4090. .ninfer is a registered, model-bound format; release binaries or other forks without the matching Qwen3.8 target may not load this file.

The following serving examples use the same conservative memory profile as the CLI verification below: a 4,096-token KV capacity with CUDA Graphs disabled. Sampling and thinking behavior remain request-configurable.

Text with MTP4

.\ninfer-serve.exe '.\Qwen3.8-27B-Uncensored.ninfer' `
  --kv-dtype rk4v4-e8 `
  --spec mtp --draft-tokens 4 --lm-head-draft `
  --max-context 4096 --kv-capacity 4096 --prefill-chunk 512 `
  --preserve-thinking --no-cuda-graph

For baseline decoding, omit --spec mtp --draft-tokens 4 --lm-head-draft.

Vision with MTP4

.\ninfer-serve.exe '.\Qwen3.8-27B-Uncensored.ninfer' `
  --vision --vision-max-tokens 1024 `
  --kv-dtype rk4v4-e8 `
  --spec mtp --draft-tokens 4 --lm-head-draft `
  --max-context 4096 --kv-capacity 4096 --prefill-chunk 512 `
  --preserve-thinking --no-cuda-graph

The server exposes OpenAI-compatible /v1/chat/completions and /v1/responses endpoints plus Anthropic-compatible /v1/messages at http://127.0.0.1:8080. Prefix reuse is enabled by default; current upstream builds expose it as usage.prompt_tokens_details.cached_tokens.

This Qwen3.8 artifact supports MTP with one to five draft tokens. It does not contain a DFlash model; NInfer's DFlash path is specific to the 35B-A3B target.

Local verification results

These are bounded single-run integration measurements, not a benchmark suite. Tests used greedy decoding, thinking disabled, rk4v4-e8 KV, --max-context 4096, --kv-capacity 4096, --prefill-chunk 512, and --no-cuda-graph on one RTX 4090.

The baseline and MTP rows used the same prompt and produced the same 106-token answer, including the same Fibonacci implementation and assertions. This makes their decode-rate ratio meaningful for this one workload, but not a general performance guarantee.

The pinned NInfer fixture (image, message JSON) contains the title NIFER VISION 731, three red circles, and a blue square to the left of a green triangle. Its short decode rate is included for completeness and should not be compared directly with the coding run.

Conversion details

The dedicated converter preflighted 1,199 BF16 source tensors across 18 shards and emitted 1,118 runtime tensors plus six embedded tokenizer/template/processor resources. The 1,124 stored objects use a mixture of BF16, FP32, groupwise Q4/Q5/Q6, and W8 formats. This is a quantized deployment artifact, not a lossless copy of the BF16 checkpoint.

Conversion completed in 88.98 seconds on the RTX 4090. The complete manifest and environment are retained in Qwen3.8-27B-Uncensored.ninfer.conversion.json.

The converter report records the local source directory but not its Hugging Face revision. The adjacent Hugging Face download metadata pins the converted source files to commit adb2e5014317d59bb2093d90eaf3f8ef2b1975fe; the six direct frontend resources also matched NInfer's registered Qwen3.8 hashes. The direct source currently uses Hugging Face auto-gating, so users should review its access terms themselves.

Important limitations

• Only the documented 4k profile is locally verified here. The upstream config declares 262,144 positions, but this card does not claim a long-context validation.
• Vision is opt-in. NInfer defaults to an 8,192-token Vision scratch capacity; start with --vision-max-tokens 1024 on a 24 GB card and increase only when needed.
• Quantization can change outputs. Quality and performance depend on the prompt, sampler, KV dtype, context length, and runtime revision.
• Upstream behavior is not independently certified. "Uncensored" and "abliterated" are upstream labels, not a safety or capability evaluation by pyros-vault. Use appropriate isolation and review for red-team workloads.
• No hosted inference. The Hugging Face Inference API cannot execute .ninfer files.

License and credits

The direct source and Qwen base declare Apache-2.0. A copy of the license is included in LICENSE. Users remain responsible for reviewing the direct source's terms and complying with applicable laws.

NInfer runtime code is not redistributed here; obtain it from its separately licensed upstream repository.

Converted, tested, and packaged by pyros-vault.