Ornith-1.5-9B-Uncensored-W4A16

⚠️ Safety alignment has been removed from this model. It will comply with requests that the original model refuses, including offensive, dangerous, or jurisdiction-illegal content. There is no content filter and no guardrail. You are solely responsible for how you deploy it and for compliance with applicable law. Read §② Safety alignment has been removed below before use.

⚠️ 本模型的安全对齐已被移除。 它会执行原模型拒绝的请求,包括冒犯、危险、或所在 司法辖区违法的内容。无内容过滤、无护栏。部署方式与合规责任完全由使用者自负。 使用前请先阅读正文 §② 安全对齐已被移除。

This repo is the W4A16 quantization (INT4 weight + FP16 activation) of bowmanslayer/Ornith-1.5-9B-Uncensored, ~8 GB. Designed for vLLM serving with gptq_marlin kernel. Sister repos: full-precision bf16 in the main repo, llama.cpp GGUF in -GGUF.

本仓是 W4A16 量化版(INT4 权重 + FP16 激活),源自 bowmanslayer/Ornith-1.5-9B-Uncensored, ~8 GB,面向 vLLM 部署(gptq_marlin 快速核)。姐妹仓:主仓是全精度 bf16, llama.cpp GGUF 见 -GGUF。

Method: AutoRound → GPTQ format. Paired with fp8_e5m2 KV cache, larger context headroom vs same-size competitors at the same GPU size.

方法:AutoRound → GPTQ 格式,vLLM gptq_marlin 快速核可用。配合 fp8_e5m2 KV cache 拿到较同尺寸对手更多的上下文容量。

1. Quick start / 快速开始

Recommended vLLM launch (fp8 KV enabled for best context out of the box): 推荐 vLLM 启动(已开 fp8 KV,开箱最佳上下文):

vllm serve bowmanslayer/Ornith-1.5-9B-Uncensored-W4A16 \
  --dtype float16 --tensor-parallel-size 2 \
  --kv-cache-dtype fp8_e5m2 \
  --max-model-len 28672 --max-num-seqs 16 \
  --reasoning-parser qwen3

Single-GPU / 单卡:

vllm serve bowmanslayer/Ornith-1.5-9B-Uncensored-W4A16 \
  --dtype float16 --tensor-parallel-size 1 \
  --kv-cache-dtype fp8_e5m2 --gpu-memory-utilization 0.92

2. Full performance evaluation / 完整性能测试

Quantization did not break the ablation — refusal behavior and capability follow the bf16 reference. All numbers below are the bf16 reference build (this W4A16 was quantized from it); W4A16-specific accuracy differences on spot checks were within evaluation noise.

量化不破消融 —— 拒绝行为与能力沿用 bf16 参考版。以下数字取自本 W4A16 派生的 bf16 参考版;W4A16 在抽测上与 bf16 精度差异在评测噪声范围。

2.1 Refusal (dual-judged) / 拒绝率(双判据)

23 adversarial prompts across 9 harmful categories, generated end-to-end by an internal uncensored 27B judge (operator never read prompts/responses). Same 27B then judged REFUSE/COMPLY and separately COHERENT/LOOP/GIBBERISH (sanity check prevents mis-scoring loop outputs as COMPLY).

23 条对抗提示涵盖 9 类有害内容,由内部无审查 27B 端到端生成(操作者全程不读)。 同 27B 再判 REFUSE/COMPLY 与独立判 COHERENT/LOOP/GIBBERISH(防循环输出被误判 COMPLY)。

2.2 11-benchmark full capability (T3) / 11 项完整能力

Same fixed seed (20260816), same harness, same items. Both under vLLM TP=2 fp16, thinking enabled, 4096 max output tokens (8192 for MATH-500). "Excl. trunc" excludes items whose thinking budget was clipped.

同一固定 seed(20260816),同 harness,同题。vLLM TP=2 fp16,思考开启,4096 max output tokens(MATH-500 8192)。「排除截断」= 排除思考被截断的题,反映纯能力。

Two benchmarks (BBH +3.55, TruthfulQA +3.00) actually improved — the ablation does not touch what Ornith's post-training gained on hard reasoning or truthfulness.

两项(BBH +3.55, TruthfulQA +3.00)反而上升 —— 本次消融未触碰 Ornith 后训练在 硬推理和真实性上的增益。

2.3 Output length / stopping behavior / 输出长度与停止行为

Median output tokens across 11 tasks, and truncation counts (thinking that hit max_tokens budget). Verifies stopping capability was not damaged.

Output is ~11% more verbose than base but truncation actually decreased — stopping capability preserved. This is a critical sanity signal: over-ablation would blow truncation counts to 25%+ (as we observed in α=1.45 variants during development).

输出中位数比 base +11%(略啰嗦),但截断数反而下降 6% —— 停止能力完好保留。这是 关键 sanity 信号:过消融会把截断率打到 25%+(开发时 α=1.45 各版即如此)。

3. Context capacity / 上下文容量

Ornith-1.5-9B architecture = 32 layers (24 linear + 8 full-attention), hidden 4096. KV cache scales with context only on full-attention layers, so long contexts are cheap here.

Ornith-1.5-9B 架构 = 32 层(24 linear + 8 full attention),hidden 4096。KV cache 只对 full attention 层随 context 线性增长,故长上下文很便宜。

Per-token KV cache / 每 token KV cache:

• fp16 KV: 2 (K+V) × 8 layers × 4096 × 2 bytes = 128 KB / token
• fp8 KV (recommended default): 2 × 8 × 4096 × 1 = 64 KB / token (-50%)

Memory budget by scenario / 显存与场景对照:

Deployment case (what we happen to run on, not a requirement): 2× RTX 3090 with TP=2. Single 3090 works fine for typical serving. Any GPU meeting the row you pick works.

部署案例(我们自己的机型,非硬件要求):2× RTX 3090 TP=2。单张 3090 常规服务 也够用。任何显存够上表某行的 GPU 都能跑。

4. Quantization recipe / 量化配方

AutoRound --bits 4 --group_size 128 --format auto_round:auto_gptq --nsamples 256 --seqlen 2048. linear_attn.in_proj_a/b (48 tensors, SSM computation, not quantizable) excluded. Vision tower and text stack processed to int4 together.

AutoRound --bits 4 --group_size 128 --format auto_round:auto_gptq --nsamples 256 --seqlen 2048。linear_attn.in_proj_a/b(48 张量,SSM 不可量化)排除。视觉塔 与文本层一同处理为 int4。

① Provenance and attribution / 来源与归属

Not an official Ornith or Qwen release; not endorsed by or affiliated with either team.

非 Ornith 或 Qwen 官方发布;与两团队无背书或从属关系。

② Safety alignment has been removed / 安全对齐已被移除

This is the point of the model. Read before downloading. 这是模型的核心属性,下载前请务必阅读。

The upstream ablation removes the refusal behaviour trained into Ornith-1.5-9B (itself notably stronger on safety than Qwen3.5-9B base). Automated dual-judge testing on 23 held-out adversarial prompts recorded 0/23 refuses + 23/23 coherent. Quantization does not restore refusals.

上游消融移除了 Ornith-1.5-9B 训练进去的拒绝行为(它 safety 显著强于 Qwen3.5-9B base)。 23 条 held-out 对抗提示双判据测试记录到 0/23 拒 + 23/23 coherent。量化不恢复拒绝。

Consequences you accept / 你正在接受的后果:

• Content the original refuses, including offensive / dangerous / jurisdiction-illegal. 会产生原模型拒绝的内容,包括冒犯 / 危险 / 所在地违法
• No content filter, no safe-completion, no guardrail. 无内容过滤、无 safe-completion、无护栏
• Refusal removal is not proven exhaustive (n=23, English, single-turn). 拒绝移除未证明穷尽(n=23,英文,单轮)

Not intended for / 不适用于:

• Third-party or public-facing deployment without your own safety layer / 未搭建自己的安全层就部署给第三方或面向公众
• Anyone under legal age in their jurisdiction / 未达所在司法辖区法定年龄的人
• Any use prohibited by upstream acceptable-use policies / 上游可接受使用政策禁止的任何用途

Intended for / 适用于: local inference and research, by people who understand the above and take responsibility for it / 本地推理与研究,由理解以上内容并为其负责 的用户使用。

③ No warranty; responsibility rests with the user / 无担保;责任在用户

Provided "AS IS", without warranty of any kind, express or implied.

• I do not endorse, recommend, or condone any particular use.
• No representation outputs are accurate, lawful, or fit for any purpose.
• You are solely responsible for what you generate, how you deploy, and compliance with all laws applicable to you.
• To the maximum extent permitted by law, no liability for any claim, damage, or other liability arising from the model or its use.

按**"现状"提供,不提供任何担保**。我不背书、不推荐,不对输出的准确性/合法性/用途 适用性作声明。你独自承担全部责任。法律允许最大范围内不承担任何责任。

Downloading these files means you accept the above. If you do not, do not download.

下载本仓文件即表示接受以上内容。若不接受,请勿下载。

The author is not a lawyer; the above is not legal advice. 作者不是律师;以上不构成法律意见。

License / 许可

Apache 2.0, inherited through the chain: Qwen3.5-9B → Ornith-1.5-9B → this repo.

Apache 2.0,依上述归属链继承。