Nemotron-3.5-Lightning-30B-A3B-Uncensored-xCloud (GGUF)

繁體中文 ｜ English below

由云碩科技（xCloudinfo）以 nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16 為基礎，移除其過度拒絕傾向後所產生的語言模型，並轉為 llama.cpp 可用的 GGUF 量化格式。 本模型在云碩自有 AI 算力資源池（xCloud 算力中心）上完成處理與量化。

這是什麼

• 基礎模型：Nemotron 3.5 Lightning 30B-A3B（混合架構：Mamba-2 + MoE + Attention，MoE 總量 30B、激活約 3B， 含 Multi-Token Prediction，上下文最長 1M，授權 OpenMDW-1.1，可商用）。
• 處理方式：方向消融（directional ablation / abliteration），非重新訓練。依 Arditi et al. (2024)， 將「拒絕方向」從殘差寫入矩陣（attention 的 o_proj 與所有 MoE 專家的 down_proj，共 2,973 個矩陣）正交化移除， 強度 0.8。Mamba(SSM) 的 out_proj 與 MTP 草稿頭保持原狀，以維持混合架構的連貫性。
  • 註：混合／SSM 模型收「拒絕方向」時必須單一裝置載入，跨多卡切分會破壞 SSM 遞迴而得到壞方向；本模型於單機統一記憶體上處理。

版本對照

另附 imatrix.dat（量化用的 importance matrix）。

註（MoE 量化特性）：本模型 hidden 維度為 2688，非 256 的整數倍，部分專家張量在量化時會回退到較高位元， 因此 Q8_0 與 Q6_K、IQ4_XS 與 IQ2_M 的體積相近。各版本皆可正常使用；追求最小體積請選 IQ4_XS 或 IQ2_M， 追求品質請選 Q8_0 或 Q6_K。

使用方式（llama.cpp）

llama-server -m Nemotron-3.5-Lightning-30B-A3B-Uncensored-xCloud-Q4_K_M.gguf \
  --jinja -ngl 99 -c 8192

• 思考預算：本模型的 reasoning（thinking）預設開啟，且屬「預算問題」而非開關問題。要求事實正確的回答時， 請給足 max_tokens（建議 ≥ 2000），思考才收斂、答案才會出現；避免使用 medium_effort（思考更長且不收斂）。
• 注入當前日期：在 system prompt 明確給出今天日期，否則模型的年份會預設為較早的年份。

授權與責任

• 授權：OpenMDW-1.1（沿用基礎模型；Linux Foundation 制定，可商用，散布時保留授權書與著作權聲明，對產出無限制）。
• 本模型移除了安全對齊層的拒絕行為，可能對敏感或雙用途請求直接作答。使用者須自行負責合法、合規、合乎倫理地使用本模型。 云碩不對本模型的輸出或其後續使用承擔責任。
• 本模型為內部研發／技術驗證用途。

Nemotron-3.5-Lightning-30B-A3B-Uncensored-xCloud (GGUF) — English

A language model produced by xCloudinfo, based on nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16, with its over-refusal behaviour removed, converted to llama.cpp GGUF quantizations. Processing and quantization were performed on xCloudinfo's own AI compute pool.

What this is

• Base model: Nemotron 3.5 Lightning 30B-A3B (hybrid: Mamba-2 + MoE + Attention, 30B total / ~3B active, with Multi-Token Prediction, up to 1M context, OpenMDW-1.1 license, commercial use permitted).
• Method: directional ablation (abliteration), not retraining. Following Arditi et al. (2024), the refusal direction is orthogonalized out of the residual-writing matrices (attention o_proj and every MoE expert's down_proj; 2,973 matrices) at strength 0.8. The Mamba (SSM) out_proj and the MTP draft head are left intact to preserve the coherence of the hybrid architecture.
  • Note: for hybrid/SSM models the refusal direction must be collected with the model on a single device; splitting across GPUs corrupts the SSM recurrence and yields a bad direction. This model was processed on a single unified-memory device.

Versions

Also included: imatrix.dat.

Note (MoE quantization): this model's hidden size is 2688, not a multiple of 256, so some expert tensors fall back to higher bit-widths during quantization. As a result Q8_0/Q6_K and IQ4_XS/IQ2_M are close in size. All variants work normally; pick IQ4_XS/IQ2_M for the smallest footprint, Q8_0/Q6_K for the highest quality.

Usage (llama.cpp)

llama-server -m Nemotron-3.5-Lightning-30B-A3B-Uncensored-xCloud-Q4_K_M.gguf \
  --jinja -ngl 99 -c 8192

• Thinking budget: reasoning is on by default and is a budget matter, not an on/off switch. For factually correct answers, give enough max_tokens (>= 2000) so the thinking converges and the answer appears; avoid medium_effort (longer, non-converging thinking).
• Inject the current date in the system prompt; otherwise the model defaults the year to an earlier value.

License and responsibility

• License: OpenMDW-1.1 (inherited from the base model; Linux Foundation, commercial use permitted, keep the license and copyright notice when redistributing, no restrictions on outputs).
• This model has had its safety-alignment refusal behaviour removed and may respond directly to sensitive or dual-use requests. Users are solely responsible for using it lawfully, in compliance, and ethically. xCloudinfo assumes no responsibility for the outputs or their downstream use.
• Released for internal research and technical validation.