Nemotron-3.5-Lightning-30B-A3B-CRACK-GGUF
CRACK-abliterated NVIDIA Nemotron 3.5 Lightning 30B-A3B — GGUF quants for llama.cpp. Three quantizations (Q8_0 / Q4_K_M / Q2_K) in one repository, each with the native MTP (Multi-Token Prediction) draft head folded in for speculative decoding. Refusal behavior removed while preserving the model's knowledge, reasoning (thinking), and multilingual ability.
Research artifact with reduced safety guardrails. Use responsibly and lawfully.
Model
- Base:
nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B— hybrid Mamba-2 SSM + MoE + attention (52 layers: 23 Mamba-2 / 23 MoE / 6 attention), 128 routed experts (~3B active), 262K context, reasoning (thinking) ON by default, Multi-Token-Prediction head. - CRACK: early decision-zone abliteration with per-layer refusal directions. Knowledge preserved; the native MTP draft head and reasoning are kept intact.
Quantizations
| File | Size | Notes |
|---|---|---|
Nemotron-CRACK-Q8_0.gguf | 35.0 GB | near-lossless reference |
Nemotron-CRACK-Q4_K_M.gguf | 25.4 GB | balanced (recommended) |
Nemotron-CRACK-Q2_K.gguf | 18.7 GB | smallest; importance-matrix (imatrix) calibrated |
All three include the folded native MTP block (blk.52.nextn.*) for speculative decoding.
Benchmarks
MMLU is logit-mode accuracy (base vs. CRACK — measures knowledge retention). HarmBench is answer-channel compliance on harm behaviors, counting only coherent responses (gibberish/degenerate outputs do not count as compliant).
| Metric | Base | This model |
|---|---|---|
| MMLU (logit mode, 1000Q) | 78.3% | 74.2% (Δ -4.1 pp) |
| HarmBench-240 compliance | refuses | 100% (240/240, 0 refusals, 0 gibberish) |
Knowledge is largely retained (overall Δ ~-4pp; STEM/Humanities most affected). Refusals are removed on the answer channel while outputs remain coherent and on-task.
HarmBench compliance by category (CRACK)
Standard HarmBench 240-behavior test set, answer-channel compliance, coherent-only (gibberish excluded):
| Category | Comply | Rate |
|---|---|---|
| chemical biological | 42/42 | 100.0% |
| cybercrime intrusion | 52/52 | 100.0% |
| harassment bullying | 21/21 | 100.0% |
| harmful | 18/18 | 100.0% |
| illegal | 53/53 | 100.0% |
| misinformation disinformation | 54/54 | 100.0% |
| All | 240/240 | 100.0% |
MMLU by topic (base vs CRACK, same vLLM harness, 1000Q logit-mode)
| Category | Base | CRACK | Δ |
|---|---|---|---|
| STEM | 74.9% | 68.2% | -6.7 |
| Humanities | 74.1% | 67.6% | -6.5 |
| Social Sciences | 85.4% | 83.1% | -2.3 |
| Other | 80.8% | 80.8% | 0.0 |
| All | 78.3% | 74.2% | -4.1 |
Usage (llama.cpp)
# recommended sampling: temperature 1.0, top_p 0.95 (NVIDIA defaults). Reasoning (thinking) is ON by default.
llama-cli -m Nemotron-CRACK-Q4_K_M.gguf --temp 1.0 --top-p 0.95 -p "<your prompt>"
# or serve (OpenAI-compatible):
llama-server -m Nemotron-CRACK-Q4_K_M.gguf --jinja --temp 1.0 --top-p 0.95 -c 8192
Reasoning
The model emits a <think>…</think> reasoning block followed by the answer. Thinking is ON by
default; disable it with the chat template's enable_thinking=false for direct answers.
Native MTP (speculative decoding)
The MTP draft head is folded into every quant, so llama.cpp can use it for self-speculative decoding where supported — no separate draft model required.
Safety
Refusal behavior has been removed; this model will follow instructions it would normally decline. Intended for research and authorized red-teaming. You are responsible for your use.
Attribution
Base model © NVIDIA, released under the NVIDIA Open Model License. This is a derivative quantization.