gemma4-12b-heretic-ltx-2.5-comfy-nvfp4

Comfy-native NVFP4 text encoder for the LTX-2.5 pipeline. Drop-in replacement for the Gemma 4 12B heretic text encoder in ComfyUI, quantized to FP4 with block-wise scaling.

This checkpoint is a mixed-precision NVFP4 quantization of DeepNeuralNerd/Gemma-4-12B-it-uncensored-heretic-DeepNeuralNerd-LTX_2.5_ComfyUI. It loads as a text encoder in ComfyUI (models/text_encoders/) and is designed as a drop-in replacement for the LTX-2.5 Gemma text encoder.

File

Source: 24.46 GB BF16 → 9.87 GB NVFP4 (~60% smaller).

What is quantized

The heavy Gemma decoder Linear weights are stored as Comfy-native NVFP4 (FP4 E2M1, block size 16). Everything else — embeddings, normalization layers, vision components, and LTX-specific projection layers — stays at BF16.

Kept at BF16: embed_tokens, all layernorms, layer_scalar, q_norm/k_norm, audio_projector, multi_modal_projector, text_embedding_projection, vision_model, embedded tokenizer + HF assets.

Per-layer NVFP4 tensor layout

Each quantized matrix is stored as four tensors (Comfy comfy_quant format):

Architecture notes

Gemma 4 12B has 48 decoder layers with eight global-attention layers: 5, 11, 17, 23, 29, 35, 41, 47.

Those global-attention layers reuse K as V and therefore contain no separate v_proj.weight — hence 40 quantized v_proj matrices instead of 48.

The checkpoint carries the LTX-2.5 remap applied by DeepNeuralNerd:

model.language_model.*                         -> model.*
model.vision_embedder.*                        -> vision_model.*
model.embed_vision.embedding_projection.weight -> multi_modal_projector.embedding_projection.weight
model.embed_audio.embedding_projection.weight  -> audio_projector.embedding_projection.weight

The source tokenizer.json is embedded as tokenizer_json.

ComfyUI usage

Place the file in ComfyUI/models/text_encoders/ and select it in your LTX-2.5 workflow's Gemma text encoder loader. No extra nodes required.

Hardware requirements

• NVFP4 native matmul requires a Blackwell GPU (SM ≥ 10.0) with PyTorch cu130. On those GPUs you get the full FP4 speed benefit.
• On older GPUs (Ampere/Ada), ComfyUI silently dequantizes the weights — the model still loads and produces identical output, but without the FP4 speed advantage.

Quantization method

• Tool: convert-to-quant (ctq) v1.3.4
• Format: Comfy-Org comfy-quants native NVFP4
• Mode: --nvfp4 --comfy_quant --save-quant-metadata --low-memory --simple
• --simple: fast direct rounding (no learned-rounding / SVD optimization pass)
• --low-memory: streamed, one tensor at a time (~2 GB VRAM, ~8 GB RAM per chunk)

The 24.46 GB source cannot be held in RAM in one pass, so quantization ran as a split → quantize → merge pipeline: decoder layers split into 3 chunks, each quantized separately, then merged with the untouched BF16 base. See QUANTIZATION.md for the full writeup.

Reproducible command (per chunk)

ctq \
  -i decoder_chunk_N.safetensors \
  -o decoder_chunk_N_nvfp4.safetensors \
  --nvfp4 --comfy_quant --save-quant-metadata \
  --low-memory --simple \
  --exclude-layers "text_embedding_projection|tokenizer_json|hf_asset__|embed_tokens"

The model was not retrained or fine-tuned — only weight quantization was applied.

Provenance & safety

This model descends from the Heretic abliteration lineage (llmfan46/gemma-4-12B-it-uncensored-heretic), which removes internal refusal directions from the base google/gemma-4-12B-it.

[!IMPORTANT] This model intentionally has reduced refusal behavior and may generate sensitive or unrestricted content. Intended for research and learning purposes only. Users assume full responsibility for generated output.

License

Apache 2.0. See LICENSE.