Cydonia-24B-v4.3-AWQ
AWQ 4-bit quantization of TheDrummer/Cydonia-24B-v4.3.
Quantization Details
- Method: AWQ (Activation-aware Weight Quantization)
- Bits: 4-bit
- Group size: 128
- Version: GEMM
- Zero point: True
- Model size: ~14 GB (vs ~48 GB FP16)
Usage
Works with vLLM, Transformers, and other AWQ-compatible inference engines.
# vLLM
vllm serve Irvollo/Cydonia-24B-v4.3-AWQ --quantization awq_marlin --dtype float16
# Transformers
from awq import AutoAWQForCausalLM
model = AutoAWQForCausalLM.from_quantized("Irvollo/Cydonia-24B-v4.3-AWQ")
Original Model
Cydonia v4.3 by TheDrummer — a Mistral Small 3.1 24B fine-tune optimized for roleplay and creative writing.
Hardware Requirements
- Minimum: 16 GB VRAM (RTX 4090, A100, etc.)
- Recommended: 24 GB VRAM for comfortable KV cache headroom