Model Card for Zero-Mistral
This is a GGUF version of ZeroAgency/Zero-Mistral-24B.
All quants made with llama.cpp version b5083.
Quants available:
- BF16
- F16
- IQ4_NL
- IQ4_NL_L - same as above but with
--leave-output-tensors - IQ4_XS
- IQ4_XS_L - same as above but with
--leave-output-tensors - Q4_K_M
- Q4_K_M_L - same as above but with
--leave-output-tensors - Q6_K
- Q6_K_L - same as above but with
--leave-output-tensors - Q8_0 - quantized from bf16 gguf
- Q8_0-direct - direct convertation from hf
- Q8_0_L - quantized from bf16 but with
--leave-output-tensors
