Spark-X2.5-4B GGUF
This repository contains GGUF format quants for XHToken/Spark-X2.5-4B.
Spark-X2.5-4B is a compact, high-efficiency language model featuring a hybrid attention architecture (combining sliding-window attention with full attention layers) and native context lengths up to 1M tokens.
These GGUF files were converted from the original safetensors weights using a custom Spark 2.5 build of llama.cpp at BF16 precision, then quantized into standard k-quants.
Available Files and Quantizations
| File | Quant Type | Size | Description / Recommendation |
|---|---|---|---|
Spark-X2.5-4B-BF16.gguf | BF16 | 7.66 GB | Full precision base conversion. Highest fidelity, largest size. |
Spark-X2.5-4B-Q8_0.gguf | Q8_0 | 4.07 GB | Near-lossless 8-bit quantization. Recommended if you have 6GB+ VRAM. |
Spark-X2.5-4B-Q6_K.gguf | Q6_K | 3.15 GB | Excellent quality retention with minimal loss. Very balanced choice. |
Spark-X2.5-4B-Q5_K_M.gguf | Q5_K_M | 2.77 GB | Good balance between memory footprint and output quality. |
Spark-X2.5-4B-Q4_K_M.gguf | Q4_K_M | 2.42 GB | Fast, lightweight 4-bit quant. Great default for laptops and mobile devices. |
Spark-X2.5-4B-Q3_K_M.gguf | Q3_K_M | 2.02 GB | Lower memory footprint when RAM or VRAM is tight. |
Spark-X2.5-4B-Q2_K.gguf | Q2_K | 1.66 GB | Maximum compression. Noticeable quality degradation, best for ultra-constrained environments. |
Requirements
Spark-X2.5 uses the Spark2_5ForCausalLM (spark2_5) architecture. To run these files, ensure your local llama.cpp installation or downstream GUI client includes Spark 2.5 runtime support.
How to Use
1. With llama.cpp
Run the model directly from the command line using llama-cli:
llama-cli -m ./Spark-X2.5-4B-Q4_K_M.gguf \
-p "You are a helpful and concise assistant.\nUser: Hello! What can you do?\nAssistant:" \
-n 512 \
-c 4096 \
--temp 0.7
To run as a local server:
llama-server -m ./Spark-X2.5-4B-Q4_K_M.gguf \
--host 127.0.0.1 \
--port 8080 \
-c 4096
2. With Ollama
Create a Modelfile in the same directory:
FROM ./Spark-X2.5-4B-Q4_K_M.gguf
TEMPLATE """<|im_start|>system
{{ .System }}<|im_end|>
<|im_start|>user
{{ .Prompt }}<|im_end|>
<|im_start|>assistant
"""
PARAMETER stop "<|im_start|>"
PARAMETER stop "<|im_end|>"
PARAMETER temperature 0.7
Then build and run the model:
ollama create spark-x2.5-4b -f Modelfile
ollama run spark-x2.5-4b
3. With LM Studio
- Download or copy any
.gguffile to your LM Studio models folder. - Search for
Spark-X2.5-4Bin your local models tab. - Load the model with default GPU offloading settings and chat.
Original Model Details
- Original weights: XHToken/Spark-X2.5-4B
- Base Architecture: Spark 2.5 (
Spark2_5ForCausalLM) - Context Length: Up to 1,048,576 tokens (1M)
- Vocabulary Size: 131,072 tokens
- License: Apache 2.0
Attribution & Credits
Original model weights and architecture developed by XHToken.
GGUF conversions produced by Pink Pixel.
Made with 💖 by Pink Pixel