cogrpo-n3-ring-qwen25-3b-x-llama32-3b-x-phi4mini-math345-groupB-llama32-end · GGUF
GGUF quantizations of logan7000/cogrpo-n3-ring-qwen25-3b-x-llama32-3b-x-phi4mini-math345-groupB-llama32-end.
Quantized by tinyopsec using llama.cpp.
Model Overview
This is a 3.2B parameter language model fine-tuned from meta-llama/Llama-3.2-3B-Instruct using the GRPO (Group Relative Policy Optimization) method introduced in the DeepSeekMath paper.
It is part of the CogRPO N3 Ring experiment by logan7000, which explores heterogeneous model merging and multi-group reinforcement learning for mathematical reasoning. The groupB-llama32-end variant represents the Llama-3.2-3B branch of the ring fine-tuning pipeline, trained on q1716523669/MATH-Level345.
Architecture: LlamaForCausalLM
Context length: 32,768 tokens
Training data: MATH Level 3–4–5 competition problems
Quantization Table
| File | Bits | Approx. Size | Use Case |
|---|---|---|---|
model_f16.gguf | 16 | 6.4 GB | Full precision, reference |
model_q8_0.gguf | 8 | 3.4 GB | Best quality, needs more RAM |
model_q6_k.gguf | 6 | 2.6 GB | Near-lossless, recommended |
model_q5_k_m.gguf | 5 | 2.3 GB | Balanced quality/size |
model_q5_k_s.gguf | 5 | 2.1 GB | Slightly smaller Q5 |
model_q4_k_m.gguf | 4 | 2.0 GB | Recommended default |
model_q4_k_s.gguf | 4 | 1.9 GB | Smaller Q4, minor quality loss |
model_q3_k_l.gguf | 3 | 1.7 GB | Low RAM, acceptable quality |
model_q3_k_m.gguf | 3 | 1.6 GB | Lower RAM |
model_q3_k_s.gguf | 3 | 1.4 GB | Minimum viable quality |
model_q2_k.gguf | 2 | 1.2 GB | Extreme compression, lowest quality |
VRAM / RAM Requirements
| Quant | Min RAM | Recommended |
|---|---|---|
| F16 | 8 GB | 10 GB |
| Q8_0 | 5 GB | 6 GB |
| Q6_K | 4 GB | 5 GB |
| Q5_K_M | 4 GB | 5 GB |
| Q4_K_M | 3 GB | 4 GB |
| Q3_K_M | 3 GB | 4 GB |
| Q2_K | 2 GB | 3 GB |
All quants run on CPU-only. GPU is optional.
Usage
llama.cpp (CLI)
./llama-cli \
-m model_q4_k_m.gguf \
-p "Solve: If x^2 + 5x + 6 = 0, find x." \
-n 512 \
--temp 0.6 \
--top-p 0.95 \
--min-p 0.0
llama.cpp (Server)
./llama-server \
-m model_q4_k_m.gguf \
--port 8080 \
--ctx-size 8192
llama-cpp-python
from llama_cpp import Llama
llm = Llama.from_pretrained(
repo_id="tinyopsec/cogrpo-n3-ring-qwen25-3b-x-llama32-3b-x-phi4mini-math345-groupB-llama32-end-GGUF",
filename="model_q4_k_m.gguf",
)
output = llm.create_chat_completion(
messages=[{"role": "user", "content": "Solve: x^2 - 7x + 12 = 0"}],
temperature=0.6,
top_p=0.95,
min_p=0.0,
max_tokens=512,
)
print(output["choices"][0]["message"]["content"])
LM Studio
- Open LM Studio → Search
- Search:
tinyopsec/cogrpo-n3-ring-qwen25-3b-x-llama32-3b-x-phi4mini-math345-groupB-llama32-end-GGUF - Download
model_q4_k_m.gguf - Load and chat
Ollama
ollama run hf.co/tinyopsec/cogrpo-n3-ring-qwen25-3b-x-llama32-3b-x-phi4mini-math345-groupB-llama32-end-GGUF:Q4_K_M
Recommended Sampling Parameters
This model benefits from low min_p (set to 0.0) to avoid repetition loops, consistent with the base Llama-3.2 family:
| Parameter | Value |
|---|---|
| temperature | 0.6 |
| top_p | 0.95 |
| min_p | 0.0 |
| repeat_penalty | 1.0 |
Original Model
- Original: logan7000/cogrpo-n3-ring-qwen25-3b-x-llama32-3b-x-phi4mini-math345-groupB-llama32-end
- Base model: meta-llama/Llama-3.2-3B-Instruct
- Author: logan7000
- Method: GRPO · DeepSeekMath paper
- Training framework: TRL