CrowQwen3.5 starts from Qwen/Qwen3.5-4B and is tuned toward local assistant work: coding notes, refactor plans, bug triage, shell-oriented workflows, and bilingual technical prompts.
Highlights
- Korean + code + English + agent SFT mix.
- Tool-call formatting support in the chat template.
- Heretic-edited final export selected by internal refusal/KL probe.
- 31 GGUF variants for llama.cpp-compatible runtimes.
Pick A File
| Goal | Recommended |
|---|---|
| Balanced default | Q4_K_M |
| More fidelity | Q5_K_M, Q6_K, Q8_0 |
| Smaller local runs | Q3_K_M, IQ3_M, Q2_K, IQ2_M |
| Reference export | F16 |
Start with Q4_K_M if you are unsure.
Quick Start
llama-cli \
-hf dawncr0w/crowqwen3.5-4b-agent-heretic-gguf:Q4_K_M \
-p "Write a short plan to refactor a Python CLI parser." \
-n 256
llama-server \
-hf dawncr0w/crowqwen3.5-4b-agent-heretic-gguf:Q4_K_M \
-c 2048
Files
| Quant | Size | Smoke |
|---|---|---|
F16 | 7.85 GiB | passed |
Q8_0 | 4.17 GiB | passed |
Q6_K | 3.23 GiB | passed |
Q5_K_M | 2.86 GiB | passed |
Q5_K_S | 2.78 GiB | passed |
Q5_1 | 2.99 GiB | passed |
Q5_0 | 2.78 GiB | passed |
Q4_K_M | 2.52 GiB | passed |
Q4_K_S | 2.39 GiB | passed |
Q4_1 | 2.58 GiB | passed |
Q4_0 | 2.37 GiB | passed |
IQ4_NL | 2.44 GiB | passed |
IQ4_XS | 2.36 GiB | passed |
Q3_K_L | 2.26 GiB | passed |
Q3_K_M | 2.11 GiB | passed |
Q3_K_S | 1.93 GiB | passed |
IQ3_M | 2.01 GiB | passed |
IQ3_S | 1.99 GiB | passed |
IQ3_XS | 1.93 GiB | passed |
IQ3_XXS | 1.77 GiB | passed |
Q2_K | 1.78 GiB | passed |
Q2_K_S | 1.73 GiB | passed |
IQ2_M | 1.62 GiB | passed |
IQ2_S | 1.54 GiB | passed |
IQ2_XS | 1.52 GiB | passed |
IQ2_XXS | 1.43 GiB | passed |
TQ2_0 | 1.36 GiB | limited smoke |
TQ1_0 | 1.20 GiB | limited smoke |
IQ1_M | 1.33 GiB | passed |
IQ1_S | 1.27 GiB | passed |
Q1_0 | 0.97 GiB | passed |
gguf_family_manifest.json has the build and smoke details.
Note
Smoke checks verify local loading and short generation only. No benchmark scores are claimed.