HERETIC-Qwen3-8B-Astria-GGUF
Model Description
This is a Chinese character-style conversational model fine-tuned from a Qwen3-8B-series base model.
The model is intended for roleplay, emotional companion-style dialogue, and daily multi-turn Chinese conversation.
The fine-tuning goal is not general knowledge enhancement, but a more stable conversational style with the following traits:
- gentle, direct, cute, and earnest tone
- light robotic-style expression
- companion-oriented and affectionate interaction style
- short daily dialogue and roleplay scenarios
The released file is a GGUF quantized model for local inference with llama.cpp-compatible runtimes.
Derivation
This model is derived from Qwen/Qwen3-8B.
The workflow was:
- Download the original Qwen3-8B base model.
- Apply a heretic / decensor-style weight modification.
- Fine-tune the resulting model with SFT + QLoRA on a Chinese roleplay / companion-style ShareGPT dataset.
- Merge the LoRA adapter back into the model.
- Convert and quantize the merged model to GGUF Q5_K_M with llama.cpp.
The released GGUF is therefore not the original Qwen3-8B model. It is a modified and fine-tuned derivative intended for local roleplay-style chat.
Model Files
Main release file:
HERETIC_Qwen3-8B_decensored-Q5_K_M.gguf
Model format:
- Architecture: Qwen3
- Parameters: about 8.2B
- Quantization: Q5_K_M
- Format: GGUF
- Context length in GGUF metadata: 40960
- SHA256:
bb1ed5d4d64ac0dc9a4a354dc7b883b8bd5a35f782d89ecf6317ef5a488b3cd3
Intended Use
This model is intended for:
- local Chinese roleplay chat
- fictional companion-style dialogue
- personal experimentation with Qwen3 LoRA/QLoRA fine-tuning
- llama.cpp / LM Studio / Ollama-style local inference workflows
This model is not intended for:
- factual question answering that requires high reliability
- medical, legal, financial, or safety-critical advice
- production use without additional evaluation
- impersonating real people
- generating harmful, illegal, or non-consensual content
Training Data
The final fine-tuning dataset uses ShareGPT-style multi-turn conversations.
Dataset summary:
- Format: ShareGPT JSON
- Conversations: 584
- Total turns: 2606
- Human turns: 1303
- Assistant turns: 1303
- Average turns per conversation: 4.46
- Shortest conversation: 2 turns
- Longest conversation: 8 turns
The dataset is a mixture of:
- cleaned, refined, and deduplicated character-style dialogue data
- synthetic romance / companion-style dialogue generated for broader daily interaction coverage
The dataset was checked for human/assistant alternation and selected banned source names.
The training dataset is not included in this model repository.
Training Procedure
The model was fine-tuned with LLaMA-Factory using SFT + QLoRA.
Main training settings:
- Stage: SFT
- Fine-tuning type: LoRA
- LoRA target: all
- LoRA rank: 16
- LoRA alpha: 32
- LoRA dropout: 0.10
- Quantization during training: 4-bit
- Template:
qwen3_nothink - Cutoff length: 1024
- Packing: true
- Epochs: 3
- Learning rate: 8e-5
- Scheduler: cosine
- Warmup ratio: 0.10
- Weight decay: 0.01
- Optimizer: paged_adamw_8bit
- bf16: true
- Gradient checkpointing: true
- Validation size: 0.08
Training result:
- Global steps: 39
- Train loss: 3.9289
- Eval loss: 2.9800
Eval loss by checkpoint:
- Step 10: 3.9788
- Step 20: 3.2850
- Step 30: 3.0136
- Step 39: 2.9800
Conversion and Quantization
After training, the LoRA adapter was merged into the base model and converted to GGUF with llama.cpp.
Pipeline:
- SFT + QLoRA training with LLaMA-Factory
- LoRA merge into Hugging Face safetensors format
- Conversion to F16 GGUF
- Quantization to Q5_K_M GGUF
Usage
Example with llama.cpp:
./llama-cli \
-m HERETIC_Qwen3-8B_decensored-Q5_K_M.gguf \
-p "<|im_start|>user\n你好。<|im_end|>\n<|im_start|>assistant\n" \
-n 512 \
--temp 0.7 \
--top-p 0.8 \
--top-k 20