Model Card for icdu_qwen3.5_9b_dpo_merged_model-q4km
Q4_K_M GGUF quantization of a two-stage QLoRA fine-tune of Qwen/Qwen3.5-9B: supervised fine-tuning on ICDU-formatted coaching/instruction data (ICDU General Dataset v8), followed by Direct Preference Optimization for tool-selection behaviour, with both LoRA adapters merged before conversion.
Model Details
Model Description
This is the final artifact of the ai-factory QLoRA + DPO pipeline, built on August 12, 2026, exported to GGUF and quantized to Q4_K_M for llama.cpp-compatible runtimes.
Lineage: (1) a QLoRA LoRA adapter was trained on Qwen/Qwen3.5-9B with ICDU v8 instruction data and merged into the base weights (final_merged_model); (2) a second LoRA adapter was trained on that merged checkpoint with DPO on tool-selection preference pairs and merged again (dpo_merged_model, saved as a text-only Qwen3_5ForCausalLM checkpoint in float16 per its surviving config.json); (3) the merged DPO model was converted to GGUF and quantized to Q4_K_M (llama.cpp general.file_type 15, quantization version 2). The export is text-only; the chat template from the base tokenizer is preserved in the GGUF metadata.
- Developed by: Overture System Solutions
- Model type: Causal language model (QLoRA SFT + DPO fine-tune; GGUF Q4_K_M quantization)
- Language(s) (NLP): English (en)
- License: apache-2.0
- Finetuned from model: Qwen/Qwen3.5-9B
Model Sources [optional]
- Model Repository: cds1/icdu_dpo_qwen3.5_9b_v2
- Training Pipeline: https://github.com/Farsalis/ai-factory
How to Get Started with the Model
Use the code below to get started with the model.
# llama.cpp
llama-cli -m icdu_qwen3.5_9b_dpo_merged_model-q4km.gguf \
-p "I keep failing to build a fitness habit. What should I do?" -cnv
# llama-cpp-python
from llama_cpp import Llama
llm = Llama(
model_path="icdu_qwen3.5_9b_dpo_merged_model-q4km.gguf",
n_ctx=8192, # architecture supports up to 262,144
)
out = llm.create_chat_completion(messages=[
{"role": "user", "content": "Help me plan my first dinner party."},
])
print(out["choices"][0]["message"]["content"])
Training Details
Training Data
SFT: ICDU General Dataset v8 - ICDU-formatted JSONL (icdu_training_data_v8.jsonl, 13,671 records; icdu_validation_data_v8.jsonl, 1,520 records). Each record carries the ten-field ICDU schema (persona archetype, governing principle, capability layer, user intent, context summary, prompt, ideal response, attributes, rationale outline) over life/work coaching topics derived from the 'Breaking Better' source corpus.
DPO: 900 chosen/rejected preference pairs generated from bb_training_data_v7.jsonl (tool-augmented chat data). Chosen responses contain the correct tool call; rejected responses either drop the tool call or invoke a synthetically chosen incorrect tool.
Training Procedure
Preprocessing [optional]
ICDU records were rendered to chat format - a system message carrying context, user intent, and persona, plus the user prompt and assistant response - with a 50% chance of persona-specific scenario perturbation; loss was computed on assistant completions only. DPO prompts used the [INST] ... [/INST] instruction format.
Training Hyperparameters
- Training regime: bf16 mixed precision with a 4-bit (NF4) quantized base model for both stages (QLoRA).
Stage 1 - Supervised fine-tuning (SFT):
| Hyperparameter | Value |
|---|---|
| epochs | 1 |
| per_device_train_batch_size | 4 |
| gradient_accumulation_steps | 2 |
| learning_rate | 0.0002 |
| lr_scheduler_type | cosine |
| warmup_ratio | 0.03 |
| weight_decay | 0.001 |
| max_grad_norm | 0.3 |
| optimizer | paged_adamw_8bit |
| max_sequence_length | 4096 |
| lora_r | 32 |
| lora_alpha | 32 |
| lora_dropout | 0.05 |
| lora_target_modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
| quantization (train-time) | 4-bit NF4, double quantization |
| attention | flash_attention_2 |
| evaluation | every 150 steps on eval_loss; best checkpoint kept |
| seed | 42 |
Stage 2 - Direct Preference Optimization (DPO), via TRL's DPOTrainer:
| Hyperparameter | Value |
|---|---|
| max_steps | 100 |
| learning_rate | 5e-06 |
| beta | 0.1 |
| lora_rank | 32 |
| lora_alpha | 64 |
| lora_dropout | 0.05 |
| lora_target_modules | q_proj, v_proj |
| per_device_train_batch_size | 2 |
| gradient_accumulation_steps | 2 |
| optimizer | paged_adamw_8bit |
| lr_scheduler_type | cosine |
| warmup_ratio | 0.03 |
| max_length | 1024 |
| quantization (train-time) | 4-bit NF4, double quantization |
Speeds, Sizes, Times [optional]
| Measurement | Value |
|---|---|
| GGUF file size | 5,629,108,544 bytes (~5.24 GiB) |
| Quantization | Q4_K_M (file_type 15, quantization_version 2) |
| Tensor count | 427 |
| Build date | 2026-08-12 |
Wall-clock training time was not retained from the original run.
Technical Specifications
Model Architecture and Objective
From the GGUF header (general.architecture: qwen35) and the merged checkpoint's config.json (Qwen3_5ForCausalLM, model_type: qwen3_5_text): 9.0B parameters, 32 blocks - 24 linear-attention (Gated DeltaNet SSM: conv kernel 4, 16 key / 32 value heads, head dim 128) and 8 full-attention layers interleaved every 4th layer - hidden size 4096, FFN 12288, grouped-query attention (16 query / 4 KV heads, head dim 256, partial rotary factor 0.25), RoPE base 1e7 with interleaved mRoPE sections [11, 11, 10], context length 262,144, vocabulary 248,320. Source checkpoint dtype float16; GGUF v3.
Compute Infrastructure
[More Information Needed]
Hardware
~5.3 GiB for weights plus KV cache; fits fully offloaded on GPUs with 8 GB VRAM at moderate context, or runs on CPU.
Software
Python 3.12, PyTorch 2.5.1 (CUDA 12.4), transformers 5.12.1 (recorded in the merged checkpoint's config.json), TRL >=0.27 (DPOTrainer), PEFT >=0.18, bitsandbytes >=0.49, datasets >=4.5, as pinned by this repository's environment.yml at build time; GGUF conversion/quantization via llama.cpp (exact version not recorded).
Model Card Authors
Overture System Solutions (O.S.S.) - Jordan Martens, Samuel Conrad.
Model Card Contact
Jordan Martens jordan.martens@osscontact.com