Triton VX Qwen3.5 2B (DPO)
A domain-adapted and DPO-aligned 2B parameter assistant for Fabrix.ai / RDAF knowledge retrieval, documentation citation, custom widget dashboards, storyboards, and dynamic pipelines.
This model was trained using two-stage alignment:
- Supervised Fine-Tuning (SFT) on Fabrix technical documentation, schemas, and citation formats.
- Direct Preference Optimization (DPO) using curated preference pairs to eliminate multi-line topic directory citations and enforce concise single-card citations (
kb/cards/*.md).
📊 Evaluation & Alignment Results
Evaluated across the Fabrix citation preference benchmark:
| Metric | SFT Baseline (ckpt-300) | DPO Aligned Model (This Model) |
|---|---|---|
| DPO Preference Accuracy | 50.0% | 100.0% |
| Log-Prob Preference Margin | -0.15 | +0.53 |
| Exact Match on Preferred Citations | 4.5% | 90.9% (20/22) |
Precise Card Citation Rate (kb/cards/*) | 40.9% | 95.5% (21/22) |
Sample Output Comparison
User Query:
"Put a group filter on the top so I can pick site type, tier, and primary transport."
- ❌ SFT Baseline:
kb/dashboards/attributes.md kb/widgets/tables.md kb/pstreams/flows.md - ✅ This DPO Model:
kb/cards/widget-filters.md
🚀 Usage with transformers
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
model_id = "Fabrix-AI-Inc/Triton-VX-Qwen3.5-2B-DPO"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
model_id,
torch_dtype=torch.float16,
device_map="auto",
)
system_prompt = (
"You are Fabrix cite-mode retrieval. Reply with repo-relative kb/ paths only, one per line. "
"Prefer kb/cards/*.md. Cite a topic file (kb/pstreams/, kb/widgets/, kb/dashboards/, kb/ux/, kb/entry/) "
"only when no card fits. Typically 1-5 paths. No prose, no HTML, no line numbers, no GET/POST examples, "
"no secrets. Zero paths only if nothing under kb/ applies."
)
messages = [
{"role": "system", "content": system_prompt},
{"role": "user", "content": "Put a group filter on the top so I can pick site type, tier, and primary transport."},
]
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
with torch.no_grad():
outputs = model.generate(
**inputs,
max_new_tokens=64,
do_sample=False,
)
response = tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True)
print(response.strip())
# Output: kb/cards/widget-filters.md
🏗️ Model Details
- Base Model:
unsloth/Qwen3.5-2B - Architecture:
Qwen3_5ForConditionalGeneration - Precision: 16-bit Float (
torch.float16) - Format: Standalone Merged Safetensors (
model.safetensors) - Framework: Unsloth / Hugging Face Transformers / TRL