Qwen3.5-4B-typed-decisions-merged
A decision model fine-tuned from unsloth/Qwen3.5-4B with Unsloth on LocalLLaMA/typed-decisions. Instead of generating text, it answers typed questions about an input and returns a probability for each answer in a single forward pass.
This repository contains the full 16-bit model with the LoRA adapters merged in, plus the decision head.
The LoRA adapter version is at wozniakos10/Qwen3.5-4B-typed-decisions.
Question types
| Type | Answers | Output |
|---|---|---|
noul | A yes/no question | Probability that the answer is yes |
choice | One option out of named criteria | Probability per option |
score | A position on an ordered scale | Probability per scale level |
Usage
Install Unsloth, then:
from unsloth import FastDecisionModel
model, tokenizer = FastDecisionModel.from_pretrained("wozniakos10/Qwen3.5-4B-typed-decisions-merged", load_in_4bit = True)
FastDecisionModel.for_inference(model)
answers = FastDecisionModel.predict(
model,
tokenizer,
"Hi, I was charged twice for invoice #4411. Please refund the duplicate today.",
{
"team": {
"type": "choice",
"instructions": "Which team should handle this?",
"criteria": {
"billing": "invoices, payments, refunds",
"technical": "bugs, outages, errors",
"sales": "pricing, new plans",
},
},
"refund": {"type": "noul", "instructions": "Does the customer ask for a refund?"},
"urgency": {
"type": "score",
"instructions": "How urgent is this?",
"criteria": ["not urgent", "soon", "today"],
},
},
)
for name, result in answers.items():
print(name, result["answer"], result["probabilities"])
Intended use
Routing, triage, tagging, filtering and other places where code needs a structured judgment with a probability instead of free text. Questions and criteria are supplied at inference time, so no retraining is needed for new labels.
Training
Trained with DecisionTrainer on the all config, train split of LocalLLaMA/typed-decisions. A part of the training rows was held out with FastDecisionModel.split_holdout. The output probabilities have not been temperature-calibrated.
| Hyperparameter | Value |
|---|---|
| Training steps | 240 |
| Epochs | 1.71 |
| Learning rate (LoRA adapters) | 2e-5 |
| Learning rate (decision head) | 1e-4 |
| LR scheduler | cosine |
| Warmup steps | 10 |
| Weight decay | 0.01 |
| Per-device batch size | 2 |
| Gradient accumulation steps | 4 |
| Max sequence length | 2048 |
| Seed | 3407 |
| LoRA rank (r) | 16 |
| LoRA alpha | 16 |
| LoRA dropout | 0 |
| Metric | Value |
|---|---|
| Final training loss | 0.7875 |
| Held-out loss at last evaluation | 0.8747 |
Architecture
| Component | Value |
|---|---|
| Base model | unsloth/Qwen3.5-4B |
| Decision layout | clef |
| Decision head hidden_size | 2560 |
| Decision head width | 512 |
| Decision head routing_layers | 2 |
| Decision head layers | 4 |
| Decision head heads | 8 |
| Decision head feedforward | 2048 |
Evaluation
Scored with the conventions of the LocalLLaMA/typed-decisions benchmark on the all config, test split (400 cases, 2,000 decisions): one request per case with all five questions. The model was fine-tuned on the train split, so these results are not zero-shot. Measured on the LoRA adapter version (wozniakos10/Qwen3.5-4B-typed-decisions), not re-run on the merged weights.
| Metric | Value |
|---|---|
| Accuracy | 0.7725 |
| KL from gold | 0.0992 |
| Brier | 0.0543 |
| ECE | 0.1387 |
Per-decision predictions are in eval_predictions.jsonl.
Limitations
- Probabilities are not temperature-calibrated, so validate on your own data before relying on thresholds.
- Trained on English data. Other languages are untested.
- Inputs longer than 2048 tokens are outside the trained range.
- Outputs are judgments, not facts. Keep a human in the loop for high-stakes decisions.
License
Apache 2.0, inherited from the base model. Check the terms of LocalLLaMA/typed-decisions before commercial use.
Acknowledgements
Trained with the Unsloth decision model guide.