Qwen3.5-0.8B-QAT-Demo
Created with MLX-LoRA-Studio · Created with MLX LoRA Studio
Overview
- Repository:
Goekdeniz-Guelmez/Qwen3.5-0.8B-QAT-Demo
- Asset type: model
- Created at: 2026-06-19 15:41:45 UTC
- Model upload mode: mergedWeights
- Base model:
/Users/Goekdeniz.Guelmez@computacenter.com/.omlx/models/mlx-community/Qwen3.5-0.8B-bf16
- Algorithm: sft
- Dataset:
/Users/Goekdeniz.Guelmez@computacenter.com/.mlxlorastudio/runs/synthetic-sft-ultrafeedback-prompts-flat-rlhf-2026-06-19-173721/generated-data
- Training length: 2 epochs
- Final train loss:
0.5110
- Final validation loss:
2.0305
This repository was prepared by MLX LoRA Studio from local training outputs.
Training Settings
Model & Data
| Field | Value |
|---|
| Model | /Users/Goekdeniz.Guelmez@computacenter.com/.omlx/models/mlx-community/Qwen3.5-0.8B-bf16 |
| Dataset | /Users/Goekdeniz.Guelmez@computacenter.com/.mlxlorastudio/runs/synthetic-sft-ultrafeedback-prompts-flat-rlhf-2026-06-19-173721/generated-data |
| Adapter path | /Users/Goekdeniz.Guelmez@computacenter.com/.mlxlorastudio/runs/Qwen3.5-0.8B-bf16-lora-sft-2026-06-19-173929/adapters |
Algorithm
| Field | Value |
|---|
| Mode | sft |
| Train type | lora |
| Optimizer | adamw |
| Quantization | 4-bit |
Optimisation
| Field | Value |
|---|
| Learning rate | 1e-05 |
| LR schedule | constant |
| Batch size | 1 |
| Grad accumulation | 1 |
| Iters / epochs | 2 epochs |
| Val batches | 1 |
| Max seq length | 128 |
LoRA Parameters
| Field | Value |
|---|
| Rank | 8 |
| Scale | 20 |
| Dropout | 0 |
Reporting & Saving
| Field | Value |
|---|
| Steps per report | 1 |
| Steps per eval | 20 |
| Save every | 100 |
Toggles
| Field | Value |
|---|
| Grad checkpoint | ✅ |
| Efficient long context | ❌ |
| Mask prompt | ❌ |
| Fuse | ✅ |
Preference / RL / GRPO
| Field | Value |
|---|
| β (beta) | 0.1 |
| Loss type | sigmoid |
| δ (delta) | 50 |
| Reward scaling | 1 |
Quantization-Aware Training
| Field | Value |
|---|
| Bits | 4 |
| Group size | 32 |
| Start step | 1 |
| Interval | 1 |
Training Metrics
- Final train loss:
0.5110 (min 0.1080)
- Final val loss:
2.0305 (min 1.0625)
- Final learning rate:
1.000e-05
- Peak memory:
1.69 GB
Loss Curve
Sparkline
- Train loss:
▇▇█▅▆▇▄▅▅▅▆▇▅▄▅█▅▆▆▆▅▃▃▃▅▃▇▅▇▅▃▂▄▅▄▁▄▅▄▂
- Validation loss:
▇▅▃▃▃▆▁█▄▅
Key Metrics
| Metric | Latest | Min | Max | N |
|---|
it_s | 247.0000 | 58.0000 | 250.0000 | 176 |
learning_rate | 1.00e-05 | 1.00e-05 | 1.00e-05 | 176 |
loss | 0.5110 | 0.1080 | 3.7760 | 176 |
peak_mem | 1.6931 | 1.6931 | 1.6931 | 176 |
tok_s | 1.6900 | 1.6900 | 1.6900 | 176 |
trained_tok | 2.19e+04 | 1.27e+02 | 2.19e+04 | 176 |
val_loss | 2.0305 | 1.0625 | 2.5901 | 10 |
val_time | 0.0475 | 0.0469 | 0.0902 | 10 |
Reproducibility
The full run_spec.json used to launch this run is included in the repository. Re-running the same spec on the same model(s), source dataset, and generation settings should reproduce an equivalent artifact (up to sampling and kernel-level non-determinism).
About
MLX-LoRA-Studio — Created with MLX LoRA Studio
MLX LoRA Studio is a SwiftUI desktop app for fine-tuning open language models on Apple Silicon with the mlx-lm-lora trainer. Curated by MLX-LoRA-Studio.
License
The license of the upstream base model(s), source dataset, generated dataset, and any included tokenizer files applies. Check the source model and dataset cards before redistribution or downstream training.