qwen3.5-0.8b-finance
Qwen/Qwen3.5-0.8B distilled from Qwen/Qwen3.5-2B + /Users/globalids/.cache/kd-runner/peft-adapter with
Generalized Knowledge Distillation (GKD),
with the LoRA adapter merged in. Loads directly with AutoModelForCausalLM -
no PEFT required.
The LoRA adapter alone is at siddhartha-addy-globalids-labs/qwen3.5-0.8b-finance-lora.
Usage
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained("siddhartha-addy-globalids-labs/qwen3.5-0.8b-finance", dtype=torch.bfloat16)
tok = AutoTokenizer.from_pretrained("siddhartha-addy-globalids-labs/qwen3.5-0.8b-finance")
messages = [{"role": "user", "content": "How does compound interest work?"}]
inputs = tok.apply_chat_template(messages, return_tensors="pt",
add_generation_prompt=True)
print(tok.decode(model.generate(inputs, max_new_tokens=128)[0]))
Training
| Student (base) | Qwen/Qwen3.5-0.8B |
| Teacher | Qwen/Qwen3.5-2B + /Users/globalids/.cache/kd-runner/peft-adapter |
| Dataset | gbharti/finance-alpaca |
| Method | GKD (on-policy, JSD loss) |
| LoRA rank / alpha | 32 / 64 |
| Target modules | down_proj, gate_proj, in_proj_qkv, in_proj_z, k_proj, o_proj, out_proj, q_proj, up_proj, v_proj |
| Steps | 300 |
| Effective batch | 4 |
| Learning rate | 0.0002 |
| GKD lmbda / beta | 0.5 / 0.5 |
Notes
Merged with transformers, so the checkpoint uses this architecture's canonical
tensor names and loads without key remapping.