Qwythos-9B nsys SFT (self-taught)
LoRA + GGUF fine-tune of Qwythos-9B for Nsight Systems (nsys) / CUDA-L1 tool-calling.
Self-taught (not API distillation)
The SFT trajectories were generated by this same local model stack, not by an external teacher LLM API (no GPT/Claude/OpenRouter demos):
- Qwythos-9B served locally with sparkinfer on an RTX 5090
- Autonomous agent loop (
prof_dataset_gen) + real nsys / CUDA-L1 tools - Quality-filtered dataset → cuda-nsys-training
- QLoRA SFT back into the same Qwythos base
Base pretrained weights still come from Empero/Qwen; the tool-use teaching signal is self-generated.
Contents
| Path | Description |
|---|---|
adapter/ | PEFT LoRA (r=16, α=32) |
gguf/Qwythos-9B-nsys-SFT-Q4_K_M.gguf | Merged no-MTP Q4_K_M (~5.3G) for sparkinfer |
gguf/Qwythos-9B-nsys-SFT-noMTP-BF16.gguf | Merged no-MTP BF16 (~17G), non-quantized |
eval/ | Holdout A/B results: base Q4, SFT Q4, SFT BF16 |
Eval (first-turn XML tool calls, n=20)
Same holdout prompts from sft_eval.jsonl, temp=0.2, max_tokens=768:
| Model | Runtime | parse_rate | valid tool names |
|---|---|---|---|
| Base Mythos Q4_K_M | sparkinfer | 0% | 0% |
| SFT Q4_K_M | sparkinfer | 30% | 30% |
| SFT BF16 (non-quantized) | llama.cpp (enable_thinking=false) | 85% | 85% |
See eval/summary.json, eval/sft_bf16_llamacpp_n20_v2.json.
Serve
Q4 (sparkinfer):
sparkinfer_server -m Qwythos-9B-nsys-SFT-Q4_K_M.gguf --tokenizer tokenizer.json --ctx 65536 --model-name qwythos-9b --port 8080
BF16 (llama.cpp):
llama-server -m Qwythos-9B-nsys-SFT-noMTP-BF16.gguf -c 4096 -ngl 99 --jinja -fa on --port 8081
# API: chat_template_kwargs={"enable_thinking": false}
Load LoRA (transformers)
from transformers import Qwen3_5ForConditionalGeneration, AutoTokenizer
from peft import PeftModel
base_id = "empero-ai/Qwythos-9B-Claude-Mythos-5-1M"
tok = AutoTokenizer.from_pretrained(base_id, trust_remote_code=True)
model = Qwen3_5ForConditionalGeneration.from_pretrained(base_id, torch_dtype="bfloat16", trust_remote_code=True)
model = PeftModel.from_pretrained(model, "gittensor-model-hub/Qwythos-9B-nsys-SFT", subfolder="adapter")
License
Apache-2.0 (inherits Qwen / Empero base terms). Dataset: see cuda-nsys-training.