Qwen3.5-0.8B chat titles

Qwen3.5-0.8B fine-tuned to turn a chat message into a short title (2-6 words), for example:

Trained on kth8/title-generation-10000x.

Files

• qwen3.5-0.8b-kth8-Q8_0.gguf: 812 MB, for llama.cpp (needs a build with the qwen35 architecture).
• model.safetensors and tokenizer files: bf16 weights for transformers.

Prompt

Use the chat template with thinking disabled (an empty think block) and this user message, with the message appended:

Write a short title (2-6 words) for the message below. Reply with the title only.

Message: <message, first 1000 characters>

from transformers import AutoModelForCausalLM, AutoTokenizer

repo = "Mabeck/qwen3.5-0.8b-kth8-titles"
tok = AutoTokenizer.from_pretrained(repo)
# float32 on CPU: the reference kernels give degraded titles in bf16. bf16 is fine on GPU.
model = AutoModelForCausalLM.from_pretrained(repo, dtype="float32")
prompt = "Write a short title (2-6 words) for the message below. Reply with the title only.\n\nMessage: "
text = tok.apply_chat_template(
    [{"role": "user", "content": prompt + "How do I reverse a linked list in Python?"}],
    add_generation_prompt=True, tokenize=False, enable_thinking=False,
)
out = model.generate(**tok(text, return_tensors="pt"), max_new_tokens=16, do_sample=False)
print(tok.decode(out[0], skip_special_tokens=True).split("\n")[-1])

llama-completion -m qwen3.5-0.8b-kth8-Q8_0.gguf -p "<formatted prompt>" -n 16 --temp 0 \
  -no-cnv --no-display-prompt -ngl 99 -c 512 -fa on

Training

Full fine-tune (embeddings frozen), AdamW, learning rate 1e-5 with 10 warmup steps and cosine decay, effective batch 16, 2 epochs, loss on the title only. Data: the English split of kth8/title-generation-10000x, keeping titles of 2-6 words (3,237 of 7,000 rows), of which 3,076 were used for training and 161 held out.

Evaluation

On the 161 held-out rows (greedy decoding):

Limitations

• About 8% of titles exceed 6 words, more often for long or rambling messages. Truncate the output if you need a hard limit.
• Trained on English messages only. For other languages the title usually follows the message language, and quality is untested.