Tailorbird v0.2
/| /|
_/\_/ |/ |_
/ __ /_
/ /_> '->
\ // .-'
___..--[==X==]___/
_.-'' _..-' / `-.
<__..-'' __.-' .---. \
__.-' / / /| |
<__/ / \/_/_/ /
___/ .-. _.-'
/____ `-'__.-'
/__.-' / /
/_/ __/_/___
Small bird. Sharp memory. Tailorbird turns sprawling conversation history into compact handoff notes without dropping the decisions that matter.
Tailorbird v0.2 is a 2.5B-parameter conversation-memory compression specialist
fine-tuned from
openbmb/MiniCPM5-2B-Midtrain.
It is trained to preserve durable facts, decisions, constraints, identifiers,
superseded values, and open work while stripping repetition and conversational
filler.
This repository contains the standalone, non-quantized BF16 model with its LoRA adapter merged into the base weights. No adapter juggling required: load it and fly.
What it is good at
- Compressing long, multi-turn conversations into terse continuation context
- Separating
FACTS,DECISIONS,CONSTRAINTS, andOPENitems - Tracking replacements with explicit
SUPERSEDES: old -> newnotation - Preserving exact ticket IDs, paths, dates, commands, URLs, and error strings
- Emitting
(no update)when a conversation adds nothing durable - Adding a short
STORYonly when the overall goal or status changes
Quick start
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "vcerny/tailorbird-v0.2"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
model_id,
torch_dtype=torch.bfloat16,
device_map="auto",
)
messages = [
{
"role": "user",
"content": """Compress conversation history for another model. Output notes only.
Append ONLY new durable information; committed notes stay unchanged.
Use terse FACTS/DECISIONS/CONSTRAINTS/OPEN lines.
Committed notes (reference only):
FACTS: Deployment uses image v1.4.
New messages:
user: Move the rollout from Tuesday to Thursday.
assistant: Noted. The rollout is now planned for Thursday.
""",
}
]
inputs = tokenizer.apply_chat_template(
messages,
add_generation_prompt=True,
return_tensors="pt",
).to(model.device)
with torch.inference_mode():
generated = model.generate(
inputs,
max_new_tokens=256,
do_sample=False,
)
print(tokenizer.decode(generated[0, inputs.shape[-1]:], skip_special_tokens=True))
For best results, keep the compression contract explicit in the prompt and use greedy decoding when deterministic handoff notes matter.
Training snapshot
| Item | Value |
|---|---|
| Base model | openbmb/MiniCPM5-2B-Midtrain |
| Method | Supervised fine-tuning with LoRA, merged after training |
| Training examples | 2,542 |
| Validation examples | 147 |
| Held-out test examples | 116 |
| Epochs | 2 |
| Maximum training sequence length | 2,048 tokens |
| LoRA | rank 16, alpha 32, dropout 0, all linear layers |
| Effective batch size | 8 |
| Optimizer | 8-bit AdamW |
| Peak learning rate | 2e-4 with linear decay |
| Final aggregate training loss | 0.9681 |
| Validation loss | 0.9464 at epoch 1; 0.9173 at epoch 2 |
| Training hardware | NVIDIA H200 NVL MIG 1g.18gb, 16 GiB visible memory |
| Output precision | Merged BF16, non-quantized |
The held-out test split was not consumed during training. No held-out benchmark is reported yet, so the validation loss should not be treated as a complete measure of real-world accuracy.
Intended use
Tailorbird is intended for conversation compaction, agent handoffs, rolling memory updates, and structured session summaries. It is especially suited to workflows where dropping a constraint or silently reviving an obsolete value is more damaging than producing slightly awkward prose.
It is not a source of truth. Review outputs before using them for operational, legal, medical, financial, or safety-critical decisions. The model can omit, merge, or misclassify details and may reproduce sensitive information present in its input. Apply access controls and retention rules appropriate to your data.
Limitations
- Training and evaluation focus on English structured-memory prompts.
- Exact output shape depends strongly on the prompt contract.
- Very long inputs were not exercised at the model's architectural context limit; fine-tuning used sequences up to 2,048 tokens.
- Validation loss measures next-token prediction, not factual retention or downstream task success.
- The merged model is approximately 5 GB and is not quantized for edge use.
Training stack
Built with Unsloth, TRL 0.24.0, Transformers 5.17.0, PyTorch 2.11.0+cu128,
Datasets 4.3.0, and Tokenizers 0.23.2. Training used seed 3407.
Base model and license
Tailorbird v0.2 derives from
openbmb/MiniCPM5-2B-Midtrain.
See the base model card for architecture details, upstream limitations, and
citations. This model is distributed under the Apache License 2.0.