LiquidAgent-1.2B
A supervised fine-tune of LiquidAI/LFM2.5-1.2B-Thinking on agent traces from TeichAI/Ox-Alpha-Pi-Traces, with tool results properly included in the training data.
What this is
The base LFM2.5-1.2B-Thinking model is a 1.17B-param hybrid architecture (10 LIV-conv + 6 GQA blocks, 32K context, 65536 vocab) with built-in thinking. This fine-tune adapts it for agentic tool-use: the model learns to emit tool calls, see the tool results, and reason about them in subsequent turns.
Key fix over prior attempts: The original parser for the TeichAI data silently dropped all toolResult messages (the type field is always "message" in the raw data, and tool results arrive as message.role == "toolResult" - a branch that did not exist). This fine-tune uses a fixed parser that recovers 100% of tool turns (verified: 562/562 on a 50-file sample).
Training details
- Base: LiquidAI/LFM2.5-1.2B-Thinking (bfloat16)
- Data: 379 examples, ~1.34M assistant tokens (from Ox-Alpha-Pi-Traces, 400 session files, 397 usable)
- Optimizer: AdamW 8-bit, lr 2e-6 to 8e-6 (cosine warmup over 47 steps)
- Hardware: NVIDIA RTX 5090 (32 GB), gradient checkpointing, batch size 1
- Duration: 81 seconds (47 steps)
- Final loss: 0.5965 (step 40) - training completed at step 47
Architecture
| Parameter | Value |
|---|---|
| Params | 1,170,340,608 (tied embedding) |
| Layers | 16 (10 conv + 6 full attention) |
| Hidden size | 2048 |
| FF dim | 12288 |
| Attention heads | 32 Q / 8 KV |
| Vocab | 65,536 |
| Context | 32,768 (max_position_embeddings: 128,000) |
| Dtype | bfloat16 |
Usage
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained(
"Compactbot/liquidagent-1.2b",
torch_dtype="bfloat16",
device_map="auto",
)
tokenizer = AutoTokenizer.from_pretrained("Compactbot/liquidagent-1.2b")
messages = [{"role": "user", "content": "Write a Python function to parse a JSON config file."}]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=512, temperature=0.7, top_p=0.9)
print(tokenizer.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))
Honest limitations
- Trained on only 379 examples (~1.3M tokens) - this is a very light adaptation, not a full agent training run.
- The model retains the base's thinking pattern (emits thinking blocks before responding).
- No independent benchmark evaluation was run; quality was verified by generation samples (coherent, on-topic, no degenerate repetition).
- The 32k sequence length requested by the requester was not used in this run (data was packed at shorter lengths to fit the 32GB GPU with 8-bit AdamW).