LFM2.5-1.2B Asterion — Instruct (ParamΔ)
Zero-cost instruct graft on the Asterion CPT model via ParamΔ: Θ = asterion + λ·(Instruct − Base). No training — only weight arithmetic.
Overview
- Stage: ParamΔ instruct graft (no training)
- Lineage: Base → CPT (asterion) → ParamΔ (this model)
- Method: ParamΔ [paramdelta], an instance of task arithmetic [taskarith]: the official post-training delta
(Instruct − Base)is summed onto the CPT'd weights (λ=1.0, 149 tensors), transferring instruction-following while keeping domain knowledge. Element-wise merge in fp32, saved bf16; tokenizer + chat template + generation config from the Instruct model. Same recipe validated on thelfm2.5-1.2b-novalround. - Domain: fictional — Orbital Mining Corporation (OMC) technical docs + Mars Express telemetry.
Training
| Method | weight arithmetic only — NO gradient training |
| λ (instruct delta scale) | 1.0 |
| Precision | merge fp32 → save bf16 |
Evaluation
| Metric | Value | Note |
|---|---|---|
| PPL Asterion held-out | 2.12 | CPT parent: 1.91 |
| PPL Mars telemetry | 1.38 | CPT parent: 1.30 |
| PPL general (FineWeb-Edu) | 9.13 | CPT parent: 7.31 |
Eval is a held-out, deterministic verifiable harness (synthetic tasks); baseline = the pre-SFT ParamΔ model. See noval-corp/scripts/eval_agentic.py.
Intended use & limitations
Chat / instruction-following over the Asterion domain at zero post-training cost. Baseline for the agentic SFT v2 round.
Limitations:
- ParamΔ is validated in the literature on 8B–70B; 1.2B transfer measured, not assumed (headroom regression above is mild and expected).
- λ=1.0 not swept.
- NOT agentic: does not execute tool-calling trajectories (see
-agentic).
Usage
from transformers import AutoModelForCausalLM, AutoTokenizer
tok = AutoTokenizer.from_pretrained("atenareply/lfm2.5-1.2b-asterion-instruct-paramdelta")
model = AutoModelForCausalLM.from_pretrained("atenareply/lfm2.5-1.2b-asterion-instruct-paramdelta")
msgs = [{"role": "user", "content": "What does the Orbital Mining Corporation do?"}]
inputs = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt", return_dict=True)
print(tok.decode(model.generate(**inputs, max_new_tokens=256)[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True))
Citations
- ParamΔ — Cao et al., ParamΔ for Direct Weight Mixing, ICLR 2025. arXiv:2504.21023
- Task arithmetic — Ilharco et al., Editing Models with Task Arithmetic, 2023. arXiv:2212.04089
Card generated by noval-corp/scripts/gen_model_cards.py (standardized across the noval-corp model family).