LFM2.5-2.6B Terminal SFT
Full-parameter supervised fine-tuning of LiquidAI/LFM2.5-2.6B for terminal-agent tasks.
The experiment reproduces the Phase 1 configuration in Liquid-CLI's
train_unsloth_processed.py, changing only the base model and using native
Transformers/TRL FlashAttention support for LFM2.5.
Training data
139,841 non-coding terminal-agent conversations prepared from the
skill_based_easy, skill_based_medium, and skill_based_mixed configurations
of nvidia/Nemotron-Terminal-Corpus using Liquid-CLI's prepare_data.py.
Training configuration
- Full-parameter BF16 fine-tuning
- Sequence length: 4,096
- Packing: enabled
- Assistant-only loss: enabled
- Per-device batch size: 4
- Gradient accumulation: 32 (effective batch size 128)
- Epochs: 1
- Learning rate: 2e-5
- Optimizer: AdamW 8-bit
- Warmup steps: 10
- Weight decay: 0.01
- Scheduler: linear
- Seed: 3407
- Attention: FlashAttention 2
- Hardware: 1x NVIDIA H100 80GB
Training results
- Optimizer steps: 1,092 / 1,092
- Final epoch: 1.0
- Training loss: 0.2867
- Mean token accuracy: 0.8947
- Tokens processed: 544,812,939
- Training runtime: 34,353 seconds (about 9 hours 33 minutes)