LFM2.5-230M-Code-Math
A fine-tune of LiquidAI/LFM2.5-230M (the instruct-tuned model, not the base checkpoint) focused on strengthening code generation and math word-problem solving, while retaining the general chat and instruction-following ability of the original instruct model.
Why this exists
LiquidAI's own model card for LFM2.5-230M states it is not recommended for reasoning-heavy workloads such as advanced math, code generation, or creative writing — the model is tuned primarily for data extraction, structured outputs, and lightweight agentic/tool-use tasks. This fine-tune is an attempt to push a small, efficient instruct model further into code and math competence without sacrificing its existing conversational ability.
Fine-tuning started from the instruct checkpoint rather than the base pretrain checkpoint, specifically to preserve chat and instruction-following behavior that the base model doesn't have. An earlier fine-tune attempt starting from LFM2.5-230M-Base produced a model that was strong at code/math but broke down on basic conversation (e.g. echoing "Hello, who are you?" back verbatim). Starting from instruct avoided this.
Training details
- Base model:
LiquidAI/LFM2.5-230M(instruct) - Method: Full fine-tune (LoRA would also work at this scale; full-FT was used here since compute wasn't a constraint)
- Datasets:
- Code:
iamtarun/code_instructions_120k_alpaca - Math:
openai/gsm8k(main split)
- Code:
- Checkpoint selection: best checkpoint by eval loss (not final step) — training showed clear overfitting past ~step 7500, where training loss kept falling but eval loss plateaued/rose slightly. The published checkpoint is from before that point.
- Sequence length: 1024 tokens (dataset is short-form; base model supports up to 32K context)
- Loss: completion-only (loss computed only on assistant responses, not prompts)
What it's good at
Based on manual testing across ~20+ prompts spanning algebra, geometry, general code tasks, and open-ended chat:
- Code: Reliable on common patterns — string/list manipulation, simple classes, recursion, file I/O, prime checking, etc. In the author's own informal side-by-side testing, output was clearer and more consistent than Qwen2.5-Coder-0.5B-Instruct on the same prompts. This is a subjective, single-user comparison, not a formal benchmark — your results may differ.
- Math: Grade-school word problems (gsm8k-style), percentages, basic algebra, geometry (area/perimeter) — mostly correct with gsm8k-style step annotations.
- Chat: Retains coherent, on-topic conversational ability inherited from the instruct base — no repetition loops or echo failures observed in testing.
- Tool calling: Spot-checked informally by the author using the Pythonic tool-call format LFM2.5 supports; not systematically benchmarked against other models.
Known limitations
- Occasional arithmetic slip on multi-step algebra (e.g., correct method shown, final division not simplified).
- Not tested on data extraction or RAG.
- Still a 230M-parameter model — do not expect deep multi-step reasoning, advanced math, or long-form creative writing at the level of much larger models.
- Not evaluated on safety-critical, medical, or legal use cases — do not use for those without additional safeguards.
Usage
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "hauser458original/lfm2.5-230m-code-math"
model = AutoModelForCausalLM.from_pretrained(model_id, dtype="bfloat16", device_map="auto")
tokenizer = AutoTokenizer.from_pretrained(model_id)
messages = [{"role": "user", "content": "Write a Python function to check if a number is prime."}]
inputs = tokenizer.apply_chat_template(
messages, add_generation_prompt=True, return_tensors="pt", return_dict=True
).to(model.device)
output = model.generate(**inputs, max_new_tokens=300, do_sample=True, temperature=0.3, top_p=0.9)
print(tokenizer.decode(output[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True))
GGUF quantized versions (Q4_K_M, Q5_K_S, Q5_K_M, Q8_0, F16) for llama.cpp/Ollama/LM Studio are available at: hauser458original/lfm2.5-230m-code-math-GGUF
License
Inherits the LFM Open License v1.0 from the base model.
Acknowledgements
Built on LiquidAI/LFM2.5-230M. See the LFM2 Technical Report for details on the base architecture.