LFM2.5-230M-Code-Math-Exp
An successful experimental fine-tune of LiquidAI/LFM2.5-230M (the instruct-tuned model, not the base checkpoint), focused on strengthening code generation across multiple languages, while retaining the general chat and instruction-following ability of the original instruct model.
Marked successful and experimental because training was stopped early (see below) and the model has only been evaluated on a fixed prompt suite, not a formal benchmark.
Shows major improvements over my previous model (hauser458b/lfm2.5-230m-code-math), delivering broader programming language support, better full-app generation, and reliable basic agentic behavior like sandbox file creation and terminal execution (tested locally via Q5_K_M quant).
Why this exists
LiquidAI's own model card for LFM2.5-230M states it is not recommended for reasoning-heavy workloads such as advanced math, code generation, or creative writing — the model is tuned primarily for data extraction, structured outputs, and lightweight agentic/tool-use tasks. This fine-tune pushes the small instruct model further into multi-language code competence.
Fine-tuning started from the instruct checkpoint rather than the base pretrain checkpoint, to preserve chat and instruction-following behavior that the base model doesn't have.
Training details
- Base model:
LiquidAI/LFM2.5-230M(instruct) - Method: Full fine-tune, bf16
- Datasets:
ise-uiuc/Magicoder-Evol-Instruct-110K— random 60k-example subsetiamtarun/code_instructions_120k_alpaca— full datasetopenai/gsm8k(main split) — full dataset
- Epochs / batch size: configured for 5 epochs, batch size 16, but training was stopped early at 9,800 steps — checkpoints beyond ~12k steps were tested and showed clearly worse output quality (less coherent, less accurate generations), so 9,800 was kept as the published checkpoint instead.
- Sequence length: 1024 tokens
- Loss: completion-only (loss computed only on assistant responses, not prompts)
- Checkpoint: saved directly from the training run at the stopping point (not selected via
load_best_model_at_end, since the run was interrupted rather than completed) — see Known Limitations below
Evaluation
Evaluated on a fixed suite of 90 hand-written prompts: 20 Python, 20 Java, 5 each of HTML/CSS/JavaScript, 5 combined HTML+CSS+JS mini-projects, 10 C++, 10 Rust, and 10 larger "build a complete program" prompts (calculator, to-do app, quiz game, etc.), compared against hauser458b/lfm2.5-230m-code-math.
This is a manual, single-run, single-temperature comparison (temperature 0.3) — not a formal benchmark, and results are anecdotal rather than statistically rigorous.
Where this model did better:
- Java: fewer broken/non-compiling outputs (e.g. no literal syntax errors, correct handling of tasks like title-casing and leap-year checks)
- JavaScript: more coherent DOM manipulation logic on the "add list item" style task
- Combined HTML+CSS+JS mini-projects: more functional end-to-end logic (e.g. quiz-answer checking actually evaluates a real answer rather than looping over hardcoded strings)
- Full "build a project" prompts: calculator, to-do app, and rock-paper-scissors implementations were more likely to be logically complete and runnable
Where it was roughly even: HTML, CSS, C++
Where it did NOT clearly improve, or regressed:
- Rust remains weak in both models. Neither dataset used here was Rust-heavy, and this shows: several Rust outputs from this model don't compile as-is (e.g. missing braces, incorrect struct/method logic). It's somewhat less broken than the comparison model's Rust output, but "less broken" is not "reliable" — treat all Rust output from this model as a rough draft, not working code.
Known limitations
- Early stopping: training was manually interrupted before the configured 5 epochs completed, and the final checkpoint was saved directly rather than through automatic best-checkpoint selection. There is no guarantee this exact checkpoint is the single best point in the run — it's simply where training was stopped.
- Rust output is unreliable — expect syntax errors and logic bugs; always compile-check before trusting Rust output from this model.
- Occasional logic bugs even in "solid" languages — e.g. one Python anagram implementation in testing called a string method that doesn't exist. Review generated code before running it, especially anything involving user input, file I/O, or external calls.
- Multi-file / multi-technology tasks (e.g. combined HTML+CSS+JS builds) are the model's weakest spot even where it improved over the comparison model — interactivity is sometimes only partially wired up (e.g. a button that updates the wrong element).
- Not evaluated on data extraction, RAG, tool-calling, or general chat — this fine-tune was evaluated purely on code-generation prompts. If you need the original model's data-extraction / agentic strengths, this model has not been validated for that use case.
- Still a 230M-parameter model — do not expect deep multi-step reasoning or production-grade code without review.
- Not evaluated on safety-critical, medical, or legal use cases — do not use for those without additional safeguards.
Usage
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "hauser458b/lfm2.5-230m-code-math-exp"
model = AutoModelForCausalLM.from_pretrained(model_id, dtype="bfloat16", device_map="auto")
tokenizer = AutoTokenizer.from_pretrained(model_id)
messages = [{"role": "user", "content": "Write a Python function to check if a number is prime."}]
inputs = tokenizer.apply_chat_template(
messages, add_generation_prompt=True, return_tensors="pt", return_dict=True
).to(model.device)
output = model.generate(**inputs, max_new_tokens=300, do_sample=True, temperature=0.3, top_p=0.9)
print(tokenizer.decode(output[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True))
License
Inherits the LFM Open License v1.0 from the base model.
Acknowledgements
Built on LiquidAI/LFM2.5-230M. See the LFM2 Technical Report for details on the base architecture. Compared against hauser458b/lfm2.5-230m-code-math during evaluation.