slm-125m-legal-sft
A 125M-parameter Llama-architecture model, supervised fine-tuned from
abhishekai/slm-125m-legal-base
for grounded question answering over legal and financial passages.
Unlike the base model (a text completer), this one follows instructions — but in a RAFT / retrieval-augmented style: it answers a question from a context passage you provide, which is the reliable behaviour a model this small can actually deliver. Remove the context and it has little to stand on.
How it was tuned
The base model was fine-tuned on ~9,100 grounded Q&A pairs (plus an 800-pair held-out eval split). The data was generated by a teacher LLM (Gemini 2.5 Flash) reading passages sampled from the same legal/financial corpus the base model was pretrained on, then aggressively filtered:
- Grounding / faithfulness — an LLM-judge self-check; only answers scored fully supported by their source passage were kept.
- Deduplication — near-identical questions removed by embedding similarity.
- Length / format / schema — truncated, empty, or malformed rows dropped.
- Task & difficulty balance — five task types (grounded QA, multi-step QA, summarization, extraction, rewriting) at a fixed mix, spanning easy → hard.
- Eval decontamination — n-gram and embedding overlap between the training and eval splits removed, so reported eval numbers are not leaked training data.
Reused the base model's own 16K byte-level BPE tokenizer (no re-tokenization). A chat template is bundled and used at both training and inference.
Independent absolute score (2026-08-15)
Scored by a Claude Sonnet judge on four axes out of 10, over 300 held-out grounded Q&A prompts, on the same scale as every other checkpoint in this project.
| mean score | 4.05 / 10 |
| 95% CI | [3.73, 4.37] |
| paired step | +2.59 over the 125M pretrain base (1.46), CI [+2.26, +2.91] |
Per axis: QA 3.16, instruction following 3.27, grounding 4.24, refusal 5.52. Both RLAIF stages built on this checkpoint score no better — DPO 3.52 (a real regression) and PPO 4.08 (indistinguishable). At this scale the SFT is the model to use.
Training procedure
| Method | full-parameter SFT, loss on answer tokens only (prompt masked) |
| Examples | 9,096 train / 794 eval |
| Epochs | 3 (1,704 steps, micro-batch 16) |
| Optimizer | AdamW (beta 0.9/0.95) |
| LR schedule | 3% warmup to 2e-5, cosine decay to 2e-6 |
| Format | chat template; context folded into the user turn, RAFT-style |
| Hardware | 1x H100 |
| Compute cost | ~$0.32 (data generation ~$2.44 on top) |
Evaluation
Cross-entropy over answer tokens only on the held-out eval split:
| Metric | Value |
|---|---|
| Eval loss (answer tokens) | 1.213 |
| Eval perplexity (answer tokens) | 3.36 |
This is not comparable to the base model's 9.155 — that is general next-token perplexity over the whole sequence; this is loss over supervised answer tokens on grounded QA, an easier target. No downstream task benchmarks (LegalBench, CaseHOLD, etc.) have been run.
Usage
Give it a question and a context passage. Greedy decoding is strongly recommended — at higher temperatures a model this small drifts into the wrong task type.
from transformers import AutoTokenizer, AutoModelForCausalLM
tok = AutoTokenizer.from_pretrained("abhishekai/slm-125m-legal-sft")
model = AutoModelForCausalLM.from_pretrained("abhishekai/slm-125m-legal-sft").eval()
question = "What standard of proof applies, and what must the plaintiff show?"
context = (
"In a civil negligence action the plaintiff must establish duty, breach, "
"causation, and damages. The standard of proof is a preponderance of the "
"evidence: the plaintiff must show it is more likely than not that the "
"defendant's conduct caused the harm."
)
messages = [
{"role": "system", "content": "You are a precise legal and financial assistant. "
"Answer using only the provided context. If the context does not contain the "
"answer, say you cannot answer from the context."},
{"role": "user", "content": f"{question}\n\nContext:\n{context}"},
]
text = tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
ids = tok(text, add_special_tokens=False, return_tensors="pt").input_ids
out = model.generate(ids, max_new_tokens=160, do_sample=False,
eos_token_id=tok.convert_tokens_to_ids("<|eos|>"),
pad_token_id=tok.convert_tokens_to_ids("<|pad|>"))
print(tok.decode(out[0, ids.shape[1]:], skip_special_tokens=True))
Limitations and intended use
A research and educational artifact — a demonstration that a coherent, grounded instruction-following model can be distilled onto a 125M base for a few dollars. Not suitable for production, and specifically:
- It needs a context passage. It was trained to answer from provided context, not from parametric memory. Closed-book questions are out of distribution.
- It does not do arithmetic. It will confidently state figures and percentages that do not follow from each other. Never trust a number it computes.
- It still hallucinates and is not a source of legal or financial fact, and is not legal or financial advice. Do not deploy it in any advisory, compliance, or decision-making capacity.
- Use greedy / low temperature. Sampling at higher temperatures makes a model this small slip into the wrong task (e.g. emitting JSON for a prose question).
- 1024-token context, so question + passage together must be short.
- English only, US-jurisdiction-skewed, and it inherits the biases of both its pretraining corpus and the teacher model that generated its fine-tuning data.
Data provenance and license
The model weights are released under Apache-2.0, inheriting the base model's license.
The fine-tuning data is synthetic, generated by Google's Gemini 2.5 Flash over passages from the base model's pretraining corpus (US case law — CC-BY-4.0; SEC filings — CC0-1.0; FineWeb-Edu — ODC-By 1.0). Synthetic data carries the teacher provider's usage terms; this artifact is published for research and education. If you intend to use it commercially, review Google's Gemini API terms and get your own counsel to look at it. None of this is legal advice.
Reproducibility
The full fine-tuning pipeline — passage sampling, teacher generation, LLM-judge grounding, deduplication, task balancing, eval decontamination, tokenization, and the SFT run — runs on Modal and is reproducible end-to-end.