slm-125m-legal-financial-sft

Instruction/QA fine-tune of thesreedath/slm-125m-base, a 125M Llama-style model for US legal & financial text. Fine-tuned on 8,000 grounded chat Q&A pairs synthesized with GPT-4o-mini and quality-judged by Gemini 2.5 Flash (dedup + eval-decontaminated). Trained 3 epochs on 1x H100.

Chat format

<|bos|><|system|>
{system}
<|user|>
Context:
{passage}

Question: {question}
<|assistant|>
{answer}<|eos|>

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
tok = AutoTokenizer.from_pretrained("rahulreddyhanu/slm-125m-legal-financial-sft")
model = AutoModelForCausalLM.from_pretrained("rahulreddyhanu/slm-125m-legal-financial-sft", torch_dtype=torch.bfloat16)
prompt = ("<|bos|><|system|>\nYou are a helpful assistant specialized in US legal "
          "and financial documents. Answer using only the provided context.\n"
          "<|user|>\nContext:\nThe Company reported net revenues of $4.2 million "
          "for fiscal 2021.\n\nQuestion: What were net revenues in fiscal 2021?\n"
          "<|assistant|>\n")
ids = tok(prompt, return_tensors="pt", add_special_tokens=False)
ids.pop("token_type_ids", None)
print(tok.decode(model.generate(**ids, max_new_tokens=60,
      eos_token_id=tok.convert_tokens_to_ids("<|eos|>"))[0][ids["input_ids"].shape[1]:],
      skip_special_tokens=True))

Limitations

125M parameters — concise grounded answers, but limited factual reliability and occasional errors. Research/demo use.