SmolLM2-135M-Alpaca-SFT
A learning project: HuggingFaceTB/SmolLM2-135M-Instruct fully fine-tuned (SFT) with TRL on
3916 examples from yahma/alpaca-cleaned on a MacBook Pro (Apple Silicon, MPS).
| split | examples |
|---|---|
| train | 3916 |
| validation | 494 |
| test | 493 |
| model | test loss | test perplexity |
|---|---|---|
| base | 1.5779 | 4.84 |
| fine-tuned | 1.4898 | 4.44 |
Hyper-parameters: epochs=2, lr=5e-05, effective batch=16, max_length=512, cosine schedule, loss on completions only.
Usage
from transformers import pipeline
pipe = pipeline("text-generation", model="1000kavya/SmolLM2-135M-Alpaca-SFT")
print(pipe([{"role": "user", "content": "Give three tips for staying healthy."}], max_new_tokens=200)[0]["generated_text"][-1]["content"])
Demo: https://huggingface.co/spaces/1000kavya/SmolLM2-135M-Alpaca-SFT-Chat
This is a 135M-parameter model built for learning; expect factual mistakes.