🏆 TinyChampion-3B-R1
TinyChampion-3B-R1 is a compact, high-efficiency 3-billion parameter reasoning language model developed by MAlexC.
Fine-tuned using distilled multi-step reasoning traces from DeepSeek-R1 (671B), TinyChampion-3B-R1 brings frontier-grade chain-of-thought <think> capabilities to edge devices and consumer GPUs (runs comfortably on <6 GB VRAM).
📊 Benchmark Results
| Benchmark | Metric | TinyChampion-3B-R1 | Base Model / Baseline 3B |
|---|---|---|---|
| GSM8K (Full 1,319 test set) | Strict Match (0-shot) | 89.31% | ~69.60% |
| GSM8K (Full 1,319 test set) | Flexible Match (0-shot) | 89.16% | ~69.83% |
| MMLU (57 subjects) | Accuracy (0-shot) | 61.01% | 60.12% |
| Hellaswag | Acc Norm (0-shot) | 62.23% | 61.46% |
| TruthfulQA | MC2 (0-shot) | 48.93% | 48.34% |
🧠 Reasoning Format (<think>)
The model reasons through problems before providing the final answer:
<|im_start|>user
Janet pays 0/hour for 3 hours of tennis lessons. She also buys 2 rackets for 0 each. How much did she spend in total?<|im_end|>
<|im_start|>assistant
<think>
1. Cost of tennis lessons: 3 hours * 0/hour = 20.
2. Cost of 2 rackets: 2 * 0 = 20.
3. Total expenditure: 20 + 20 = 40.
</think>
The answer is 240.<|im_end|>
🚀 Quickstart (Transformers)
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
model_id = "MAlexCCBC/TinyChampion-3B-R1"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
model_id,
dtype=torch.bfloat16,
device_map="auto"
)
messages = [
{"role": "system", "content": "You are TinyChampion, an expert reasoning AI assistant developed by MAlexC."},
{"role": "user", "content": "Solve: What is the sum of all prime numbers between 10 and 30?"}
]
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=1024, do_sample=False)
print(tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True))
🛠 Training Details
- Base Architecture: 3B Parameters
- Alignment: Distilled Long Chain-of-Thought (DeepSeek-R1 style) + Multi-task SFT
- Context Length: 2,048 tokens
- Precision: Pure 16-bit (
bfloat16) merged weights
Developed by MAlexC.