Edu_slm v1.0
Team Edu_slm domain-specific SLM for JEE Main / JEE Advanced Physics, Chemistry, and Mathematics.
This folder is a merged Hugging Face checkpoint. Load it with:
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained("Edu_slm")
tokenizer = AutoTokenizer.from_pretrained("Edu_slm")
No custom trust_remote_code conversion is required. Architecture is Qwen2ForCausalLM.
Key facts
- Base model:
deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B - Training method: LoRA SFT (
r=32,alpha=64), thenmerge_and_unload - Parameter count: 1,777,088,000
- Vocab size: 151,936 (matches
config.json) - pad_token: eos_token (
<|end▁of▁sentence|>, id 151643) - Precision: bfloat16
- Chat template: DeepSeek-R1 / Qwen2 (
<|User|>/<|Assistant|><think>) - Training log:
train_log.jsonl