Shreyansh-STEM-1.5B-Base
Shreyansh-STEM-1.5B-Base is an open-weights foundational Small Language Model (SLM) engineered specifically for multi-disciplinary STEM reasoning, algorithmic synthesis, and scientific computing.
Built upon the high-throughput OLMo-2 architecture (Olmo2ForCausalLM), this base checkpoint provides a dense, un-instructed parameter space optimized for downstream domain adaptation, post-training SFT, and parameter-efficient fine-tuning (LoRA/QLoRA).
Model Overview
- Architecture: OLMo-2 Autoregressive Transformer
- Parameters: ~1.5 Billion
- Vocabulary Size: 100,352
- Target Domains: Physics, Pure Mathematics, Organic Chemistry, Computational Biology, Hardware/EDA Logic (Verilog/VHDL), Computer Science
- Format: SafeTensors (bfloat16 compatible)
- License: Apache 2.0
Intended Use
This repository hosts the raw pre-trained base model weights. It is designed for:
- Domain Adaptation & SFT: Fine-tuning on specialized instruction or Chain-of-Thought (CoT) datasets.
- Edge & Mobile Deployment: Compiling and quantizing into GGUF/AWQ/EXL2 formats for on-device, offline inference.
- Knowledge Distillation: Acting as a high-density student model for distilling knowledge from larger teacher models (7B/32B/70B).
Note: As a base model, it is not instruction-tuned or aligned with RLHF. For zero-shot chat or question answering, please apply appropriate prompting, few-shot demonstration, or use downstream instruction-tuned variants.
Quickstart (Transformers)
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "shreyansh12183/Shreyansh-STEM-1.5B-Base"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
model_id,
torch_dtype=torch.bfloat16,
device_map="auto"
)
prompt = "Derive the Schrödinger wave equation from first principles:\n"
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
with torch.no_grad():
outputs = model.generate(
**inputs,
max_new_tokens=256,
temperature=0.7,
top_p=0.9
)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
Citation & Attribution
If you use this model in your research or downstream engineering workflows, please cite:
@misc{shreyansh2026stem15b,
author = {Shreyansh Singh},
title = {Shreyansh-STEM-1.5B-Base: A Foundational Small Language Model for STEM Reasoning},
year = {2026},
publisher = {Hugging Face},
howpublished = {\url{https://huggingface.co/shreyansh12183/Shreyansh-STEM-1.5B-Base}}
}