Employee Micro-SLM (15.3M - In-Context Document QA & Summarizer)
A custom 15.3-million parameter decoder-only Transformer built from scratch and trained for In-Context Reading Comprehension and Summarization on arbitrary employee data (JSON, XML, Key-Value text).
Highlights
- In-Context Learning: Accepts any arbitrary employee data (JSON / XML / text) in the prompt and extracts facts or generates summaries.
- Byte-Level BPE Tokenizer: 2,960 vocabulary items with 100% byte-fallback (0% UNK on any Unicode character or syntax).
- Ultra Compact: 15.3M parameters (~29.2 MB FP16 / ~14.6 MB INT8).
- Zero Hallucination: Answers and summaries are grounded strictly in the provided input document.
- Fast CPU & GPU Inference: Optimized Rotary Position Embeddings (RoPE), SwiGLU MLP, and RMSNorm.
Architecture
- Hidden Size: 384
- Layers: 6
- Attention Heads: 6
- FFN Dimension: 1,536 (SwiGLU)
- Vocabulary Size: 2,960
- Max Sequence Length: 512
- Parameters: 15,297,408
How to Use
import json
from tokenizers import Tokenizer
from transformers import AutoModelForCausalLM
from huggingface_hub import hf_hub_download
import torch
repo_id = "venky1/employee-microslm-int8"
tok = Tokenizer.from_file(hf_hub_download(repo_id, "tokenizer.json"))
model = AutoModelForCausalLM.from_pretrained(repo_id, trust_remote_code=True)
model.eval()
def analyze_employee(data, task, max_new_tokens=64):
context_str = json.dumps(data, indent=2) if isinstance(data, dict) else str(data).strip()
prompt = f"<bos>### Context:\n{context_str}\n\n### Task:\n{task}\n\n### Answer:\n"
tokens = tok.encode(prompt).ids
inp = torch.tensor([tokens])
out = model.generate(input_ids=inp, max_new_tokens=max_new_tokens, do_sample=False)
return tok.decode(out[0][len(tokens):].tolist()).replace("##", "").strip()
# 1. Provide ANY JSON
sample_json = {
"employee_id": "EMP-9481",
"name": "Siddharth Verma",
"department": "Cyber Security",
"role": "Security Architect",
"hours_worked": "8 hours and 45 minutes",
"project": "Zero Trust Firewall",
"worklog": "Audited firewall rules and patched SSL certificate vulnerability."
}
# Ask questions or request summaries
print(analyze_employee(sample_json, "What is the designation of Siddharth?"))
print(analyze_employee(sample_json, "Summarize this employee's workday."))