🦉 Owl-v1
1. Model Intro
Owl-v1 is a lightweight 10.8 M parameter language model built from scratch for learning and integration purposes.
2. What Can It Generate?
Character name / scene opener prompts.
| Prompt | Output | Quality |
|---|---|---|
"HAMLET:\n" | Character dialogue in Early Modern English | ⭐⭐⭐ |
"Enter KING and QUEEN.\n" | Stage directions + speech | ⭐⭐ |
"To be, or not to be" | Probabilistic monologue continuation | ⭐⭐ |
"Shall I compare thee" | Poetic continuation (metre may drift) | ⭐ |
"\n" (cold start) | Free-form dramatic scene | ⭐⭐ |
3. Usage
Requires trust_remote_code=True — CharTokenizer and ModelArchitecture are custom classes.
from transformers import AutoTokenizer, AutoModelForCausalLM
import torch
tokenizer = AutoTokenizer.from_pretrained("IstishadAlamTishad/Owl-v1", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("IstishadAlamTishad/Owl-v1", trust_remote_code=True)
model.eval()
ids = tokenizer.encode("HAMLET:\n", return_tensors="pt")
with torch.no_grad():
out = model.generate(ids, max_new_tokens=200, do_sample=True, temperature=0.8, top_k=40)
print(tokenizer.decode(out[0].tolist()))
4. Model Training & Specs
| Dataset | Tokens | Steps | Batch |
|---|---|---|---|
| TinyShakespeare | 1,115,394 | 5,000 | 64 |
| Property | Value |
|---|---|
| Parameters | 10.8 M |
| Layers / Heads / D_embd | 6 / 6 / 384 |
| Context length | 256 tokens |
| Vocab | 65 chars |
5. Performance Metrics Curves
 |  |
|---|---|
| Fig 1 — Perplexity: Train PPL → ~1, Val PPL bottoms at ~5 (step ~800) then climbs to ~50 — clear overfitting on the small corpus. | Fig 2 — Throughput: Stable ~37–38 k tokens/sec after a brief warm-up spike. |
Built for learning and integration