Dwarf-15M-Instruct
A 15.54M parameter shell/bash specialist language model that translates natural language into Linux commands.
Quick Start
from transformers import AutoTokenizer, AutoModelForCausalLM
tokenizer = AutoTokenizer.from_pretrained("ThingAI/Dwarf-15M-Instruct", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("ThingAI/Dwarf-15M-Instruct", trust_remote_code=True)
prompt = "<|user|>\nFind all Python files modified in the last 3 days\n<|end|>\n<|assistant|>\n"
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=100, do_sample=False)
print(tokenizer.decode(outputs[0], skip_special_tokens=False))
# → find . -name '*.py' -mtime -3
What It Does
Dwarf-15M takes natural language descriptions of tasks and produces the corresponding Linux/bash command:
| Prompt | Output |
|---|---|
| Show current date | date |
| List files | ls |
| Kill process 1234 | kill 1234 |
| Delete all .tmp files in current directory | rm ./*.tmp |
| Compress the /home/user/project folder | tar -czf project.tar.gz /home/user/project/ |
| Check if port 8080 is in use | ss -tlnp | grep 8080 |
| Restart the nginx service | sudo systemctl restart nginx |
| Find all files containing TODO | grep -rl 'TODO' . |
| Change owner of /var/www to www-data | sudo chown -R www-data:www-data /var/www |
| Run script.sh in background and log output | nohup ./script.sh > log.txt 2>&1 & |
| Find and replace foo with bar in config.txt | sed -i 's/foo/bar/g' config.txt |
| What does chmod 755 do? | chmod 755 sets read+write+execute for owner, read+execute for group and others. |
| Write a bash function that counts lines | count_lines() { wc -l < "$1"; } |
Architecture
| Parameter | Value |
|---|---|
| Parameters | 15.54M |
| Layers | 12 |
| Hidden dim | 320 |
| Attention | GQA (5 query, 1 KV head) |
| FFN | SwiGLU (d_ff=864) |
| Normalization | RMSNorm |
| Positional | RoPE (θ=10000) |
| Vocabulary | 8,202 (DwarfGoToken) |
| Max sequence | 2,048 |
| Weight tying | Yes (embed ↔ lm_head) |
Training
Pretraining: 21.6B tokens (ratio 1,390:1) on 11 datasets:
- Shell/bash: The Stack (shell, batchfile), GunA-SD/bash_code — 38.5%
- Code: The Stack (Python, C), CodeFeedback — 39.1%
- Instructions: ShellLife (52K NL→command), rlvr-code-data-bash (133K problems) — 11%
- English: helpful-instructions, FineWeb — 10.3%
- CoT: Magpie-Reasoning — 1.1%
SFT: 557 curated Linux command pairs, 5 epochs, lr=4e-5. Training time: 19 seconds.
Tokenizer: DwarfGoToken — 8,202 token BPE with syntax-aware pre-tokenization for shell operators (2>&1, &&, >>).
Chat Template
<|user|>
Your question here
<|end|>
<|assistant|>
Model response here
<|end|>
Intended Use
Dwarf-15M is designed as a CLI assistant that suggests commands for user review before execution. It is NOT a general-purpose chatbot. Best results on:
- Simple to medium Linux commands (file ops, process management, networking)
- Bash one-liners and short functions
- Command explanations ("What does chmod 755 do?")
Limitations
- 15M parameters — cannot handle complex multi-step reasoning
- May produce incorrect commands for unusual or very specific requests
- Should always be used with human review before executing any suggested command
- English only
- Trained primarily on Ubuntu/Debian commands
Hardware
- Pretrained on RTX 3070 (8GB VRAM) at ~127K tokens/sec
- Inference: runs on any hardware including CPU, ~30MB model size
License
Apache 2.0
Citation
@misc{dwarf15m2026,
title={Dwarf-15M-Instruct: A Shell Specialist Language Model},
author={ThingsAI},
year={2026},
url={https://huggingface.co/ThingAI/Dwarf-15M-Instruct}
}