GPT-OSS-Nano
Compact Reasoning Model with Mixture of Experts
9B parameters • 12 experts • 128K context • Chain-of-thought reasoning
📋 Model Description
GPT-OSS-Nano is a fine-tuned Mixture of Experts (MoE) language model optimized for step-by-step reasoning and problem solving. Built on the GPT-OSS architecture with sparse expert activation, it achieves strong reasoning performance while using only ~3B active parameters per forward pass.
✨ Key Features
| Feature | Description |
|---|---|
| 🧠 Sparse MoE | 12 experts, 4 active per token — efficient compute |
| 📝 Chain-of-Thought | Fine-tuned on reasoning datasets with step-by-step solutions |
| ⚡ 128K Context | Long context with YaRN rope scaling |
| 🔮 Q-GPT Ready | Compatible with quantum confidence estimation |
| 📦 GGUF Available | Run locally with llama.cpp or Ollama |
🏗️ Architecture
┌─────────────────────────────────────────────────────────┐
│ GPT-OSS-Nano │
├─────────────────────────────────────────────────────────┤
│ Total Parameters │ 9.0 Billion │
│ Active Parameters │ ~3 Billion (per forward pass) │
│ Hidden Dimension │ 2880 │
│ Attention Heads │ 64 (8 KV heads, GQA) │
│ Layers │ 24 │
│ Experts │ 12 total, 4 active │
│ Context Length │ 131,072 tokens │
│ Vocabulary Size │ 201,088 │
│ Precision │ BFloat16 │
└─────────────────────────────────────────────────────────┘
💻 Usage
Quick Start with Transformers
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
model = AutoModelForCausalLM.from_pretrained(
"squ11z1/gpt-oss-nano",
torch_dtype=torch.bfloat16,
device_map="auto",
trust_remote_code=True,
)
tokenizer = AutoTokenizer.from_pretrained(
"squ11z1/gpt-oss-nano",
trust_remote_code=True,
)
prompt = """Solve this step by step:
A store offers 20% off on all items. If a jacket costs $85,
what is the final price after discount?"""
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(
**inputs,
max_new_tokens=256,
temperature=0.7,
do_sample=True,
)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
⚡ With Unsloth (2x Faster)
from unsloth import FastLanguageModel
model, tokenizer = FastLanguageModel.from_pretrained(
"squ11z1/gpt-oss-nano",
dtype=None,
load_in_4bit=True, # 4-bit quantization for efficiency
)
# For inference
FastLanguageModel.for_inference(model)
📦 With GGUF (llama.cpp)
# Download the quantized model
wget https://huggingface.co/squ11z1/gpt-oss-nano/resolve/main/gpt-oss-9b-q4_k_m.gguf
# Run inference
./llama-cli -m gpt-oss-9b-q4_k_m.gguf \
-p "Solve step by step: What is 15% of 240?" \
-n 256 --temp 0.7
🦙 With Ollama
# Create Modelfile
echo 'FROM ./gpt-oss-9b-q4_k_m.gguf' > Modelfile
ollama create gpt-oss-nano -f Modelfile
# Run
ollama run gpt-oss-nano "Explain quantum computing simply"
🎓 Training
| Parameter | Value |
|---|---|
| Base Model | openai/gpt-oss-20b |
| Method | QLoRA (4-bit quantized LoRA) |
| LoRA Rank | 32 |
| LoRA Alpha | 32 |
| Learning Rate | 2e-4 |
| Batch Size | 2 (gradient accumulation: 8) |
| Epochs | 2 |
| Framework | Unsloth + TRL |
| Hardware | NVIDIA H200 |
Dataset: Superior-Reasoning — chain-of-thought examples with step-by-step problem solving.
🔮 Q-GPT: Quantum Confidence
GPT-OSS-Nano is compatible with Q-GPT — a quantum neural network that estimates response confidence.
from q_gpt import load_qgpt
model, tokenizer = load_qgpt("squ11z1/gpt-oss-nano")
outputs = model.generate_with_confidence(inputs, max_new_tokens=256)
print(f"Response confidence: {outputs['confidence_label']}")
# Output: "high", "moderate", "low", etc.
if outputs['should_refuse']:
print("⚠️ Model is uncertain — consider refusing to answer")
Learn more: squ11z1/Q-GPT
⚠️ Limitations
- Language: Primarily optimized for English; multilingual performance varies
- Hallucinations: May generate plausible but incorrect information on obscure topics
- Safety: Not designed for safety-critical applications without validation
- Math: Strong at arithmetic reasoning; weaker on advanced mathematics
📜 License
This model is released under the Apache 2.0 License.
🙏 Acknowledgments
- Unsloth — 2x faster fine-tuning
- OpenAI — GPT-OSS base model
- llama.cpp — GGUF format and quantization
📖 Citation
@misc{gptossnano2026,
title={GPT-OSS-Nano: Compact MoE Reasoning Model},
author={squ11z1},
year={2026},
publisher={Hugging Face},
url={https://huggingface.co/squ11z1/gpt-oss-nano}
}
Pro Mundi Vita