Hush-Nano-Chat
Hush-Nano-Chat is an English, single-turn instruction-tuned version of Soulitude/Hush-Nano. It starts from the 22M-parameter pretrained model and uses supervised fine-tuning (SFT) on instruction–response pairs.
Due to the model's limited parameters, its response can be inaccurate, incomplete, or inconsistent.
Model Details
Hush-Nano-Chat has the following features:
- Type: Causal Language Models
- Training Stage: Pretraining & Post-training
- Architecture: transformers with RMSNorm, RoPE, SwiGLU, QK-Norm and tied word embeddings
- Number of Parameters: 22M (22,621,056)
- Number of Layers: 12
- Number of Attention Heads (GQA): 6 for Q and 3 for KV
- Context Length: 1,024
SFT Data
The base model was pretrained on 8.5B tokens (8,554,042,292) drawn from the following subsets:
| Source | Training tokens | Share |
|---|---|---|
| FineWeb-Edu | 4,539,286,619 | 53.07% |
| DCLM | 2,890,209,171 | 33.79% |
| FineMath4plus | 1,124,546,502 | 13.15% |
Then it was fine-tuned on a mixture of the following datasets: unsloth/alpaca-cleaned, databricks/databricks-dolly-15k, and HuggingFaceH4/no_robots.
Each example is formatted as:
<|bos|>User:
{instruction + input}
Assistant:
{response}<|eos|>
Only the assistant response and ending EOS token contribute to the training loss.
Evaluation
Zero-shot normalized accuracy, evaluated in fp32 using EleutherAI/lm-evaluation-harness. Scores may vary slightly with the evaluation setup and environment.
| PIQA | ARC-Easy | ARC-Challenge | HellaSwag | |
|---|---|---|---|---|
| Hush-Nano | 58.27% | 38.93% | 21.84% | 28.89% |
| Hush-Nano-Chat | 59.09% | 39.94% | 22.44% | 28.75% |
Usage
This model includes custom Transformers code and so requires trust_remote_code=True.
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "Soulitude/Hush-Nano-Chat"
device = "cuda" if torch.cuda.is_available() else "cpu"
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
model_id,
trust_remote_code=True,
).to(device).eval()
message = [
{"role": "user", "content": "Explain why the sky appears blue in one sentence."},
]
inputs = tokenizer.apply_chat_template(
message,
add_generation_prompt=True,
return_tensors="pt",
return_dict=True,
).to(device)
with torch.inference_mode():
output = model.generate(
**inputs,
max_new_tokens=512,
do_sample=True,
temperature=0.7,
top_p=0.95,
repetition_penalty=1.1,
pad_token_id=tokenizer.pad_token_id,
eos_token_id=tokenizer.eos_token_id,
)
reply_ids = output[0, inputs["input_ids"].shape[1]:]
print(tokenizer.decode(reply_ids, skip_special_tokens=True).strip())
Intended Use and Limitations
Hush-Nano-Chat is intended for experimentation with small instruction-tuned language models. Its 22M-parameter size and 1,024-token context limit constrain instruction following, reasoning, factual reliability, and longer responses. The supervised data is primarily English, and this checkpoint is designed around single-turn prompts. Review generated output before relying on it.
License
Apache 2.0