Table of Contents
TL;DR
Model Details
Model Description
- Developed by: https://www.tii.ae
- Model type: Causal decoder-only
- Architecture: Hybrid Transformers + Mamba architecture
- Language(s) (NLP): English
- Number of Parameters: 90M
- License: Falcon-LLM License
Training details
For more details about the training protocol of this model, please refer to the Falcon-H1-Tiny technical blogpost.
Usage
Currently to use this model you can either rely on Hugging Face transformers, vLLM, sglang, llama.cpp, ollama or mlx library.
Inference
🤗 transformers
Refer to the snippet below to run H1 models using 🤗 transformers:
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "tiiuae/Falcon-H1-Tiny-R-90M"
model = AutoModelForCausalLM.from_pretrained(
model_id,
torch_dtype=torch.bfloat16,
device_map="auto"
)
# Perform text generation
or
transformers serve tiiuae/Falcon-H1-Tiny-R-90M
llama.cpp
You can find all GGUF files compatible with llama.cpp under our official collection - an example setup could be:
brew install llama.cpp
pip install huggingface_hub
hf download tiiuae/Falcon-H1-Tiny-R-90M-GGUF Falcon-H1-Tiny-R-90M-Q8_0.gguf --local-dir ./
llama-cli ./Falcon-H1-Tiny-R-90M-Q8_0.gguf -cnv
ollama
ollama run hf.co/tiiuae/Falcon-H1-Tiny-R-90M:Q8_0
Apple mlx
mlx_lm.chat --model tiiuae/Falcon-H1-Tiny-R-90M
vLLM
For vLLM, simply start a server by executing the command below:
# pip install vllm>=0.9.0
vllm serve tiiuae/Falcon-H1-Tiny-R-90M --tensor-parallel-size 2 --data-parallel-size 1
sglang
python -m sglang.launch_server \
--model ttiiuae/Falcon-H1-Tiny-R-90M \
--tensor-parallel-size 1
Evaluation
For detailed evaluation of Falcon-H1-Tiny series, please refer to our technical blogpost
Useful links
- View our release blogpost.
- Feel free to join our discord server if you have any questions or to interact with our researchers and developers.
Citation
If the Falcon-H1-Tiny family of models were helpful to your work, feel free to give us a cite.
@misc{falcon_h1_tiny,
title={Falcon-H1-Tiny: A series of extremely small, yet powerful language models redefining capabilities at small scale},
author={Falcon-LLM Team},
year={2026},
}