EXLLM-0.005B-Instruct

EXLLM-0.005B-Instruct is a 0.005377824B-parameter Japanese instruction model for local inference on low-memory embedded hardware. Physical-device inference is verified on the CASIO EX-word XD-B4800 (DATAPLUS 6) and XD-N6500 (DATAPLUS 7). A PyTorch reference runtime is provided for host-side inference and evaluation. This repository also includes a separately trained 0.005441184B-parameter Llama-compatible GGUF companion for LM Studio and llama.cpp.

Here, LLM stands for Little Language Model.

The embedded checkpoint uses a custom decoder-only Transformer and a hybrid character/UTF-8-byte tokenizer; it is not directly compatible with standard transformers auto classes or GGUF runtimes. Use EXLLM-0.005B-LMStudio-F16.gguf for LM Studio or llama.cpp. The GGUF is a companion model trained on the EXLLM project data, not a numerical conversion of the embedded checkpoint.

日本語版の説明はこちらです。

Highlights

• 0.005377824B parameters
• 0.000868M-token vocabulary with UTF-8 byte fallback
• 0.000128M-token context length
• 0.021526737 GB fp32 checkpoint
• 0.005443105 GB int8 and EXQ12 deployment artifacts
• Reference inference at 239.082 token/s on an RTX 3060
• Integer inference at 0.52–0.55 token/s on a 0.024184 GHz SH-4A-class CPU
• Apache-2.0 weights and reference code
• Ready-to-run 0.010915168 GB F16 GGUF companion for LM Studio

Model Details

The parameter count reports unique trainable parameters. The tied language-model head and token embedding refer to the same parameter tensor.

Training

EXLLM was trained only on project-generated Japanese instruction/response data. It does not use an external pretrained checkpoint or imported public text corpus. The EXLLM project originates from random initialization. The 0.005B release was expanded from an earlier in-project EXLLM checkpoint by transplanting dimension-compatible parameters, followed by staged training and corrective fine-tuning. No external pretrained checkpoint was used.

The published staged files include overlapping base, robustness, recovery, and corrective sets; 0.086408M is a file-record total, not a deduplicated example count. Exact end-to-end training time and training-device telemetry were not retained in the release checkpoint and are therefore not reported.

Data scope and weight lineage are documented in DATA_PROVENANCE.md. The machine-readable release metadata is available in training/release-5m.json. The direct v1.0 parent and exact 5M stage parameters and hashes are published in training/release-5m-stages.json. The source repository provides all staged datasets and tools/replay_5m.py.

Inference Performance

All results use greedy decoding. TTFT means time to first generated token. Decode throughput excludes the first generated token.

RTX 3060 methodology

• Driver 580.178.04; Python 3.12.13; CUDA 13.0.
• 40 warm-up inferences followed by 120 measured inferences.
• Median TTFT: 0.003990 s; observed range: 0.003913–0.004072 s.
• Median total generation time: 0.079589 s.
• Tukey 1.5 IQR filtering removed 0/120 observations for TTFT, total time, and decode throughput.
• The first cold warm-up inference recorded 0.193056 s TTFT.
• Raw per-run results are published in benchmarks/rtx3060-cuda-robust-20260925.json.

EX-word methodology

• Five-run baseline: CASIO EX-word XD-B4800 / DATAPLUS 6.
• SH7305-family SH-4A, 1 core / 1 thread; measured instruction clock 0.024184 GHz.
• Five prompts, Thinking disabled, model stored in internal storage.
• Application UI rendering and UTF-8 display handling are included.
• Median TTFT: 23.711 s; observed range: 12.705–28.440 s.
• Median total generation time: 54.145 s.

A matched v1.2.0 deployment check used the same model.q12 and Thinking OFF setting on both devices. Two こんにちは runs per device produced the same 24-token answer at 0.55 token/s. Median TTFT / total time was 12.702 s / 54.138 s on XD-B4800 and 12.707 s / 54.164 s on XD-N6500. Four matched fixed prompts differed by at most 11 ms in TTFT and 32 ms in total time, with no consistent winner. The recovered logs therefore validate deployment compatibility but do not establish a meaningful performance difference between these generations.

The two runtimes use the same architecture and training lineage but different numerical representations. The results are deployment measurements, not a direct hardware-compute comparison.

Evaluation

The release was evaluated with a project-specific regression suite after dequantizing the EXLLM8 artifact into the reference runtime.

These are internal regression tests for the intended application scope. They are not standard language-model benchmarks and should not be interpreted as evidence of broad knowledge, general reasoning, or benchmark comparability with larger models.

Usage

LM Studio / llama.cpp

In LM Studio, search for ToTo-40417/EXLLM, download EXLLM-0.005B-LMStudio-F16.gguf, and start a chat. The required single-turn chat template is embedded in the GGUF. Set the loaded context length to 512 tokens and use greedy decoding or a low temperature; the model is intentionally tiny and is intended for short Japanese prompts. Training sequences were limited to 128 tokens; the larger runtime window reserves space for LM Studio's chat wrapper and short history.

With llama.cpp:

llama-cli -m EXLLM-0.005B-LMStudio-F16.gguf \
  -p "RAMとは何ですか？" -n 48 --temp 0 --ctx-size 512 --single-turn

The GGUF companion uses a Llama-compatible decoder, a 1,024-piece SentencePiece tokenizer with byte fallback, 6 layers, hidden size 288, intermediate size 608, 9 attention/KV heads, 128-token training sequences, and a 512-token runtime context. It was trained separately for five epochs (3,325 optimizer steps) on 85,005 project-data records; best validation loss was 0.0390303. Exact metadata is in lmstudio/training_manifest.json.

Original EXLLM reference runtime

git clone https://huggingface.co/ToTo-40417/EXLLM
cd EXLLM
python -m venv .venv
. .venv/bin/activate
pip install -r requirements.txt
python chat.py こんにちは

from src.loader import load_release_model
from src.runtime import answer

model, tokenizer = load_release_model()
print(answer(model, tokenizer, "RAMとは何ですか？", temperature=0.0))

Generation Examples

Release Artifacts

EXLLM8 and EXQ12 have the same file size but are not interchangeable formats. Rebuild and verify model.q12 with python tools/export_exq12.py; see EXQ12_FORMAT.md.

Intended Use

• Ultra-small language-model research
• Embedded and low-memory inference experiments
• Short Japanese UI responses and narrow-domain instruction handling
• Integer-runtime and quantization experiments
• Educational analysis of language models below 0.01B parameters

Limitations

• The model is not a general-purpose assistant.
• The original embedded checkpoint and the GGUF training sequences use 0.000128M tokens. The GGUF advertises a 0.000512M-token runtime window for LM Studio framing and short history; quality beyond the trained 128-token range is not guaranteed.
• Knowledge coverage is restricted to the project-generated training scope.
• Unseen concepts, long instructions, translation, and multi-step reasoning are unreliable.
• No RLHF, DPO, tool use, retrieval, web access, or current-information source is included.
• The deterministic calculator is runtime routing, not learned arithmetic capability.
• The model must not be used as an authority for medical, legal, financial, safety-critical, or other high-stakes decisions.
• Standard Hugging Face Inference Providers cannot serve the original embedded checkpoint without custom architecture support. The bundled GGUF companion is intended for local LM Studio / llama.cpp execution.

Links

• Source and training code: ToTo-40417/exllm
• EX-word runtime: ToTo-40417/exllm-exword
• EXQ12 validator: ToTo-40417/exllm-model-check
• Hardware probe: ToTo-40417/exword-hardware-dump
• RAM qualification tool: ToTo-40417/exword-ram-scanner
• Gnuboy save-transfer utility: ToTo-40417/exword-gnuboy-save-importer
• Project article (Japanese): 高校生用の電子辞書でLLMを動かしてみた ―0.024GHz/0.016GB

日本語

EXLLM-0.005B-Instructは、低メモリ組込み機器でのローカル推論を目的とした、0.005377824Bパラメータの日本語instruction modelです。CASIO EX-word XD-B4800（DATAPLUS 6）とXD-N6500（DATAPLUS 7）で実機推論を確認しています。同じプロジェクトデータから別途学習した、LM Studio / llama.cpp向けの0.005441184BパラメータGGUF companionも収録しています。

LLMとは、Little Language Modelの略です。

電子辞書用checkpointは独自のdecoder-only Transformerとtokenizerを使用するため、GGUFに直接変換したものではありません。LM StudioではToTo-40417/EXLLMを検索し、EXLLM-0.005B-LMStudio-F16.ggufを選択してください。チャット用templateはGGUFに内蔵済みで、LM Studioのロード時contextは512 tokensに設定します。学習時の系列長は128 tokensであり、追加領域はチャット制御情報と短い履歴のための余白です。このGGUFはプロジェクトデータで別途学習したPC用companionであり、電子辞書版と同一重みではありません。

RTX 3060でのfp32参照実行は、40回のwarm-up後に120回測定し、TTFT中央値0.003990秒、生成速度239.082 token/sでした。EX-word整数runtimeでは、0.024184 GHzのSH-4A-class CPU上でTTFT中央値23.711秒、生成速度0.52–0.55 token/sでした。

用途は、0.01B未満の言語モデル、組込み推論、量子化、整数runtimeの研究です。汎用的な質問応答、長文処理、専門判断、最新情報の取得を目的としたモデルではありません。

コード、重み、付属データはApache License 2.0で公開します。

Citation

@software{toto_exllm_2026,
  author  = {ToTo},
  title   = {EXLLM-0.005B-Instruct},
  year    = {2026},
  url     = {https://huggingface.co/ToTo-40417/EXLLM}
}