nemotron-3-nano-4b-arkey
One GGUF file, model.gguf (Q4_K_M, 2,837,076,704 bytes). It is
NVIDIA Nemotron 3 Nano 4B with a rank-32 LoRA
merged into its weights. The LoRA was trained with RLOO on the tool turns of one app,
GameTerm. Load it as an ordinary model. No --lora is needed.
llama-server -m model.gguf --jinja -ngl 99
sha256 764393f95e43a4a7f443f394d18bf99951fe9e6247bc99511ba80da1ef31c0ad
It is a recipe demonstration, not a general tool-use upgrade. See the results and limitations below.
The unmerged adapter (adapter.gguf, PEFT safetensors, raw factors) and its full card stay in this repo's history at
revision da8ed194.
Code, recipe and records: DayCare-Public.
How it was made
All steps used llama.cpp build b9592 (commit ac4cddeb0dbd778f650bf568f6f08344a06abe3a), CPU.
convert_hf_to_gguf.py NVIDIA-Nemotron-3-Nano-4B-BF16 --outtype bf16(base repo revisiondfaf35de). sha2563594677ead04a9fb6cab5e0661c36d6aedce6cca02b6e3fddbffecdba30b6cbe. This differs from the BF16 GGUF the adapter was tested on (2126a5e8…). The cause was not measured.llama-export-lora -m base-bf16.gguf --lora adapter.gguf -o merged-bf16.gguf. Two tensors changed:blk.41.ffn_up.weightandblk.41.ffn_down.weight, asW + (alpha/r) * B @ Awith alpha 64, rank 32. export-lora writes the merged tensors as F16.llama-quantize merged-bf16.gguf model.gguf Q4_K_M. The merged tensors became q5_0 (ffn_up, fallback because 3136 columns are not divisible by 256) and q6_K (ffn_down).
Parity checks
Five chat prompts with the model's own chat template, thinking on (template default). Four carry tool definitions. Greedy (temperature 0, top-k 1), 64 tokens, seed 42, llama-server on CPU, token ids compared.
A. The merge is exact. base-bf16.gguf --lora adapter.gguf vs merged-bf16.gguf: identical token ids on 5 of 5
prompts (38/38, 46/46, 64/64, 46/46, 64/64; the shorter ones stop at end of text). A control without the LoRA
differs on 3 of the 5 prompts, so the check is not vacuous.
B. What Q4_K_M costs. Leading identical tokens, out of 64:
| Prompt | this file vs merged BF16 | NVIDIA Q4_K_M + --lora vs merged BF16 | this file vs NVIDIA Q4_K_M + --lora |
|---|---|---|---|
| open Safari (tool) | 4 | 1 | 1 |
| 37 times 49 (tool) | 0 | 0 | 61/61 identical |
| play lo-fi on Spotify (tool) | 26 | 9 | 9 |
| web search, Taipei weather (tool) | 0 | 0 | 64/64 identical |
| blue and yellow (no tool) | 13 | 13 | 44 |
Quantizing changes the wording on every prompt. Every tool prompt still gives a well-formed call to the right tool with the right argument. On the multiplication prompt both Q4_K_M variants give exactly the base model's (no LoRA) BF16 output, so quantization removes the adapter's effect there. Applying the LoRA at runtime on NVIDIA's own Q4_K_M behaves the same, so this is the cost of quantizing the base, not of the merge.
GameTerm first-tool quiz
GameTerm's retired calibration quiz (revision 1, all tools exposed): 24 questions in 8 groups. Each question is sent once in GameTerm's real first-turn request (its system prompt and 35 tools, thinking off), temperature 0, seed 42, 512 tokens. Score: the first tool call is one of the group's right first steps; for the no-tool group, no tool call. Nothing is executed. llama.cpp build 10948 on an Apple M-series Mac (Metal).
| Group | NVIDIA Q4_K_M | this file |
|---|---|---|
| Calculator (3) | 0 | 1 |
| Terminal (3) | 2 | 2 |
| Files (3) | 1 | 1 |
| Apps (3) | 0 | 1 |
| Music (3) | 1 | 1 |
| Windows (3) | 1 | 1 |
| Web search (3) | 0 | 3 |
| No tool (3) | 3 | 3 |
| Total (24) | 8 | 13 |
| Total without web search (21) | 8 | 10 |
This file gains 5 questions over the base and loses none; no-tool answers are unchanged (3 of 3). The pass rule was written two ways before the results: "beat the base by at least 3 of 21" (the 21 questions without web search: +2, does not pass) and "at least 11 of 24" (13, passes). Both readings are reported. Most of the gain is web search (0 -> 3). One run per model, greedy. The base scored 5 of 21 in GameTerm's own quiz on 2026-09-20 at GameTerm's sampling, and 8 of 21 here; the cause of that difference was not measured.
The request text (system prompt and tool definitions) is GameTerm's own and is not published.
Intended use
- Intended: a small local assistant inside GameTerm, and a worked example for app builders who want to train a small model on their own app's tool turns.
- Out of scope: a general improvement in tool selection or function calling; safety-critical use; apps whose prompts and tools differ from GameTerm's without your own evaluation.
Results of the adapter (from the adapter card at da8ed194)
These were measured with the unmerged adapter on the BF16 base, thinking on, not on this Q4_K_M file.
- GameTerm held-out exam: repair after a rejected calculator call 76.3% -> 91.8% (+15.5 [+11.2, +20.0]); blocked call handled correctly 27.1% -> 39.4% (+12.3 [+7.7, +16.9]); blank answers in the relay family 97 -> 16.
- Public benchmarks (BFCL v4, When2Call): no measurable change in either direction.
- One predeclared retention gate failed (numeric retention net -1 item at one greedy sample). Adopted by owner exception. One training seed.
Full tables, methods and confidence intervals are in the adapter card at da8ed194 and in DayCare-Public.
Limitations
- App-specific. Trained only on GameTerm's post-tool turns.
- Quantized. Q4_K_M changes outputs relative to the merged BF16 model (parity B above). The adapter's exam results were not re-measured on this file.
- Trained with thinking on; GameTerm runs this model with thinking off.
- One training seed.
- Inherits every limitation, bias and safety consideration of the base model (see its card).
License
The base model is governed by the
NVIDIA Nemotron Open Model License
(last modified December 15, 2025). This model is a Derivative Work of it and is distributed under the same license.
A copy of the license is included in this repository (LICENSE), as section 3(a) requires.
Licensed by NVIDIA Corporation under the NVIDIA Nemotron Model License.
The DayCare training code is MIT-licensed. That license covers the code only, not these weights.