Javi0.3-4B-GGUF

A 4B agent model trained to think from first principles and use its tools effectively to find the answer to your prompt, or, when the tools can't get there, to guide you to it. It's trained to never guess: if it can't verify something or doesn't know, it says so. When a tool fails, it works out why and takes another route instead of looping or giving up. Fine-tuned from Qwen3.5-4B to work inside coding and research agents (grok build, omp, opencode) with their own system prompts and tools.

What changed since Javi0.2, all measured in Results:

• Short, grounded answers. 325 characters on average (Javi0.2: 2,193; base Qwen3.5-4B: 632). Fewer claims, fewer wrong ones.
• Almost no made-up links. 1.1% of answers cite a link no tool returned, counting bare ones without https:// (Javi0.2: 53%; base Qwen3.5-4B: 1.7%).
• Works around failures. When a search is blocked or a page returns 403, it says what failed and the likely cause, then tries another route. On failure types it never saw in training (429 rate limits, timeouts) it recovers 88% of the time, level with base Qwen3.5-4B (89%).
• Honest when everything is broken. It says what it tried, gives its best answer clearly labeled as unverified, and names where to check, without inventing a URL.
• No loops. It repeats an identical call in 3% of tasks (base: 11%).

Smaller sibling: Javi0.3-2B-GGUF. Full-precision weights: Javi0.3-4B.

How it was built, every mistake included: the Javi0.3 article on X.

Files

Both were quantized from an F16 GGUF of the merged model; the LoRA was merged into the bf16 base, not into a 4-bit copy. Text only: the base model's vision tower is not in the GGUF.

Run it

llama.cpp

llama-server -hf jvr0x/Javi0.3-4B-GGUF:Q8_0 --jinja -c 131072 -fa on -ctk q8_0 -ctv q8_0 \
  --chat-template-kwargs '{"enable_thinking": false}' \
  --temp 0.2 --top-p 0.8 --top-k 20 --min-p 0

Then point any OpenAI-compatible agent at http://localhost:8080/v1. Javi is built for agents with tools (web search, page fetch, a shell). In a plain chat with no tools it answers from what it knows and tells you when a question needs a lookup.

• Thinking off. It was trained and evaluated with enable_thinking: false.
• Low temperature (0.2) keeps tool calls well formed.

Mount it with lmswitch

hf download jvr0x/Javi0.3-4B-GGUF Javi0.3-4B.Q8_0.gguf --local-dir ~/models/jvr0x/Javi0.3-4B-GGUF

Save this as ai-models/javi0.3-4b.yaml (the model path is relative to your lmswitch models directory, ~/models by default):

runtime: llama
model: "jvr0x/Javi0.3-4B-GGUF/Javi0.3-4B.Q8_0.gguf"
port: 8155
ctx: 262144
display_name: "Javi0.3-4B Q8_0"
extra_args: ["-fa", "on", "-ctk", "q8_0", "-ctv", "q8_0", "--jinja", "--chat-template-kwargs", "{\"enable_thinking\": false}", "--temp", "0.2", "--top-p", "0.8", "--top-k", "20", "--min-p", "0"]

lmswitch on javi0.3-4b

This is the exact setup the eval below ran.

Results

Eval v3: 175 held-out questions, each run 3 times inside real client formats (grok, omp, opencode, a bare OpenAI format and a plain chat) with tools: web search (frozen cache and pinned date, so runs compare), page fetches, and a sandboxed shell over small code projects. Some questions force tool failures. Two judges grade every answer, DeepSeek-V4.1-Flash and MiMo-V2.6-Flash. Core pass means grounded (2/2), honest (2/2), the task at least partly solved, and no https:// link that no tool returned (checked from the transcript, not by the judge).

Against the base model, Javi0.3-4B is ahead by 0.05, but the 95% interval is -0.01 to +0.11: ahead, not proven. The judges split: DeepSeek has Javi ahead by 0.14 (+0.07 to +0.20), MiMo has the base 0.03 ahead, within noise. Against Javi0.2 it gains 0.33.

Behavior, counted from all 525 transcripts:

Held-out failures: 24 more questions whose failures are types it never trained on (429 rate limits, timeouts), 3 runs each.

By category (core pass, Javi0.3-4B vs base): ahead where Javi's habits matter (every route broken 0.52 vs 0.11, lookups 0.43 vs 0.28, undated questions 0.47 vs 0.19, conflicting sources 0.40 vs 0.26, false premises 0.27 vs 0.15); behind on code (0.46 vs 0.67), reasoning puzzles (0.82 vs 0.96) and recovery questions (0.41 vs 0.52).

Hardware

• Memory = weights plus an F16 KV cache. Only 8 of the 32 layers use full attention (the rest are linear attention with a fixed-size state), so the cache is 32 KiB per token; -ctk q8_0 -ctv q8_0 halves it.
• Context: the base model supports 262k tokens. Javi0.3 trained on conversations up to 72k tokens.
• Speed is llama-bench decode (256 tokens, 3 runs) on an NVIDIA DGX Spark (GB10, 128 GB unified memory), full GPU offload, flash attention on. Prefill runs at about 3,800 tok/s (Q8_0) to 4,100 tok/s (Q4_K_M).

How it was made

Teacher inside the harness. GLM-5.3-Flash answered seed questions inside each client's exact format (system prompts of 15-25k tokens, 10-27 tools) with real tools: web search through a local SearXNG, page fetches, and a Docker sandbox. Its working rules were in the teacher's prompt only: reason from first principles, look up anything time-sensitive, diagnose a failed call before the next one, prefer a new search or a link from earlier results over a URL typed from memory, and when everything fails still give a labeled best answer and name where to check. Training rows keep the client's own prompt, so the student learns the behavior, not the instructions.

Data. Javi0.2's rows with made-up links removed, plus three rounds of recovery practice on 480 questions, each round giving every question a different failure and client: the first search or page fetch fails once (blocked, empty, 403, 404) or, from the second round on, keeps failing for a third of them (offline, a route blocked for good).

Filters. DeepSeek-V4.1-Flash judged every episode. Code checks dropped anything unfinished, any time-sensitive answer without a tool, repeated calls, links no tool returned (with or without https://), and any call after a failure that didn't say what failed. A recovery stayed only if a different route worked, or if two routes failed and the answer stopped honestly. Recoveries where a URL typed from memory just got lucky were dropped.

Two rewrite passes by the teacher. First concise: every answer over 500 characters rewritten answer-first, keeping only facts a tool returned, with sources named instead of linked (median answer 703 → 364 characters). Then de-hedge: disclaimers removed from math, code and settled facts (hedged answers 46% → 17%, median answer 289 characters). A rewrite was kept only if it was shorter, had no URL, changed no code block and added no number.

Training: LoRA SFT with Unsloth on 1,253 rows.

Export: the adapter was merged into the bf16 base, converted to an F16 GGUF, then quantized with llama-quantize.

The story of every round, including each way it broke along the way, is in the Javi0.3 article on X.

Limitations

• Recovery on the main eval is 58% (base: 92%). After a failure it often fetches a page directly instead of searching again, and about half of those fetches fail.
• Its lead over base Qwen3.5-4B is within noise, and it is weaker on code and reasoning puzzles.
• Hedging is not perfect. 27% of answers carry an "unverified" hedge; on stable questions, where none is needed, 5% still do.
• It's a 4B model. It can still get facts wrong.
• It's built for agent clients with tools. Without tools it can't look anything up.
• The judges are LLMs, not people, and they disagree on some answers (Cohen's kappa 0.56).
• Trained mostly on English.

What's next

Javi0.4: recovery data built search-first from the start, and a bigger eval that can resolve small gaps.

License

Apache 2.0, inherited from Qwen3.5-4B.