Nex-N2.5-mini APEX GGUF

APEX (Adaptive Precision for EXpert models) quantizations of nex-agi/Nex-N2.5-mini, covering every deployment scenario from maximum quality down to ~10 GB.

• Quantized by: MrFuzzihead
• Method: APEX — a MoE-aware mixed-precision quantization technique for llama.cpp, by the LocalAI team
• More APEX quants: MrFuzzihead's APEX collection

APEX assigns quantization precision per tensor role (routed expert / shared expert / attention / SSM) and per layer position (edge layers high, middle layers compressed), exploiting the sparsity of Mixture-of-Experts models. On the APEX reference model (Qwen3.5-35B-A3B), APEX tiers match or beat Q8_0 quality at ~60% of the size — see the APEX technical report for details.

Note on the hardware-compatibility widget: Hugging Face detects quantization variants from standard quant names (Q4_K_M etc.) in the filename. APEX tiers are mixed-precision and use APEX names, so the widget cannot classify them (this also affects other APEX repos). Use the table below for sizes; mmproj files are vision projectors and never appear in the widget by design.

Files

Sizes are in GB (2³⁰ bytes), as displayed by the Hub. Full-GPU offload needs the file size plus the mmproj (if you use vision) plus KV cache and overhead.

I- files are quantized with an importance matrix built from diverse calibration data (chat, code, reasoning, tool-calling, agentic traces — no Wikipedia). In APEX evaluations, I-variants trade a negligible perplexity increase for better downstream accuracy and lower KL divergence. The Mini, Micro and Nano tiers use the diverse imatrix by design (per the APEX method), which is why they ship as I--named files (I-Micro, I-Nano) with no separate non-imatrix version.

The model

Nex-N2.5-mini is Nex-AGI's agentic multimodal model, the smallest member of the Nex-N2.5 family (mini / Pro / Max). It is built for long-horizon tasks in real-world environments — computer use, web browsing, and visually grounded agentic workflows — where vision is the interface through which the agent perceives its environment, verifies outcomes, and self-corrects.

• ~35 B parameters, Mixture-of-Experts: 256 routed experts, 8 active per token, plus a shared expert.
• 40 layers, hybrid attention: three linear-attention (SSM) layers interleaved per full-attention layer (qwen3_5_moe architecture, same family as Qwen3.5). Only 10 of 40 layers use full attention, so KV-cache overhead is minimal.
• Multimodal: vision tower with video support (temporal patches); 262,144-token context.
• Reasoning: adaptive thinking via reasoning_effort (none / medium / high), emitted in thinking… response blocks.
• Tool calling: Qwen3-Coder style <tool_call> / <function=…> XML calls.
• License: Apache-2.0.

How APEX quantizes Nex-N2.5-mini

Routed experts are ~94% of the weights here, but only 8 of 256 fire for any given token, so they tolerate lower precision than the parts every token passes through. Attention + SSM are only ~2.7% of the weights but dense, and the shared expert (~0.4%) is always active and heavy-tailed. APEX classifies each tensor by role and applies a layer-wise precision gradient — edge layers keep higher precision, middle layers compress harder:

Embeddings and the LM head (~3% of weights) are kept at Q6_K (Q3_K on Mini) to preserve output-logit fidelity. Routers and norms stay F32. Because routing decisions use full-precision gates, quantization noise in inactive experts never affects the output.

For the tier configs and reproduction scripts, see the APEX repository.

Usage

Needs a recent llama.cpp build with qwen35moe (Qwen3.5 / Nex-N2.5) support. The low tiers additionally use IQ1_M / IQ2_XS / IQ2_S / IQ2_XXS expert formats, and the BF16 mmproj needs current BF16 support — when in doubt, use the latest release.

# text
llama-cli -m Nex-N2.5-mini-APEX-I-Quality.gguf -p "Your prompt" -ngl 99

# OpenAI-compatible server (tool calls + reasoning via --jinja)
llama-server -m Nex-N2.5-mini-APEX-I-Quality.gguf \
  --mmproj mmproj-Nex-N2.5-mini-BF16.gguf \
  --jinja --host 0.0.0.0 --port 8080 -ngl 99

# vision, standalone
llama-mtmd-cli -m Nex-N2.5-mini-APEX-I-Quality.gguf \
  --mmproj mmproj-Nex-N2.5-mini-Q8_0.gguf \
  --image image.png -p "What is in this image?" -ngl 99

The GGUF embeds the official chat template, including reasoning_effort thinking modes and the <tool_call> function-calling format. With llama-server --jinja, you can force a thinking mode with --chat-template-kwargs '{"reasoning_effort":"high"}' (omit for the default adaptive medium).

Recommended sampling parameters (from the Nex-AGI model card):

• temperature: 0.7
• top_p: 0.95
• top_k: 40

Choosing an mmproj

mmproj-Nex-N2.5-mini-BF16.gguf (0.84 GB) is the full-precision projector; mmproj-Nex-N2.5-mini-Q8_0.gguf (0.57 GB) is the quantized one. Either pairs with any APEX tier above. If you are not using images or video, skip the mmproj entirely.

Notes

• No MTP head: the base repo's config declares an MTP layer (mtp_num_hidden_layers: 1), but no MTP tensors are shipped in nex-agi/Nex-N2.5-mini, so these files do not bundle a speculative draft head — --spec-type draft-mtp does not apply.
• Benchmarks: no perplexity or accuracy benchmarks were run on these specific files. The published APEX numbers are from the reference Qwen3.5-35B-A3B evaluation in the APEX repository; see the base model card for Nex-N2.5-mini's own evaluation results.
• License: Apache-2.0, same as the base model.

Credits

• Base model: nex-agi/Nex-N2.5-mini by Nex-AGI (Apache-2.0)
• Quantization method: APEX by the LocalAI team — built on llama.cpp
• Quants: MrFuzzihead