Tiel-Coder-35B-A3B-Genesis-Hermes-NVFP4-GGUF

This repository contains the NVFP4 (NVIDIA 4-bit Floating Point) quantized GGUF release of LuffyTheFox's Tiel-Coder 35B A3B Genesis Hermes model.

• Original Creator & Fine-Tuner: LuffyTheFox
• Base Model: peculiar-ragdoll/Tiel-Coder-35B-A3B-GGUF
• Hermes Finetune Source: DJLougen/hermes-qwen3.5-35b-a3b-GGUF
• Quantization Format: NVFP4 (NVIDIA FP4) GGUF
• Architecture: Mixture of Experts (MoE) — 35B total parameters / ~3B active per token (8 active experts)
• Capabilities: Agentic coding, Hermes tool-calling, Thinking mode (DeepSeek-style reasoning), Vision (Multimodal)

⚡ What is Genesis?

During training, ALL models don't just learn knowledge — they also accumulate random noise in their tensors. This noise builds up and creates something I call the Noise Gate — a fundamental barrier that stops LLM models from learning further and makes them unstable, verbose, and prone to hallucinations. My approach reduces this noise. It repairs the signal in tensors without touching the learned knowledge and gradient using Marchenko–Pastur distribution as a core criteria. The result is a model that is consistent in performance, context clarity and following instructions, because it's no longer fighting its own internal chaos.

Genesis is a post-training data regeneration and calibration algorithm for neural networks (LLM) in GGUF format, developed over almost half a year with AI assistance. It is optimized, architecture-independent, works with any model in GGUF format, and is based on mathematical statistics. No retraining or finetuning — pure numerical surgery on the raw bytes of the file.

Three stages:

1. Scan ssm_conv1d tensors (long-context memory) and repair balance between heads.
2. Scan blocks via chunks (3 parameters) and pick the best fit to the weight distribution; replace zero chunks in broken tensors without touching learned structure.
3. Scan for noise via custom SVD (excluding token_embd.weight, output.weight, 1D tensors, bias, and norms). Reduce training noise via custom SVD based on Marchenko–Pastur law, preserving 99% of signal and learned gradient.

For this model, Hermes finetune data (around 2k blocks from two FFN expert tensors) was transferred from the DJLougen Hermes finetune onto the peculiar-ragdoll Tiel-Coder base, then processed via Genesis.

⚡ Sampling Recommendations (Genesis SVD Stabilization)

According to tests after Marchenko–Pastur SVD noise-gate stabilization, the model is most stable when Top P and Min P sampling are disabled for coding/precise tasks (the tensor noise gate has been removed). Only Temperature and Top K sampling should be used for those profiles.

Recommended Parameters:

Note on the General preset: top_p=0.0 + min_p=1.0 behaves near-greedy in llama.cpp (min_p=1.0 filters to only tokens with relative probability ≥ 1.0, i.e. effectively the argmax). This is intentional for deterministic, high-stability thinking output.

🛠️ Tool Calling & Hermes Agent Compatibility

If running with the qwen3.6-froggeric chat template through Hermes Agent, the template defaults to XML tool call output (<function=...><parameter=...>), whereas Hermes Agent expects standard OpenAI JSON tool calls.

To ensure seamless JSON tool calling, pass tool_call_format: "json" in --chat-template-kwargs:

{"enable_thinking": true, "tool_call_format": "json"}

(Or set line 2 of chat_template.jinja default from xml to json.)

💡 Recommended System Prompts

1. Default / First Experience (High Stability)

You are Qwen (Tongyi Qianwen), a large language model developed by Alibaba Group's Tongyi Lab.

2. Agentic Tasks (JSON Schema Output)

You are Qwen (Tongyi Qianwen), a large language model developed by Alibaba Group's Tongyi Lab. You are a helpful assistant that answers in JSON. Here's the json schema you must adhere to:
<schema>
{schema}
</schema>

🚀 Usage Examples (llama-server / llama.cpp)

1. Hermes Agent (Thinking ON, 128K Context)

llama-server \
  --host 0.0.0.0 --port 8080 \
  --model Tiel-Coder-35B-A3B-Genesis-Hermes-NVFP4-v4.gguf \
  --override-kv qwen2moe.expert_used_count=int:8 \
  --n-gpu-layers all \
  --flash-attn on \
  --cache-type-k f16 \
  --cache-type-v f16 \
  --ctx-size 131072 \
  --parallel 1 \
  --batch-size 2048 \
  --ubatch-size 512 \
  --temp 0.6 \
  --top-p 1.0 \
  --top-k 20 \
  --min-p 0.0 \
  --seed 42 \
  --presence-penalty 0 \
  --repeat-penalty 1.05 \
  --jinja \
  --chat-template-file chat_template.jinja \
  --reasoning on \
  --reasoning-effort high \
  --reasoning-preserve \
  --reasoning-format deepseek \
  --chat-template-kwargs '{"enable_thinking":true, "tool_call_format":"json"}'

2. Coding / Precise (Thinking ON, repeat_penalty disabled)

Same as above but with --repeat-penalty 1.0.

3. Creative (Thinking OFF)

llama-server \
  --host 0.0.0.0 --port 8080 \
  --model Tiel-Coder-35B-A3B-Genesis-Hermes-NVFP4-v4.gguf \
  --override-kv qwen2moe.expert_used_count=int:8 \
  --n-gpu-layers all \
  --flash-attn on \
  --cache-type-k f16 \
  --cache-type-v f16 \
  --ctx-size 131072 \
  --parallel 1 \
  --batch-size 2048 \
  --ubatch-size 512 \
  --temp 0.7 \
  --top-p 0.8 \
  --top-k 20 \
  --min-p 0.0 \
  --seed 42 \
  --presence-penalty 0 \
  --repeat-penalty 1.0 \
  --jinja \
  --chat-template-file chat_template.jinja \
  --reasoning off \
  --chat-template-kwargs '{"enable_thinking":false, "tool_call_format":"json"}'

4. Relaxed (Thinking OFF, looser samplers for chat/brainstorm)

llama-server \
  --host 0.0.0.0 --port 8080 \
  --model Tiel-Coder-35B-A3B-Genesis-Hermes-NVFP4-v4.gguf \
  --override-kv qwen2moe.expert_used_count=int:8 \
  --n-gpu-layers all \
  --flash-attn on \
  --cache-type-k f16 \
  --cache-type-v f16 \
  --ctx-size 131072 \
  --parallel 1 \
  --batch-size 2048 \
  --ubatch-size 512 \
  --temp 1.0 \
  --top-p 0.95 \
  --top-k 40 \
  --min-p 0.05 \
  --seed 42 \
  --presence-penalty 0 \
  --repeat-penalty 1.0 \
  --jinja \
  --chat-template-file chat_template.jinja \
  --reasoning off \
  --chat-template-kwargs '{"enable_thinking":false, "tool_call_format":"json"}'

5. Multimodal (Vision) Support

Simply add the mmproj projector parameter to any of the above profiles:

  --mmproj mmproj-Tiel-Coder-35B-A3B-Genesis-Hermes-BF16.gguf

6. Speculative Decoding (MTP Version)

For the transplanted MTP version Tiel-Coder-35B-A3B-Genesis-Hermes-NVFP4-v4-MTP.gguf, add speculative decoding flags:

  --model Tiel-Coder-35B-A3B-Genesis-Hermes-NVFP4-v4-MTP.gguf \
  --spec-type draft-mtp \
  --spec-draft-n-max 2 \
  --spec-draft-p-min 0.75

📦 Repository Files

v4 — Recommended (inline scales, LM Studio compatible)

v4 uses --nvfp4-inline-scales-only to emit native inline UE4M3 scales without separate .scale/.input_scale tensors. Required for LM Studio / Pelican and other runtimes that do not support the extended NVFP4 scale tensor contract.

Deprecated (separate scales, broken in LM Studio)

Specs

• 35B total parameters, ~3B active per forward pass (MoE)
• 256 experts, 8 routed + 1 shared per token
• Hybrid architecture: Gated DeltaNet linear attention + full softmax attention (3:1 ratio)
• 40 layers, pattern: 10 × (3 × DeltaNet-MoE + 1 × Attention-MoE)
• 262K native context (extendable to 1M with YaRN) — keep at least 128K to preserve thinking capabilities
• Natively multimodal (text, image, video)
• 248K vocabulary, 201 languages

Compatibility

Works with llama.cpp, LM Studio, koboldcpp, and other GGUF-compatible runtimes.

Important:

• Keep at least 128K context to preserve thinking capabilities.
• Use --jinja flag with llama.cpp for proper chat template handling.
• Vision support requires the mmproj file alongside the main GGUF.
• Set qwen2moe.expert_used_count=int:8 via --override-kv for correct expert routing.
• Set K cache and V cache quantization to f16 for best performance on APEX/NVFP4 quants.

Credits & Attribution

• LuffyTheFox — Genesis post-training calibration, Hermes finetune transfer, and NVFP4 quantization.
• peculiar-ragdoll — original Tiel-Coder-35B-A3B-GGUF base.
• DJLougen — hermes-qwen3.5-35b-a3b-GGUF finetune source.
• HauhauCS — original Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive upstream.
• Chat template by froggeric & qweefchief.

Contact & Support

⚡ If you like this Genesis LLM release you can donate via @Tribute bot in Telegram to support future Genesis LLM development.

• Contact: luffythefox@mail.ru, azakharchenko92@gmail.com
• Telegram: @LuffyTheFox
• Join the Discord for updates, roadmaps, projects, or just to chat.

Links:

• Original Tiel-Coder base
• Hermes finetune source
• Quantization Script with Unsloth profiles support

good luck on your journey, YMMV!