⚡ TripleTrouble-V3 ⚡

The Triad of Code, Tool Agency, and World Simulation

Model Details • The Triad • Merge Engineering • Serving with vLLM • Transformers

🌌 Overview

TripleTrouble-V3 is a 35B-class sparse Mixture-of-Experts (MoE) foundation model created by fusing three divergent, specialized post-trained models of the Qwen 35B-A3B architecture:

1. The Coder (KAT-Coder-V2.5-Dev): Autonomous codebase manipulation, SWE-bench refactoring, repository traversal, and concrete AST syntax trees.
2. The Agent (Ornith-1.5-35B-A3B): Multi-turn API calling, complex structured JSON emissions, mathematical deduction, and competitive programmatic reasoning.
3. The Simulator (Qwen-AgentWorld-35B-A3B): Large World Model (LWM) dynamics, environment state transition modeling (MCP, OS, Web, Android), and next-state observation forecasting.

By moving beyond naive weight averaging and avoiding the rank collapse of pseudo-base TIES pruning, TripleTrouble-V3 preserves the singular value spectrum of all 256 experts and maintains the spectral radius of Qwen's hybrid Gated DeltaNet linear recurrence dynamics.

🧬 The Triad

flowchart TD
    subgraph MERGED ["⚡ TripleTrouble-V3"]
        ROOT["<b>TripleTrouble-V3</b><br/>34.7B MoE • ~3.3B Active per token"]
    end

    ROOT -->|"40% Weight"| KAT["💻 <b>KAT-Coder-V2.5-Dev</b><br/>• Code & SWE-bench Refactoring<br/>• AST & Syntax Trees<br/>• Terminal Command Logic"]
    ROOT -->|"35% Weight"| ORN["🦅 <b>Ornith-1.5-35B-A3B</b><br/>• Multi-Turn Tool Calling<br/>• MCP Protocol Execution<br/>• Structured JSON & Math"]
    ROOT -->|"25% Weight"| AGW["🌐 <b>Qwen-AgentWorld-35B</b><br/>• Large World Model (LWM)<br/>• Environment Dynamics<br/>• State Observation Loops"]

    classDef default fill:#1e293b,stroke:#475569,stroke-width:1px,color:#f8fafc;
    classDef highlight fill:#4338ca,stroke:#818cf8,stroke-width:2px,color:#ffffff;
    class ROOT highlight;

🧮 Merge Mathematics

Traditional model mergers suffer from two major failure modes on 35B hybrid MoE models:

• Naive Linear Soups: Cause high-dimensional variance collapse ($\mathbb{E}[|\sum w_i \theta_i|] \ll \mathbb{E}[|\theta_i|]$), dampening activations across 40 layers.
• Pseudo-Base TIES/DARE: Truncate 70% of delta coordinates, destroying the singular value spectrum of small ($2048 \times 1024$) MoE experts and shattering the Householder contraction operator of Gated DeltaNet attention.

TripleTrouble-V3 was built using a continuous hyperspherical framework designed specifically for this architecture:

1. Decoupled Normalized Geodesic Consensus (NGC)

Directional consensus and magnitude scaling are strictly decoupled. Checkpoints with larger gradient norms cannot skew the consensus angle:

$$\hat{W}_i = \frac{W_i}{|W_i|_F}$$

$$D_{\text{consensus}} = \sum_{i=1}^3 w_i(l) \hat{W}i, \quad \hat{D} = \frac{D{\text{consensus}}}{|D_{\text{consensus}}|_F}$$

$$W_{\text{final}} = \hat{D} \times \left( \sum_{i=1}^3 w_i(l) |W_i|_F \right)$$

2. Row-Wise Router Manifold Calibration

In an MoE with 256 routed experts, the router gate $W_{\text{gate}} \in \mathbb{R}^{256 \times d_{\text{model}}}$ consists of 256 distinct hyperplanes $g_e$. Global matrix scaling alters individual expert activation probabilities. We calibrate every expert hyperplane row-by-row:

$$g_{e, \text{final}} = \frac{\sum_i w_i \frac{g_{e, i}}{|g_{e, i}|2}}{\left| \sum_i w_i \frac{g{e, i}}{|g_{e, i}|2} \right|2} \times \left( \sum{i=1}^3 w_i |g{e, i}|_2 \right)$$

This guarantees router logit sharpness, prevents entropy collapse, and keeps expert dispatch distributions intact.

3. Functional Depth-Aware Dynamic Scheduling

Layer weights dynamically modulate across the 40 layers to match functional network mechanics:

⚙️ Architectural Specifications

🚀 Fast Inference with vLLM

Thanks to its sparse MoE architecture, TripleTrouble-V3 runs at the inference speed of a ~3.3B dense model while delivering 35B-scale reasoning.

Installation

pip install vllm>=0.6.0

Launch an OpenAI-Compatible API Server

vllm serve OliviaRossi/TripleTrouble-V3 \
  --tensor-parallel-size 2 \
  --max-model-len 32768 \
  --gpu-memory-utilization 0.90 \
  --trust-remote-code

(For a single 80GB GPU, run using FP8 or AWQ quantization: --quantization fp8).

💻 Quickstart: Transformers

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "OliviaRossi/TripleTrouble-V3"

tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
    trust_remote_code=True
)

messages = [
    {
        "role": "system",
        "content": (
            "You are TripleTrouble, an expert agent combining deep codebase mastery, "
            "rigorous multi-turn tool planning, and environment simulation capabilities."
        )
    },
    {
        "role": "user",
        "content": "Analyze the concurrency bottlenecks in an asynchronous Python producer-consumer queue and write a resilient implementation using asyncio.Queue."
    }
]

prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

outputs = model.generate(
    **inputs,
    max_new_tokens=2048,
    temperature=0.6,
    top_p=0.9,
    repetition_penalty=1.05
)

response = tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True)
print(response)

🛠️ Recommended Sampling Parameters

⚖️ License & Attribution

This model is released under the Apache 2.0 License.

Source Models:

• KAT-Coder-V2.5-Dev by Kwaipilot
• Ornith-1.5-35B-A3B by ornith-ai
• Qwen-AgentWorld-35B-A3B by Qwen / Alibaba Cloud

@misc{tripletrouble2026,
  author = {Olivia Rossi},
  title = {TripleTrouble-V3: A Geodesic Manifold Merge of Code, Tool Agency, and World Simulation},
  year = {2026},
  publisher = {Hugging Face},
  howpublished = {\url{https://huggingface.co/OliviaRossi/TripleTrouble-V3}}
}