⚡ TripleTrouble (V1) ⚡

The Sovereign Foundation: Code, Tool Agency, and World Simulation

Model Overview • The Triad • Merge Engineering • Serving with vLLM • Transformers

🌌 Overview

TripleTrouble (V1) is the original, foundational release in the TripleTrouble series. It unites three specialized fine-tunes of the Qwen 35B-A3B sparse Mixture-of-Experts (MoE) family into a single agentic model:

1. The Coder (KAT-Coder-V2.5-Dev): Autonomous repository manipulation, SWE-bench refactoring, codebase traversal, and concrete AST syntax generation.
2. The Agent (Ornith-1.5-35B-A3B): Complex multi-turn tool calling, structured JSON emissions, mathematical deduction, and competitive programmatic reasoning.
3. The Simulator (Qwen-AgentWorld-35B-A3B): Large World Model (LWM) dynamics, environment state transition modeling (MCP, OS, Web, Android), and next-state observation prediction.

Why V1 Outperformed Task-Vector Pruning

While discrete coordinate-pruning methods (such as TIES or DARE) often break down on fine-grained MoE models by zeroing out coordinates and shattering expert rank, TripleTrouble V1 preserves full matrix rank across all 256 experts. By combining continuous directional consensus with Frobenius Norm Restoration, V1 acts as a continuous Bayesian ensemble that smooths parameter drift across Qwen 3.5 and 3.6 lineages without introducing coordinate discontinuities.

🧬 The Triad

flowchart TD
    subgraph MERGED ["⚡ TripleTrouble (V1)"]
        ROOT["<b>TripleTrouble (V1)</b><br/>34.7B MoE • ~3.3B Active per token<br/><i>Continuous Hyperspherical Soup</i>"]
    end

    ROOT -->|"w₁ = 0.40"| KAT["💻 <b>KAT-Coder-V2.5-Dev</b><br/>• Code & SWE-bench Refactoring<br/>• AST & Syntax Trees<br/>• Terminal Command Logic"]
    ROOT -->|"w₂ = 0.35"| ORN["🦅 <b>Ornith-1.5-35B-A3B</b><br/>• Multi-Turn Tool Calling<br/>• MCP Protocol Execution<br/>• Structured JSON & Math"]
    ROOT -->|"w₃ = 0.25"| AGW["🌐 <b>Qwen-AgentWorld-35B</b><br/>• Large World Model (LWM)<br/>• Environment Dynamics<br/>• State Observation Loops"]

    classDef default fill:#1e293b,stroke:#475569,stroke-width:1px,color:#f8fafc;
    classDef highlight fill:#4338ca,stroke:#818cf8,stroke-width:2px,color:#ffffff;
    class ROOT highlight;

🧮 Merge Mathematics

Traditional model mergers suffer from severe variance decay when combining independently trained checkpoints. In high dimensions ($d \approx 10^6$), parameter vectors are nearly orthogonal ($\cos \theta \approx 0$). Taking an unscaled weighted mean causes the expected norm to shrink by roughly $\approx 1/\sqrt{K}$, resulting in activation dampening and signal loss across the 40 layers.

TripleTrouble V1 solves this by routing parameter types to specialized geometric operators:

1. Directional Consensus with Frobenius Norm Restoration

For all attention projections (GQA query/key/value/output), Gated DeltaNet recurrence matrices, and all 256 routed MoE experts, the directional consensus vector is projected back to match the weighted expected Frobenius magnitude:

$$W_{\text{linear}} = \sum_{i=1}^3 w_i W_i$$

$$W_{\text{merged}} = W_{\text{linear}} \cdot \frac{\sum_{i=1}^3 w_i |W_i|F}{|W{\text{linear}}|_F}$$

• Static Weights: $w_{\text{kat}} = 0.40$, $w_{\text{ornith}} = 0.35$, $w_{\text{agentworld}} = 0.25$.
• Rank Preservation: Unlike coordinate pruning, no weights are zeroed out; the full singular value spectrum of each expert is preserved.

2. Router Gate Sharpness Preservation

Averaging router gate weights ($W_{\text{gate}} \in \mathbb{R}^{256 \times d_{\text{model}}}$) without scale correction decreases their Frobenius norm, which flattens the router's softmax distribution and leads to uniform, unconfident expert selection. V1 rescales the merged gate weights to the maximal source Frobenius norm:

$$W_{\text{gate, merged}} = \left( \sum_{i=1}^3 w_i W_{\text{gate}, i} \right) \cdot \frac{\max_i |W_{\text{gate}, i}|F}{\left| \sum{i=1}^3 w_i W_{\text{gate}, i} \right|_F}$$

This maintains expert routing temperature and keeps the top-8 dispatch boundaries sharp.

3. Component Policy Breakdown

⚙️ Architectural Specifications

🚀 Fast Inference with vLLM

Thanks to its sparse MoE architecture, TripleTrouble generates text with the low latency of a ~3.3B dense model while leveraging 35B-scale parametric knowledge.

Installation

pip install vllm>=0.6.0

Launch an OpenAI-Compatible API Server

vllm serve OliviaRossi/TripleTrouble \
  --tensor-parallel-size 2 \
  --max-model-len 32768 \
  --gpu-memory-utilization 0.90 \
  --trust-remote-code

(For a single 80GB GPU, run using FP8 or AWQ quantization: --quantization fp8).

💻 Quickstart: Transformers

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "OliviaRossi/TripleTrouble"

tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
    trust_remote_code=True
)

messages = [
    {
        "role": "system",
        "content": (
            "You are TripleTrouble, an expert agent combining deep codebase mastery, "
            "rigorous multi-turn tool planning, and environment simulation capabilities."
        )
    },
    {
        "role": "user",
        "content": "Design an event-driven system architecture in Python that monitors filesystem changes, extracts AST diffs, and publishes structured webhooks."
    }
]

prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

outputs = model.generate(
    **inputs,
    max_new_tokens=2048,
    temperature=0.6,
    top_p=0.9,
    repetition_penalty=1.05
)

response = tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True)
print(response)

🛠️ Recommended Sampling Parameters

⚖️ License & Attribution

This model is released under the Apache 2.0 License.

Source Checkpoints:

• KAT-Coder-V2.5-Dev by Kwaipilot
• Ornith-1.5-35B-A3B by ornith-ai
• Qwen-AgentWorld-35B-A3B by Qwen / Alibaba Cloud

@misc{tripletrouble2026,
  author = {Olivia Rossi},
  title = {TripleTrouble: A Component-Aware Norm-Balanced Hyperspherical Merge of Code, Tool Agency, and World Simulation},
  year = {2026},
  publisher = {Hugging Face},
  howpublished = {\url{https://huggingface.co/OliviaRossi/TripleTrouble}}
}