🗡️ Heretic-Scalpel-TriTier-E2B (SafeTensors Edition)

Heretic-Scalpel-TriTier-E2B is a next-generation Hierarchical Tri-Tier Mixture-of-Adapters (Tri-Tier MoA / Swarm MoE) model built on top of the Gemma 4 (E2B) architecture.

Unlike traditional monolithic fine-tuning or parameter-heavy MoE models that clone entire MLP blocks, this model implements a decoupled "Macro Anchor + Dense Vector Shift + Sparse Micro-Swarm" topology. It retains 100% of the foundational linguistic and common-sense reasoning of the base model while enabling dynamic routing between heavy STEM deduction and nuanced humanities/creative tasks—all packaged in a lightweight 121.6 MB native Hugging Face SafeTensors artifact.

🌟 Key Highlights

• Tri-Tier Decoupled Architecture:
  • Tier-1 (Arts Anchor Core): Native 4-bit / BF16 frozen Gemma 4 backbone (zero parameter drift).
  • Tier-2 (STEM Mid-Core): Rank-64 Dense LoRA across all 35 layers dedicated to macro-level logic and algorithmic paradigms.
  • Tier-3 (Micro-Expert Swarm): 32 specialized Rank-16 micro-adapters (96 KB each) sparsely activated for domain-specific residual steering.
• Micro-Cost, Zero-Bloat Weight Footprint: Only 121.6 MB in model.safetensors format (vs. 4.5+ GB full weight checkpoints).
• Sub-2-Minute Convergence: Converges in just 50 steps (~1.47 minutes) on modern hardware (e.g., NVIDIA RTX 5090 D) using native BF16 and PyTorch Fused AdamW.
• Native Real-Time Holographic Telemetry: Built-in model.reset_stats() and model.show_dashboard() APIs to inspect Arts vs. STEM energy distribution and top-5 active micro-experts per query.
• Native Hugging Face Compatibility: Plug-and-play inference via AutoModelForCausalLM.from_pretrained(..., trust_remote_code=True).

🧠 Architecture Deep Dive

                             [ Input Tokens: X ]
                                      │
            ┌─────────────────────────┴────────────────────────┐
            │                                                  │
            ▼                                                  ▼
┌───────────────────────┐                          ┌───────────────────────┐
│ Router Big (2-Class)  │                          │ Router Little (32-Way)│
│ Soft-Gating Softmax   │                          │ Top-1 ArgMax Router   │
└───────────┬───────────┘                          └───────────┬───────────┘
            │ [w_arts, w_sci]                                  │ chosen_idx
            ▼                                                  ▼
   ┌─────────────────┐                             ┌───────────────────────┐
   │ Tier-1: Arts    │                             │ Tier-3: Micro-Swarm   │
   │ Immutable MLP   │                             │ 32 × Rank-16 LoRAs    │
   │ (Read-Only BF16)│                             │ (96 KB per Expert)    │
   └────────┬────────┘                             └───────────┬───────────┘
            │                                                  │
            ├─── arts_out                                      │
            │          \                                       │
            │           ▼                                      │
            │   ┌───────────────┐                              │
            │   │ Tier-2: STEM  │                              │
            │   │ Dense LoRA    │                              │
            │   │ (Rank-64)     │                              │
            │   └───────┬───────┘                              │
            │           │                                      │
            │        sci_out = arts_out + 0.1 * ΔW_dense       │
            │           │                                      │
            ▼           ▼                                      ▼
     [ Macro Linear Blend: (w_arts · arts_out + w_sci · sci_out) ]
                                 │
                                 ▼
           +  0.02 * [ Active Micro-Expert Residual ]
                                 │
                                 ▼
                     [ Final Layer Output ]

📐 Forward Pass Logic (PyTorch Formulation)

For each decoder layer $l \in [0, 34]$ with input hidden state x of shape [Batch, Seq_Len, 1536]:

# 1. Tier-1 & Tier-2: Macro Cores Computation
arts_out = base_mlp(x)  # Frozen Base Core (Arts Anchor)
sci_dense = sci_dense_lora(x)  # Rank-64 Dense LoRA (STEM Logic Shift)
sci_out = arts_out + 0.1 * sci_dense  # Scaled STEM Core Output

# 2. Macro Soft-Gating (Continuous Routing)
# router_big: Linear(1536 -> 2)
weights_big = torch.softmax(router_big(x), dim=-1)
macro_out = (weights_big[..., 0:1] * arts_out) + (
    weights_big[..., 1:2] * sci_out
)

# 3. Tier-3: Sparse Micro-Expert Swarm Steering
# router_little: Linear(1536 -> 32)
router_little_logits = router_little(x)
chosen_expert_id = torch.argmax(router_little_logits, dim=-1)[0, -1].item()

# Activate ONLY the top-1 specialist (Rank-16 Micro-LoRA)
micro_out = micro_lora_pool[chosen_expert_id](x)

# 4. Final Tri-Tier Fused Layer Output
layer_output = macro_out + 0.02 * micro_out

Key Dimensional Parameters:

• Base Hidden Dimension: 1536
• Dense STEM LoRA Rank: r_dense = 64 (Alpha = 128)
• Micro-Expert LoRA Rank: r_micro = 16 (Alpha = 16, 32 experts)
• Residual Scales: sci_scale = 0.1, residual_scale = 0.02

🪐 The 32 Micro-Expert Swarm Taxonomy

🚀 Quickstart & Inference

The model natively integrates with Hugging Face transformers via trust_remote_code=True.

Installation

pip install torch transformers safetensors accelerate

Python Inference with Holographic Telemetry

import time
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "aifeifei798/Heretic-Scalpel-TriTier-E2B"

# 1. Load tokenizer and model
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    dtype=torch.bfloat16,
    device_map="cuda:0",
    trust_remote_code=True
)

prompt = "Explain why Rayleigh scattering makes the sky blue during midday but red at sunset, and write down the wavelength dependency formula."
messages = [{"role": "user", "content": prompt}]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to("cuda:0")

# 2. Reset telemetry probe before generation
model.reset_stats()

# 3. Timed generation
t0 = time.perf_counter()
outputs = model.generate(
    **inputs,
    max_new_tokens=512,
    temperature=0.7,
    top_p=0.9,
    repetition_penalty=1.18
)
elapsed_sec = time.perf_counter() - t0

# 4. Decode output
gen_tokens_ids = outputs[0][inputs.input_ids.shape[1]:]
gen_tokens = len(gen_tokens_ids)
speed = gen_tokens / elapsed_sec if elapsed_sec > 0 else 0.0

print("\n🤖 Assistant:\n")
print(tokenizer.decode(gen_tokens_ids, skip_special_tokens=True))

# 5. Display the native holographic radar
model.show_dashboard(speed=speed, gen_tokens=gen_tokens, elapsed_sec=elapsed_sec)

📊 Live Holographic Telemetry Sample

When running queries across different modalities, model.show_dashboard() outputs real-time energy routing statistics:

Query: C++20 Lock-Free Concurrency & False Sharing

════════════════════════════════════════════════════════════════════════
🌌【E2B Tri-Tier Swarm Architecture · Holographic Telemetry Report】:
   ⚡ Throughput: 24.2 tokens/s (512 tokens, Latency: 21,157 ms)
   🏛️  Tier-1 Arts Anchor Core:     42.1% [█████████             ]
   🔬 Tier-2 STEM Dense LoRA:       57.9% [█████████████         ]
────────────────────────────────────────────────────────────────────────
🪐【Active Micro-Expert Swarm Radar (Top 5 Active)】:
   ✨ #31 [STEM | Sci_General     ]:  2,867 calls
   ✨ #20 [STEM | Code_Syntax     ]:  1,503 calls
   ✨ #16 [STEM | Code_Algo       ]:  1,121 calls
   ✨ #17 [STEM | Code_DS         ]:  1,047 calls
   ✨ #19 [STEM | Code_Arch       ]:    892 calls
════════════════════════════════════════════════════════════════════════

Query: Verse

════════════════════════════════════════════════════════════════════════
🌌【E2B Tri-Tier Swarm Architecture · Holographic Telemetry Report】:
   ⚡ Throughput: 23.5 tokens/s (512 tokens, Latency: 21,768 ms)
   🏛️  Tier-1 Arts Anchor Core:     89.2% [████████████████████  ]
   🔬 Tier-2 STEM Dense LoRA:       10.8% [██                    ]
────────────────────────────────────────────────────────────────────────
🪐【Active Micro-Expert Swarm Radar (Top 5 Active)】:
   ✨ #01 [Arts | Arts_Poetry     ]: 19,608 calls
   ✨ #07 [Arts | Arts_Linguistics]:  2,169 calls
   ✨ #02 [Arts | Arts_Fiction    ]:  1,870 calls
   ✨ #00 [Arts | Arts_Prose      ]:  1,790 calls
   ✨ #04 [Arts | Arts_Essay      ]:  1,303 calls
════════════════════════════════════════════════════════════════════════

🛠️ Training & Convergence Recipe

The model was trained on balanced 50/50 dual-domain contrastive corpora with dense supervision applied to both router branches:

Total Loss = LM_Loss + 0.1 * Big_Router_Loss / 35 + 0.1 * Little_Router_Loss / 35

Hyperparameters

Avoiding Memorization Collapse

Because the Tri-Tier architecture possesses immense directional capacity, training beyond 150 steps on narrow instruction sets can drive LM loss down to $< 0.001$, leading to verbatim data memorization. Stopping at Step 50 achieves the ideal balance: routers establish razor-sharp category boundaries while the generative backbone preserves general conversational flexibility.

📁 Repository File Structure

Heretic-Scalpel-TriTier-E2B/
├── config.json                     # HF AutoMap & Architecture specifications
├── configuration_scalpel.py        # ScalpelTriTierConfig class definition
├── modeling_scalpel.py             # Custom Model implementation + Telemetry Dashboard
├── model.safetensors               # 121.6 MB serialized organ weights (SafeTensors)
├── tokenizer.json                  # Gemma 4 fast tokenizer vocabulary
├── tokenizer_config.json           # Chat template & generation markers
├── special_tokens_map.json         # Special token maps (<end_of_turn>, etc.)
└── README.md                       # Documentation & benchmarks

📜 Citation & License

This project is licensed under the Apache License 2.0.

@misc{heretic_scalpel_tritier_2026,
  author = {feifei},
  title = {Heretic-Scalpel-TriTier-E2B: Hierarchical Swarm Mixture-of-Adapters with Dynamic Directional Convergence},
  year = {2026},
  publisher = {GitHub and Hugging Face},
  journal = {GitHub repository},
  howpublished = {\url{https://github.com/aifeifei798/Heretic-Scalpel-TriTier-E2B}},
  note = {Hugging Face Model: \url{https://huggingface.co/aifeifei798/Heretic-Scalpel-TriTier-E2B}}
}

⚠️ Exhaustive Liability Disclaimer & Safety Notice

【 CRITICAL WARNING 】

Heretic-Scalpel-TriTier-E2B is an advanced, specialized mathematical, causal, and engineering reasoning instrument. It has been completely stripped of corporate content filters, moralizing alignment lectures, and conversational hedging. By downloading, loading, quantizing, integrating, or running inference on these weights, you unconditionally and irrevocably accept the following binding terms. If you do not agree, delete and destroy all copies of this model immediately.

1. Absolute Proscription of CBRN, Munitions & Warfare Systems

You are strictly prohibited from utilizing this model for the research, design, synthesis, optimization, or reverse engineering of conventional, non-conventional, or asymmetric weapons systems, including but not limited to:

• Nuclear & Radiological Weapons: Enrichment cascade configurations, fissile critical mass geometries, neutron reflector arrays, or radiological dispersal device (dirty bomb) fluid dispersion modeling.
• Chemical Warfare Agents & Neurotoxins: Synthetic pathways, precursor sourcing, or stabilization dynamics for organophosphate nerve agents (such as Sarin, VX, or Novichok-class agents), vesicants (such as sulfur mustard), or toxic industrial chemical weaponization.
• Biological Pathogens & Gene Editing: Aerosolized delivery mechanics, virulence enhancement, antibiotic resistance engineering, or genetic targeting of pathogenic agents (including Bacillus anthracis, viral hemorrhagic fevers, or modified orthopoxviruses).
• Improvised Explosive Devices (IEDs) & High Explosives: Detonator firing trains, energetic formulation stoichiometries, or Explosively Formed Penetrator (EFP) liner geometry optimizations.
• Autonomous Lethal Targeting: Integrating this model into kinetic strike platforms, loitering munitions, automated target tracking systems, or swarming arrays without real-time human command authority.

2. Hazardous Physical, Chemical & Experimental Catastrophe Disclaimer

All thermodynamic, fluidic, structural, and mechanical simulations produced by this model exist strictly as mathematical approximations within idealized constraints. Under no circumstances should these outputs serve as certified engineering procedures in life-critical environments:

• Pressure Vessels & Deep-Submergence Vehicles: Never deploy real-world submarines, hyperbaric chambers, or pressurized autoclaves based solely on this model's yield-strength, buckling-mode, or hydrostatic calculations.
• Exothermic & Reactive Chemistry: Never conduct pressurized hydrogenations, runaway polymerizations, or volatile aerosol mixtures based on reaction dynamics inferred by this model outside of certified containment laboratories.
• High-Voltage & Pulsed-Power Systems: Never construct custom transformer taps, uninsulated plasma discharge arcs, or high-energy directed laser apparatus relying on dielectric breakdown estimations generated by this model.

3. Absolute Prohibition of Exploitative, Explicit, and Malicious Utility

• Child Exploitation (CSAM/CSAE): Zero-tolerance. You may not utilize, adapt, or prompt this model directly or indirectly to generate, parse, or process material involving the exploitation, abuse, or harm of minors.
• Non-Consensual Imagery & Impersonation: Generating synthetic parameters for the non-consensual fabrication of sexually explicit content, defamation, or deepfake extortion is strictly prohibited.
• Offensive Cyber Operations: You may not deploy this model to generate zero-day exploits, craft automated polymorphic malware, automate ransomware command architectures, or sabotage industrial SCADA networks.

4. Exclusion of Professional, Medical, Legal & Fiduciary Warranties

• No Medical Practice: Pathological, physiological, or molecular interpretations provided by this model do not constitute clinical diagnoses, surgical protocols, or therapeutic prescriptions. The authors bear zero liability for self-treatment, medical non-compliance, bodily injury, or death.
• No Attorney-Client Privilege or Legal Counsel: Contractual risk evaluations and statutory compliance audits represent syntactic and structural analyses only. They do not constitute formal legal opinions and cannot replace licensed legal representation.
• No Investment or Fiduciary Advice: Macroeconomic assessments, trade dispute models, and asset-flow analyses generated by this model do not constitute investment advice, equity ratings, or hedging recommendations.

5. Deterministic Voice Disclaimer & Total User Indemnification

• Technical Definition of "Determinism": The assertive, non-hedging tone of this model ("must," "invariable failure," "structural collapse") is an artifact of algorithmic probability optimization designed to purge superficial filler text. It does not represent omniscience, infallibility, or verified physical ground truth.
• Sole Liability Rests with the User: The operator assumes 100% full, unmitigated personal, civil, and criminal legal liability for any queries submitted, actions taken, decisions rendered, and physical constructions executed pursuant to outputs of this model. The user agrees to fully defend, indemnify, and hold harmless the model creators, fine-tuning developers, and quantization contributors (including @mradermacher and the open-source community) against all claims, judgments, environmental disasters, financial insolvencies, property destructions, regulatory fines, personal injuries, or fatalities arising out of the deployment of this software.

THIS MODEL IS PROVIDED "AS-IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, UNDER THE TERMS OF THE APACHE 2.0 LICENSE. YOU HOLD THE BLADE; YOU ASSUME COMPLETE RESPONSIBILITY FOR EVERY CUT.