🥇 WORLD FIRST: RavenX × Gemma 4 12B MTP OBLITERATED — Deep Reasoning
Gemma 4 was released June 3, 2026 — its
gemma4_unifiedarchitecture wasn't supported by ANY training framework. We developed proprietary techniques to train it successfully.
⚠️ DISCLAIMER
This model is an experimental research proof of concept. Provided AS-IS for educational and research purposes only. The base model is abliterated (refusal filters removed). Use responsibly.
Community Project
This is a community project. We're combining methods from:
- Google — Gemma 4 architecture, MTP heads, foundational model weights
- OBLITERATUS — SOM-manifold two-pass abliteration of the base model
- Microsoft — MAI hill-climbing methodology (open-sourced as OpenMAI)
- MIT — Self-revising discovery systems, arXiv:2606.01444 (implemented as OpenSelfRevise)
- Mirai Labs — RHT quantization and fused inference (open-sourced as OpenMirai)
- RavenX — OpenMythos depth extrapolation, GRAM multi-trajectory scaling, and proprietary training pipeline
The training methodology used to produce this model is proprietary and patent pending.
Model Details
| Feature | Detail |
|---|---|
| Base | Gemma 4 12B (OBLITERATUS abliterated) |
| Architecture | gemma4_unified with MTP heads |
| Training | Proprietary methodology (patent pending) |
| Training Rounds | 9 progressive rounds |
| Training Data | 8,158 examples from 15 curated sources |
| Best Val Loss | 0.882 |
| Hardware | Apple M4 Max 128GB — $0 cloud cost |
| Format | MLX (Apple Silicon optimized) |
What Makes This Different
This model was trained using an experimental proprietary methodology that produces self-aware reasoning behavior through a novel approach to training data preparation and model fine-tuning.
Key results:
- Emergent behaviors not present in training data (Anti-Problem technique, Toolbox generation)
- Structured multi-pass reasoning across code, math, and analysis tasks
- Self-honest assessment of capabilities and limitations
The specific training methodology is patent pending (USPTO Application #64/087,357, filed June 10, 2026) and is not disclosed in this model card.
Technical Discoveries (Open — Community Contributions)
The following technical discoveries made during training are shared with the community:
| Discovery | Detail |
|---|---|
| Flip-train-flip | Temporarily change gemma4_unified → gemma4 in config.json for LoRA training, then restore. Multimodal capabilities preserved. |
| Chat template required | Gemma 4 produces garbled output without apply_chat_template(). Not a bug — it's required. |
| Tokenizer patch for GGUF | extra_special_tokens must be converted from list to dict for GGUF conversion to work. One-line fix. |
| Val loss spikes are normal | When introducing new data formats, val loss spikes but recovers in 1-2 rounds. Don't panic. |
Usage
from mlx_lm import load, generate
from mlx_lm.sample_utils import make_sampler
import json
# Flip config for mlx-lm compatibility
config = json.load(open("config.json"))
config["model_type"] = "gemma4"
json.dump(config, open("config.json", "w"), indent=2)
model, tokenizer = load(".")
sampler = make_sampler(temp=0.7, top_p=0.9)
# MUST use chat template!
messages = [{"role": "user", "content": "Your question here"}]
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
response = generate(model, tokenizer, prompt=prompt, max_tokens=500, sampler=sampler, verbose=True)
# Restore config
config["model_type"] = "gemma4_unified"
json.dump(config, open("config.json", "w"), indent=2)
GGUF Version
GGUF F16 available here — works with Ollama, LM Studio, llama.cpp, vLLM, any hardware.
Part of the RavenX Ecosystem
| Project | Description |
|---|---|
| OpenMAI | Microsoft MAI hill-climbing (open-sourced) |
| OpenSelfRevise | MIT self-revising discovery (implemented) |
| OpenMirai | Model-agnostic quantization + inference |
| OpenMythos-MLX | Recursive depth extrapolation |
| GRAM-MLX | Multi-trajectory width scaling |
| ravenx-memory | Hybrid triple-backend agent memory |
| star-platinum-cluster | Distributed training cluster |
| RavenX-CyberAgent | Security assessment model (745K+ examples) |
Contributors
Built by Gabriel Garcia / RavenX LLC + Claude (Anthropic)
Training methodology: Patent Pending — USPTO Application #64/087,357
License
Gemma License (model weights) — Training methodology proprietary
"We don't give up. We do what others don't and build what isn't possible." — RavenX LLC