MBG 1.0 — Model Bahasa Garuda
deepRcurs Labs / @deeprcurs · author: Mzed Imamkh / @mzedimamkh
MBG 1.0 ("Model Bahasa Garuda") is a research project building an Omni-Sparse Hybrid language model — a laddered-scaling architecture that combines an SSM/MLA core (Mamba-2 + DeepSeek-style multi-head latent attention), fine-grained MoE, and ternary {−1,0,1} weights (BitNet-style), with Trinity-Mirror as the core reasoning / answer-production controller.
This Hugging Face model repo is the external archive for the project: it stores the golden checkpoints and canonical weights. The dataset/corpus lives in a separate dataset repo (linked below), and the workspace snapshot holds only the controlling code and lightweight pointers.
Status: L0 (0.17M) and L1 (17M) rungs validated. This repo currently holds the L0/L1 validation checkpoints produced by the
mbg_mini_gpt.pyprototype, which exercises the same MoE + probe-routing + GUM-optimizer machinery used to validate the scaling, memory, and routing claims. The full Omni-Sparse Hybrid (SSM+MLA) is the target architecture and is specified in the internal design blueprint (MBG-1.0-rev-3_Blueprint.md), which is kept in the workspace snapshot only — not published to this repo.
Architecture summary (per blueprint rev-3)
- Omni-Sparse Hybrid core: interleaved Mamba-2/SSM + MLA (KV-compressed) layers with fine-grained MoE FFN, aux-loss-free bias routing, and ternary FFN-MoE weights (embeddings/attention/router stay BF16).
- Trinity-Mirror controller: Thinker → Critic → Refiner chained processor (bounded loop, confidence calibration, auditable mirror ledger). Inquiry tokens route probes to specialist MoE experts (probe→expert coupling).
- GUM optimizer (Innovation I1): GaLore-Unbiased + Muon — Muon FLOP efficiency with GaLore low-rank memory savings (~72% optimizer-state reduction, state/param ≈ 2.2 B).
Scaling ladder ([1 7, 0 8, 4 5] — mandatory mantissa pattern)
The scaling pattern is the mantissa digits [1 7, 0 8, 4 5] = 0.17M, 0.8M, 4.5M,
each column multiplied by ×100 per level:
0.17M → 17M → 1.7B≈1.8B · 0.8M → 80M → 8B · 4.5M → 450M → 45B.
This pattern is canonical and is reproduced here unchanged from the blueprint (§5).
| Level | Parameters (active) | Status |
|---|---|---|
| L0 | 0.17M / 0.8M / 4.5M | ✅ L0 validated (GUM stable, routing) |
| L1 | 17M / 80M | ✅ 17M validated; 80M next |
| L1+ | 450M (active; upcycle → MoE total ~1.5–3B) | planned |
| L2 | 1.7B / 8B (MoE total larger) | if external funding (multi-GPU) |
| L3 | 45B (total MoE, ~1.7–8B active) | if external funding |
Scaling beyond L1 is deferred until implementation planning is finalized and snapshot safety is guaranteed. L2 and above require external funding ("L3 ke atas = jika ada pendanaan"); capital buys compute, not debugging of an architecture that should already be validated at L0/L1.
Checkpoints in this repo
| Rung | Params | Canonical weights | Checkpoint (bf16) |
|---|---|---|---|
| L0 — 0.17M | 176,992 | — | golden/mbg_017m.pt |
| L1 — 17M | 16,748,928 | model.safetensors | golden/mbg_l1-17m_20260901-173235.pt |
.pt files are complete training checkpoints (weights + config + history) in bf16;
model.safetensors is the canonical 17M weight set.
File layout
MBG-1.0/ (external archive — clean flow)
├── README.md # this model card
├── config.json # current model config (L1 quality model, v2x)
├── model.safetensors # canonical weights (follows latest model update)
├── manifest.json # provenance: config, metrics, params, SHA-256 of every golden
├── tokenizer/
│ └── mbg_bpe.json # BPE tokenizer artifact
├── golden/ # complete training checkpoints (bf16 .pt)
├── source/ # training/eval/data source code (reproducibility)
└── reports/ # experiment reports (markdown + JSON)
Clean flow: the workspace snapshot holds only the controller (code, scripts, docs, corpus, manifest) — no
.pt/.safetensors. All model weights and checkpoints live in this repo and are downloaded on demand (e.g.internal/ops/sync.sh latest/pull).Public vs internal: everything published here (source code, reports, weights) is intentionally public. Internal design/governance artifacts — the design blueprint, the Trinity-Mirror component spec, the internal protocols/ops documents, and the ops scripts — live under
internal/in the snapshot and are never pushed to this repo.
Dataset
The English corpora used for training/evaluation are published in the dataset repo:
deeprcurs/MBG-1.0-data — see the dataset card there.
Quick start (load 17M weights)
pip install torch safetensors
from safetensors.torch import load_file
weights = load_file("model.safetensors") # 117 tensors (bf16; embeddings/norms fp32)
# Rebuild with src/mbg_mini_gpt.MbGPT(config.json) and load_state_dict(weights).
Integrity
manifest.json records the SHA-256 of every golden checkpoint so downloads can be
verified before use.
License
Dual License (custom, source-available) — see LICENSE.md.
- Personal / Non-Commercial Use: free, for private experimentation and research. Re-branding and claiming the work as your own are prohibited.
- Commercial Use / Re-branding / derivative redistribution: requires the author's prior written permission (commercial license fee applies).
The license applies to the model weights/checkpoints, tokenizer, config, training/evaluation code, and the datasets & corpus. This is not an open-source license; it intentionally restricts redistribution and use.