ISOM-R1-Edge-130M-MoE: Continuous Recurrent SSM + Mixture-of-Experts
0.0469 MB Invariant State Footprint • Verified on NVIDIA Tesla T4 Cloud GPU • 100K Continuous Token Stream
Overview
ISOM-R1-Edge-130M-MoE is an ultra-compact 134.89-million parameter (58.27M active per token) continuous recurrent state-space architecture. Engineered specifically as a high-speed speculative decoding drafter and edge deliberation engine, it processes arbitrarily long token streams without materializing quadratic attention matrices.
| Model | Primary Architecture Role | Base Lineage (Independent Derivative) | Total / Active Parameters | Max Context | Cache Complexity | Hardware Target |
|---|---|---|---|---|---|---|
| ISOM-R1-Coder-16B-MoE | 160K Bounded Code & MLA MoE | DeepSeek-Coder-V2-Lite (Non-Endorsed) | 15.71B / 2.36B Active | 163,840 (160K) | O(1) Bounded Manifold (Architectural Spec) | 16GB Cloud / Multi-GPU |
| ISOM-R1-Enterprise-40B | 40B System-2 Foundation Reasoning | Falcon-40B (Non-Endorsed) | 40.0B Dense | 32,768 (32K) | O(1) Bounded State (Architectural Spec) | Enterprise Multi-GPU (24GB-80GB) |
| ISOM-R1-Coder-1.5B-Instruct | 128K Repository Code Intelligence | Qwen2.5-Coder-1.5B-Instruct (Non-Endorsed) | 1.54B Dense | 131,072 (128K) | O(1) Bounded State (Tesla T4 Verified) | 8GB Developer Laptops / Edge |
| ISOM-R1-Reasoning-1.5B-Instruct | 32K System-2 Mathematical Deliberation | Qwen2.5-1.5B-Instruct (Non-Endorsed) | 1.54B Dense | 32,768 (32K) | O(1) Bounded State (Tesla T4 Verified) | 8GB Edge / Consumer GPUs |
| ISOM-R1-Edge-130M-MoE | Unbounded Recurrent Drafter & SSM | Standalone Continuous SSM + MoE | 134.89M / 58.27M Active | Unbounded Recurrence | O(1) Recurrent State (0.0469 MB Verified) | Ultra-Low Power Edge & CPU |
📊 Audited Empirical Hardware Telemetry (NVIDIA Tesla T4, Kaggle Cloud)
Evaluated on an NVIDIA Tesla T4 (14.56 GB / 14,911.7 MB total VRAM, PyTorch 2.10.0+cu128, CUDA 12.8, Kaggle Cloud) across an authentic, unpadded continuous literature stream (Pride and Prejudice, 728,846 characters) from 10,000 up to 100,000 continuous tokens:
Continuous Recurrent Scaling (10,000 to 100,000 Tokens)
| Continuous Stream Length | Active Recurrent State | Allocated GPU Memory | Peak GPU VRAM | Generation Throughput | Spatial Complexity Profile | Hardware Status |
|---|---|---|---|---|---|---|
| 10,000 tokens | 0.0469 MB | 1,176.9 MB | 2,631.8 MB | 8,645.2 tok/s | Constant O(1) (< 1 MB) | SUCCESS |
| 25,000 tokens | 0.0469 MB | 1,656.4 MB | 4,795.1 MB | 14,476.6 tok/s | Constant O(1) (< 1 MB) | SUCCESS |
| 50,000 tokens | 0.0469 MB | 2,614.9 MB | 7,686.7 MB | 17,844.6 tok/s | Constant O(1) (< 1 MB) | SUCCESS |
| 75,000 tokens | 0.0469 MB | 2,614.9 MB | 8,645.2 MB | 19,885.7 tok/s | Constant O(1) (< 1 MB) | SUCCESS |
| 100,000 tokens | 0.0469 MB | 2,614.9 MB | 8,645.2 MB | 21,049.0 tok/s | Constant O(1) (< 1 MB) | SUCCESS |
Key Architectural Verification: Across the entire 100,000 continuous token stream, the recurrent hidden state footprint remains strictly invariant at 0.0469 MB (48 KB). The continuous Cayley SO(d) manifold preserves numerical isometry, delivering processing speeds scaling up to 21,049.0 tokens/second on a single Tesla T4 GPU.
Model Specifications
| Parameter | Value |
|---|---|
| Total Parameters | 134.89M (58.27M active per forward pass) |
| Architecture | Continuous Isometric SSM + Mixture-of-Experts |
| Layers | 24 ISOM Recurrent Layers |
| Hidden Dimension (d_model) | 512 |
| SSM State Dimension (d_state) | 8 |
| Experts | 8 SwiGLU Experts per layer (TopK=2) |
| Recurrent Working State | 0.0469 MB (O(1) constant footprint) |
| Context Window | Unbounded (bounded-memory recurrence) |
| Precision | bfloat16 / float32 |
Quickstart Inference
from transformers import AutoTokenizer, AutoModelForCausalLM
import torch
model_id = "Prannesshkva/ISOM-R1-Edge-130M-MoE"
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
model_id,
torch_dtype=torch.bfloat16,
device_map="auto",
trust_remote_code=True
)
prompt = "Solve step by step: If a car travels 90 km/h for 3.5 hours, what is the total distance traveled?"
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
with torch.no_grad():
outputs = model.generate(
**inputs,
max_new_tokens=150,
temperature=0.7,
do_sample=True
)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
Citation & Licensing
@software{isom_edge_130m_2026,
author = {Prannessh K.V.A.},
title = {ISOM-R1-Edge-130M-MoE: Continuous Recurrent SSM + Mixture-of-Experts Drafter},
year = {2026},
publisher = {Zenodo},
doi = {10.5281/zenodo.22649142},
url = {https://doi.org/10.5281/zenodo.22649142}
}
- Sole Author & Architect: Prannessh K.V.A.
- LinkedIn: Prannessh K.V.A.
- License: Governed by CC BY-NC-ND 4.0 (Non-Commercial Research) & Enterprise Commercial Terms. See LICENSE.
Notice of Non-Endorsement & Independent Lineage
[!IMPORTANT] Independent Architecture Work:
ISOM-R1-Edge-130M-MoEis an original standalone research architecture engineered solely by Prannessh K.V.A. (Author, Architect & IP Holder). It implements continuous isometric state operator manifolds (Cayley SO(d)) combined with 8 SwiGLU Mixture-of-Experts feedforward layers. Governed by CC BY-NC-ND 4.0 & Enterprise Commercial Terms (see LICENSE).