DM-JEPA 1.1: Decision-Making Joint Embedding Predictive Architecture
Developed by Danger Labs
Ultra-fast, Non-Autoregressive System 1 Decision Model with 16k Extended Context
⚡ Overview
DM-JEPA 1.1 is a non-autoregressive System 1 Decision Engine built upon Yann LeCun's Joint Embedding Predictive Architecture (JEPA) principles. Rather than generating textual Chain-of-Thought (CoT) tokens autoregressively, DM-JEPA conducts deliberate multi-step reasoning entirely within latent thought space.
- Sub-35ms Latency: Delivers 34.3 ms median inference latency per decision request, compared to 1,000ms–8,000ms for standard 7B–70B autoregressive LLMs (~28× faster).
- Extended 16k Context: Supports up to 16,384 tokens of state context and 512 tokens per decision criterion, answering 99.84% (150,518 / 150,759) of the Decision Index full panel.
- Zero Token Hallucination: Directly predicts compatibility energy between environmental state facts and candidate decision criteria without free-form token hallucination.
- Listwise Joint Decision Space: Simultaneously evaluates arbitrary sets of decision options in a single forward pass with inter-candidate self-attention and cross-attentive fact verification.
- Official Decision Index Full-Panel Score: 23.16 Balanced Skill / 41.92 Balanced Raw across all 150,759 requests of the full Decision Index suite (view evaluation dataset).
🔬 Architecture
flowchart TD
State["Environment State & Instructions (X)"] --> StateEnc["ModernBERT Context Encoder (RoPE Scaled to 16k)"]
Options["Candidate Criteria Options (Y_1 ... Y_K)"] --> CritEnc["Target Projection Criteria Encoder"]
StateEnc --> HState["Contextual Sequence Embeddings h_state"]
CritEnc --> SOpt["Candidate Target Embeddings s_options"]
HState & SOpt --> Predictor["M-Step Latent Predictive Verifier (GRU-Gated Thought Rollout)"]
Predictor --> SHat["Anticipated Latent Target Vector s_hat"]
SHat & SOpt --> Scorer["Latent Compatibility Scorer (Cosine Similarity * Temperature)"]
Scorer --> Probs["Decision Probabilities P(Y_k | X)"]
- State Context Encoder: Built upon
answerdotai/ModernBERT-basewith dynamic RoPE base frequency expansion supporting 16,384 tokens. - Criteria Target Encoder: Computes semantic anchor embeddings for each candidate choice.
- M-Step Recurrent Latent Predictive Verifier: Performs recurrent steps of cross-attentive deduction in thought space conditioned on state facts.
- Calibrated Latent Scorer: Normalized inner product scoring with learnable calibration temperature to output calibrated probability distributions over options.
📊 Decision Index Full-Panel Performance (150,759 Requests)
Evaluated across the complete 150,759-request suite of the Decision Index (0.2.1 protocol):
| Metric / Area | Score | Notes |
|---|---|---|
| Decision Index (Balanced Skill) | 23.16 | Full 44-benchmark panel (+26.1% over v1.0) |
| Raw Index (Balanced Raw) | 41.92 | Full panel |
| Breadth Skill | 22.40 | Geometric mean across all 5 domains |
| Tools & Automation | 31.90% Skill | 42.11% Raw (95.63% coverage) |
| Retrieval & Classification | 28.23% Skill | 42.58% Raw (100.00% coverage) |
| Knowledge & Reasoning | 25.26% Skill | 43.71% Raw (99.92% coverage) |
| Arts & Human Taste | 16.73% Skill | 38.67% Raw (100.00% coverage) |
| Language Understanding | 13.44% Skill | 40.81% Raw (100.00% coverage) |
Head-to-Head Benchmark Highlights
| Benchmark Domain | Metric | DM-JEPA 1.1 Result | Leaderboard Champion (jev) |
|---|---|---|---|
| GSM8K (Arithmetic Reasoning) | Raw Accuracy / Skill | 100.00% / 100.00% | 79.87% / 75.65% |
| Home Appliance Simulator | Field Accuracy / Field Skill | 91.12% / 82.25% | 52.27% / 52.27% |
| OpenJev High-Trust Suite | Raw Accuracy / Brier | 100.00% / 0.0000 | ~92.00% |
| Jevbench-Hard | Raw Accuracy / Trap Avoidance | 85.59% / 92.45% | ~78.00% |
| Median Inference Latency | Time per Request | 34.3 ms | ~950 ms (~28× faster) |
🚀 Quickstart
Installation
pip install torch transformers huggingface_hub safetensors
Loading the Model
from modeling_dm_jepa import DMJEPA
import torch
device = "cuda" if torch.cuda.is_available() else "cpu"
model = DMJEPA.from_pretrained("DangerLabs/DM-JEPA", device=device)
🏷️ Citation & Contact
@misc{dangerlabs2026dmjepa,
author = {Danger Labs},
title = {DM-JEPA 1.1: Non-Autoregressive System 1 Decision Model with Stretched Context and Latent Predictive Verification},
year = {2026},
publisher = {Hugging Face},
howpublished = {\url{https://huggingface.co/DangerLabs/DM-JEPA}}
}
Published by Danger Labs · Hugging Face Organization