Model Card: JanusIR-500M-Research
Model Overview
- Model Name: JanusIR-500M-Research
- Scale Tier: 500M (Active Parameter Target: 545.7M)
- Architecture: Compiler-Aware Autoregressive Hybrid (Causal Transformer + RGAT + Gated Cross-Attention)
- Primary Domain: LLVM Intermediate Representation (IR) Static Analysis and Compiler Reasoning
- Active Parameter Count: 545.7M (545,692,999 parameters) (Trainable: 545,692,999)
- License: Apache 2.0 (Permissive Open-Weights)
Architectural Specification
| Parameter | Value | Description |
|---|---|---|
| Model Dimension (d_model) | 1280 | Hidden representation width |
| Transformer Layers (n_layers) | 20 | Depth of causal sequence decoder |
| Attention Heads (n_heads) | 16 | Number of query/key/value attention heads |
| Feed-Forward Dimension (d_ff) | 5120 | SwiGLU / MLP projection width |
| Max Sequence Length | 1024 tokens | Input IR context window |
| Vocabulary Size | 4096 | BPE Compiler Tokenizer vocabulary |
| Graph Hidden Dimension | 384 | RGAT structural node feature width |
| Graph Layers | 4 | Relational Graph Attention layers |
| Fusion Layers | 3 | Cross-modal text-graph attention layers |
| Positional Embeddings | Rotary (RoPE) | theta = 10000.0 |
| Layer Normalization | RMSNorm | epsilon = 1e-05 |
Training Curriculum
- Stage A (Lexical Language Pretraining): Causal language modeling on AnghaBench and open-source systems LLVM IR code corpora.
- Stage B (Structural Graph Representation): Multi-relational GAT pretraining over CFG, Def-Use, and SSA-Phi graph edges.
- Stage C (Joint Cross-Modal Pretraining): Dual-stream fused pretraining with structural alignment regularization.
- Stage D (Compiler Reasoning SFT): Multi-task instruction tuning across CFG reachability, def-use tracing, and SSA phi invariant resolution with completion-only loss masking.
Benchmark Evaluation Results (Champion Variant: Full Architecture + SFT)
The following table reports the verified performance achieved by the champion model across held-out compiler reasoning tasks:
| Benchmark Evaluation Task | Accuracy (Jaccard) | Precision | Recall | F1 Score |
|---|---|---|---|---|
| Control-Flow Graph (CFG) | 33.74% | 60.77% | 39.43% | 43.37% |
| Data-Flow Graph (DFG) | 22.47% | 30.00% | 27.13% | 25.28% |
| SSA Phi Invariants | 43.06% | 46.80% | 45.28% | 44.58% |
| Overall Combined | 33.09% | 45.86% | 37.28% | 37.74% |
- Syntactic Validity and Instruction Parsing: 100.0%
- Benchmark Evaluation Data: 50 held-out real-world C compiler programs.
- Evaluation Methodology: Exact compiler-grounded verification with deterministic greedy generation (temperature = 0.0).
Visual Benchmark Performance
Scientific Architectural Ablation & Failure Analysis
Intended Use Cases
- LLVM IR Control-Flow Graph (CFG) basic block analysis and reachability prediction
- Value tracing and def-use dependency extraction
- SSA phi provenance resolution and invariant verification
- Compiler optimization pass assistance (constant propagation, dead code elimination)
Out-of-Scope and Limitations
- Not intended for general conversational natural language dialogue or arbitrary source-to-source transpilation.
- Context window is bounded to 1024 tokens for this active tier (500M).
- Verification against standard LLVM tools (such as opt or llc) is recommended prior to deploying transformations in mission-critical environments.
License and Distribution
Released under the Apache License, Version 2.0.