🤖 A.M.I.T. 1.0 — Anchored Multi-depth Inference Transformer

A.M.I.T. 1.0 (Anchored Multi-depth Inference Transformer) is an autonomous, ultra-efficient Small Language Model (SLM) engineered by Amit Pathak, built on top of the Qwen base model architecture backbone. It introduces dynamic compute allocation and residual state stabilization to solve static compute inefficiencies in modern transformer architectures.

🌟 Key Architectural Innovations

1. ⚡ Dynamic Compute GRPO Policy Routing

Rather than processing every token through a fixed, heavy neural stack, A.M.I.T. 1.0 incorporates a stochastic policy router trained via Group Relative Policy Optimization (GRPO) with task-correctness rewards.

• Token-Norm Variance Analysis: The router extracts token-norm variance features to assess sequence complexity in real-time.
• Dual Execution Tracks: Automatically allocates compute between a ⚡ Shallow Fast Pass (8 Layers) for ultra-low latency queries and a 🔥 Deep Core Pass (32 Layers) for complex reasoning challenges.

2. ⚓ 80/20 Residual Core Stabilizer

To prevent feature degradation and vanishing gradients across deep recurrent or multi-depth execution loops, A.M.I.T. 1.0 implements an 80/20 residual core stabilizer: $$\mathbf{h}{\text{next}} = 0.8 \cdot \text{FFN}(\mathbf{h}) + 0.2 \cdot \mathbf{x}{\text{input}}$$ This mechanism anchors deep hidden representations back to the input embeddings, preserving semantic fidelity across variable execution depths.

📊 Model Specifications

💻 How to Use

🐍 Standard Transformers Inference

You can load and run A.M.I.T. 1.0 directly using Hugging Face transformers:

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

model_id = "Amit0392/AMIT-1.0"

tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_id, 
    torch_dtype=torch.float16, 
    device_map="auto"
)

messages = [
    {"role": "system", "content": "You are AMIT 1.0, an autonomous AI model developed by Amit Pathak."},
    {"role": "user", "content": "Explain quantum computing in simple terms."}
]

prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

outputs = model.generate(**inputs, max_new_tokens=512, do_sample=True, temperature=0.7)
response = tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True)

print(response)

📜 Citation & Attribution

If you use A.M.I.T. 1.0 or its underlying Anchored Multi-depth architecture in your research or applications, please cite:

@article{pathak2026amit,
  title={A.M.I.T. 1.0: Anchored Multi-depth Inference Transformer with GRPO Compute Routing},
  author={Pathak, Amit},
  year={2026}
}

📈 Official Benchmark Results

Evaluated using lm-evaluation-harness on standard zero-shot and few-shot evaluation tasks:

🏆 Comparative Leaderboard (0.5B – 3B Scale)

Comparison against leading open-source models in the sub-3B parameter class: