MCGPT-1: Mixture of Experts (MoE) Language Model
MCGPT-1 is a custom-built MoE model developed by TopAI-IL. It is designed to demonstrate specialized knowledge in Minecraft, Reddit-style conversations, and model self-identity.
Model Details
- Architecture: Mixture of Experts (MoE)
- Total Experts: 4
- Layers: 4
- Attention Heads: 8
- Hidden Size: 256
- Training Domains: 1. Identity (TopAI-IL) 2. Minecraft Technical Data & Guides 3. Reddit/Web Slang & Conversations 4. General Hebrew/English Knowledge 5. Instructions Syntetic-Data
How to use
This model uses a custom architecture (mcgpt). To run inference, ensure you include the architecture class in your code or use the trust_remote_code=True flag if the modeling script is provided.
Use example
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
model_id = "TopAI-1/MCGPT-1"
# 2. load the model and tokienizer
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
model_id,
trust_remote_code=True,
torch_dtype=torch.float32
)
# 3. GPU If have
device = "cuda" if torch.cuda.is_available() else "cpu"
model.to(device)
model.eval()
# 4. fast text generation
def generate(prompt, max_new_tokens=50):
inputs = tokenizer(prompt, return_tensors="pt").to(device)
with torch.no_grad():
outputs = model.generate(
**inputs,
max_new_tokens=max_new_tokens,
do_sample=True,
top_k=50,
temperature=0.8,
pad_token_id=tokenizer.eos_token_id
)
return tokenizer.decode(outputs[0], skip_special_tokens=True)
# Infrence Check
print("Testing MCGPT-1 from Hub:")
prompt = "use the following search parameters to narrow your results: e.g."
print(generate(prompt))
Capabilities
The model successfully identifies itself as MCGPT-1 and can switch between experts based on the prompt (e.g., providing Minecraft-related advice when prompted with "help").
Developed by @ TopAI-IL (2026)