Alya-1B Base
Developer: Prosperity AI
Repository: Efe2898/Prosperity-Family-Alya-Base
This repository currently contains a randomly initialized architecture checkpoint. It is not a pretrained language model yet.
Architecture
| Field | Value |
|---|---|
| Parameters | 1,026,353,824 |
| Layers | 28 |
| Gated DeltaNet | 21 |
| Local Gated GQA | 4 |
| Global Gated GQA | 3 |
| Hidden size | 1280 |
| SwiGLU intermediate | 4480 |
| Query / KV heads | 10 / 2 |
| Head dim | 128 |
| Partial RoPE | 64 / 128 |
| Local window | 4096 |
| Max position metadata | 65,536 |
| Vocabulary | 50,176 |
| Embedding / LM head | tied |
Tokenizer
Alya uses the tokenizer from vngrs-ai/Kumru-2B-Base pinned at revision:
55711ea224e4bf5d4e11a4baf79130ae73785ece
Usage
from transformers import AutoTokenizer, AutoModelForCausalLM
repo = "Efe2898/Prosperity-Family-Alya-Base"
tokenizer = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(
repo,
trust_remote_code=True,
)
Review custom code before enabling trust_remote_code=True.
Implementation note
modeling_alya.py is the readable PyTorch/Hugging Face reference implementation.
Production TPU training and long-context inference should use an optimized JAX/Pallas implementation that follows alya_architecture.json.
Status
- Native AlyaConfig: complete
- Native AlyaForCausalLM: complete
- Kumru tokenizer integration: complete
- Causal test: complete
- Save/load AutoClass test: complete
- ~1.026B parameter budget: complete
- Base pretraining: pending
- Long-context continuation: pending
- SFT: pending