🦋 Overview
SOVYTHOS-66M-Instruct is the very first experimental instruction-tuned release of the SOVYTHOS language model family.
Unlike many open-source projects that begin from existing checkpoints, SOVYTHOS was built entirely from randomly initialized weights using a custom tokenizer and a custom training pipeline.
This release represents the first Proof of Concept (PoC) demonstrating that the SOVYTHOS architecture can learn instruction-following capabilities from scratch.
🇪🇬 Proudly designed and trained in Egypt.
⚠️ Experimental Release (v1)
This repository contains an early experimental version.
It is NOT a stable release.
You should expect:
- Hallucinations
- Broken generations
- Incomplete answers
- Incorrect reasoning
- Context drift
- Repetition
- Mixed-language outputs
- Inconsistent coding ability
This behavior is completely expected.
The goal of this release is to demonstrate the project's progress rather than provide a production-ready assistant.
🎯 Proof of Concept
SOVYTHOS-66M-Instruct exists primarily to prove that:
- A language model can be built completely from scratch.
- Instruction tuning can be performed without relying on pretrained foundation models.
- The Sovereign training pipeline works end-to-end.
- Future larger models can be built upon the same foundation.
This is Version 1 of a much larger roadmap.
🚀 Experience the Sovereign Platform
The public model is only one part of the Sovereign ecosystem.
The complete Sovereign Platform combines multiple AI systems and provides a modern AI experience beyond this standalone model.
📊 Model Specifications
| Property | Value |
|---|---|
| Model | SOVYTHOS-66M-Instruct |
| Parameters | ~66.7 Million |
| Architecture | Decoder-only Transformer |
| Hidden Size | 512 |
| Layers | 12 |
| Attention Heads | 16 |
| Framework | PyTorch |
| Tokenizer | Custom |
| Training | From Scratch |
| Fine-tuning | Instruction Tuning |
| Status | Experimental |
| Version | v1 |
🌍 Languages
Current multilingual support includes:
- 🇪🇬 Arabic
- 🇪🇬 Egyptian Arabic
- 🇺🇸 English
- Programming Languages
- Technical Documentation
- Mathematical Content
- General Text
Support quality is still under active development.
💻 Intended Uses
This model is intended for:
- Research
- Experiments
- Community Feedback
- Educational Purposes
- Benchmarking
- Continued Training
- Fine-tuning
It is not recommended for production environments.
⚠️ Known Limitations
Because this is an early instruction model, it currently struggles with:
- Long conversations
- Mathematical reasoning
- Accurate factual recall
- Multi-step reasoning
- Reliable code generation
- Stable instruction following
- Context consistency
Future releases aim to improve all of these areas.
🏗️ Built From Scratch
SOVYTHOS was developed using:
- Custom Tokenizer
- Custom Training Pipeline
- PyTorch
- CUDA
- Mixed Precision Training
- Automatic Checkpoint Selection
No pretrained language model weights were used.
Everything starts from randomly initialized parameters.
🇪🇬 Made in Egypt
SOVYTHOS is an independent Egyptian AI research project developed under the Sovereign AI Project.
Its long-term goal is to contribute to open-source AI with a strong focus on:
- Arabic Language
- Egyptian Arabic
- Programming
- Open Research
- Accessible AI Technology
🛣️ Roadmap
✅ SOVYTHOS-66M-Base
🟡 SOVYTHOS-66M-Instruct (Experimental v1)
⬜ Stable Instruct Release
⬜ Better Arabic Understanding
⬜ Better English Understanding
⬜ Better Coding
⬜ Better Reasoning
⬜ Longer Context
⬜ 150M Model
⬜ Larger Foundation Models
⬜ Sovereign Native AI Models
🌌 Future Vision
This repository is only the beginning.
Future generations of SOVYTHOS will focus on:
- Better reasoning
- Cleaner instruction following
- Improved multilingual understanding
- Stronger programming abilities
- Larger parameter models
- Better efficiency
- More reliable outputs
The long-term objective is to build an independent family of open-source language models developed under the Sovereign AI Project.
📖 Citation
@misc{sovythos66minstruct,
title={SOVYTHOS-66M-Instruct},
author={Mahmoud Yasser},
year={2026},
publisher={Hugging Face},
}
👨💻 Creator
Mahmoud Yasser
Founder of the Sovereign AI Project
📄 License
Apache License 2.0
🦋 Every model starts somewhere.
SOVYTHOS-66M-Instruct v1 is an experimental milestone and a proof that building a language model completely from scratch is possible.
It is not the destination—it is the first step toward a larger family of Sovereign AI models.
Thank you for being part of the journey.
⭐ If you find this project interesting, consider starring the repository and following future releases.