AEGIS

A domain-specialized 7B code model for embedded systems engineers. Built on free hardware by independent researchers.

A.E.G.I.S — Automated Embedded Generative Intelligence System

A domain-specialized 7B code model for embedded systems engineers. Built on free hardware by independent researchers.

What Is A.E.G.I.S?

A.E.G.I.S — Automated Embedded Generative Intelligence System. A General-purpose code models fail at embedded systems. They suggest malloc on a 2KB SRAM device. They produce recursive algorithms on platforms with no call stack budget. They give you Linux /dev/ttyUSB0 code when you asked about a microcontroller UART peripheral.

A.E.G.I.S was built to fix that.

It is a 7B parameter language model fine-tuned specifically for embedded systems development using QLoRA on a single NVIDIA T4 GPU — Google Colab free tier. It understands registers, hardware constraints, deterministic timing, and the low-level reasoning that general models get wrong.

Builders: C-28 & A-47 — Independent Researchers Base model: unsloth/Qwen2.5-Coder-7B-bnb-4bit Training: 12 runs, 6 version checkpoints, ~14 hours, 5,000 steps Final loss: 0.16366977691650392 Grad_norm: 0.07397811114788055 Learning_rate: -> 0-1k - 5e-5 -> 1k-2k - 2e-5 -> 2k-5k - 3e-5

Supported Domains

Model Card

Training — Staged Learning Rate

A key methodological contribution is the non-monotonic staged learning rate schedule:

Steps    0  – 1,000:  lr = 5e-5   ← peak — aggressive early domain acquisition
Steps 1,000 – 2,000: lr = 2e-5   ← pullback — consolidate weight updates
Steps 2,000 – 5,000: lr = 3e-5   ← recovery — steady domain convergence

Each stage uses cosine decay internally. This peak-pullback-recovery pattern differs from standard cosine warmup schedules and was developed through 12 iterative training runs.

Dataset

All sources are open-licensed. Full attribution in dataset/README.md.

Layer 1 — Reasoning Foundation (preserves general code reasoning)

Layer 2 — Domain Injection (embedded systems specialization)

Total datasets added ~37000

System Prompt Architecture

AEGIS uses a runtime-injected system prompt. The prompt is not baked into weights — it is loaded from system_prompt.md at inference time. This allows behavioral updates without retraining.

The prompt uses XML-tagged sections:

<aegis_identity>     — model identity and role
<aegis_code_principles> — 11 non-negotiable embedded coding rules
<aegis_debugging_protocol> — classify → root cause → mechanism → fix → verify
<aegis_response_format>    — platform → approach → code → notes
<aegis_tone> - to make it direct and concise
<aegis_constraints>
<aegis_identity_responses>
<aegis_easter_eggs> 

Known Limitations

• No validation split — training loss only, generalization unverified
• Single epoch (~1.06) — unknown if 0.1637 is true convergence floor
• Trained at 1024 tokens — long-context performance untested
• LeetCode domain bleed — Python algorithm patterns may surface without system prompt
• No formal benchmark evaluation — addressed in V2.0

Citation

@misc{aegis2026,
  title         = {A.E.G.I.S: Domain-Specialized QLoRA Fine-Tuning
                   for Embedded Systems Code Generation},
  author        = {C-28 and A-47},
  year          = {2026},
  month         = {June},
  note          = {Independent Researchers. No institutional affiliation.},
  url           = {https://github.com/eluricharles/AEGIS},
  archivePrefix = {arXiv},
  primaryClass  = {cs.LG}
}

Acknowledgements

The authors thank the open-source community whose datasets made this work possible, including m-a-p, NVIDIA, and the individual contributors listed in dataset/README.md. Training was tracked using Weights & Biases. Writing assistance was provided by AI language model tools; all scientific content, experimental design, and results are the authors' own. This work was conducted without institutional funding or compute resources.

License

Apache 2.0 — see LICENSE.

Built on free hardware. No institution. No shortcuts on the parts that matter. — C-28 & A-47