NeuralAI-Nae1

🚀 Quick facts

📈 Training

Nae1 is pretrained from random initialization in eight cosine cycles: 0 → 4,000 · 4,000 → 8,000 · … · 24,000 → 28,000 · 28,000 → 32,000, each resumed from the previous checkpoint. The chart shows cycle eight — the first cycle trained with corrected next-token labels (an earlier double-shift bug had models predicting token t+2; the held-out gate was always standard next-token, so historical PPL numbers remain valid). Under the corrected objective the logged loss resumes at 28.87, falls to a best of 15.72 (step 31,500) and finishes at 17.93 as the learning rate anneals to zero. Held-out PPL improves 147.97 → 146.17 → 54.16 — a 63% drop from fixing the training objective; a fresh-corpus cross-check (unseen FineWeb CC-MAIN-2014-10: 57.09 vs step-28000's 143.64) confirms the jump is genuine generalization, not eval-set leakage.

• Data pipeline: FineWeb parquet → chunked into 513-token windows served as zero-copy views (peak-RAM-safe at 768M tokens)
• Checkpoints: every 250 steps; this release ships the final step-28000 checkpoint

📊 Evaluation

Teacher-forced perplexity on a held-out FineWeb slice (992 documents / 1,027,766 tokens, n_ctx=512), scored token-by-token with exact row alignment — not the echo=True logprob path, which mispairs positions and inflates NLL.

Raw report: eval_step32000_q4_n512.json — mean NLL 3.9919 nats/token; fresh-corpus cross-check: eval_fresh_step32000_q4_n512.json / eval_fresh_step28000_q4_n512.json (prior releases kept alongside: eval_step24000_q4_n512.json, eval_step20000_q4_n512.json, eval_step16000_q4_n512.json, eval_step8000_q4_n512.json). Fixed probe ("The quick brown fox", 10 tokens): 174.73 — the 10-token probe is noisy; held-out decides.

⚠️ Perplexity is the trust signal here, not raw capability. At 100M params over 768M tokens, expect exploratory, often garbled text — this is a research-lineage model, not a chat assistant.

Logits Parity Notice

Logits parity with the Nae1 native forward pass is not achievable by design: llama.cpp uses RMSNorm (no bias) while the native model uses LayerNorm (with bias), so conversion skips the bias tensors. Tensor integrity is fully verified (109/109 tensors match the source safetensors, max diff 0.00e+00) — small logit drift is a runtime constraint, not a conversion bug.

🛠️ Usage

llama.cpp (recommended)

llama-server -m nae1-llama-Q4_K_M.gguf -c 512 --host 127.0.0.1 --port 8080

llama-cpp-python

from llama_cpp import Llama

llm = Llama(
    model_path="nae1-llama-Q4_K_M.gguf",
    n_ctx=512,
    verbose=False,
)

out = llm("The future of AI", max_tokens=32, temperature=0.7)
print(out["choices"][0]["text"])

Files

🧰 What Is NeuralAI?

NeuralAI is a local-first, private generative AI engine built by De'Andrew Preston Harris. The mission: your AI, on your hardware, under your control. Nae1 is the project's first end-to-end pretrained model — trained from random init on public data, converted with verified tensor integrity, and evaluated with a reproducible protocol.

⚠️ Limitations

• Scale: 100M parameters pretrained on 768M tokens is a research checkpoint — long-form reasoning, coding, and factual recall are limited.
• Vocabulary: a 439-token BPE trained on this corpus; out-of-domain text will tokenize inefficiently.
• No chat template: base completion only — it was never instruction-tuned (SFT completed; checkpoint selection in progress).
• No internet access: pair with a tool layer if you need live data.

👤 Who Created NeuralAI?

• Founder & Lead Architect: De'Andrew Preston Harris (D. Harris / Dre)
• Hugging Face: @Subject-Emu-5259
• GitHub: @Subject-Emu-5259
• LinkedIn: linkedin.com/in/deandrewharris94
• Location: Memphis, Tennessee / West Memphis, Arkansas
• Education: AI Software Engineering at Maestro College

NeuralAI was built from resilience, fatherhood, and the belief that personal computing deserves personal intelligence. Every release is handcrafted, iterated, and documented in the open.

📖 Citation

@software{neuralai_nae1_2026,
  author       = {Harris, De'Andrew Preston},
  title        = {NeuralAI — Nae1},
  year         = {2026},
  url          = {https://huggingface.co/Subject-Emu-5259/NeuralAI-Nae1},
  version      = {step-32000},
  description  = {A 100M-parameter language model pretrained from scratch on 768M FineWeb tokens}
}

Built with discipline by De'Andrew Preston Harris. Maintained in the open. Updated whenever the model, dataset, or project state changes.