Conway-Omega (188M - v8)
Conway-Omega (v8 — Checkpoint Step 14,760)
An independently trained, deep-thin conversational language model by ConwAI.
Overview
Conway-Omega is a 188M-parameter generative language model trained completely from scratch by ConwAI. It uses a deep-thin transformer decoder architecture optimized for low-latency conversational inference and edge hardware execution (such as Apple Silicon unified memory / Metal and local workstations).
- Official Website & Live Web Platform: https://conw.ai
- Inference Provider:
ConwAI - Architecture Version: v8 (
source_step: 14760) - License: GNU General Public License v3.0 (GPLv3)
Architecture & Specifications
| Hyperparameter | Value | Description |
|---|---|---|
| Parameters | 188,299,776 (~188M) | Deep-thin transformer decoder |
Layers (n_layers) | 26 | Deep configuration for expressive feature hierarchy |
Hidden Dimension (dim) | 768 | Model embedding / residual stream width |
Attention Heads (n_heads) | 12 | Query heads |
KV Heads (n_kv_heads) | 4 | Grouped Query Attention (GQA 3:1 ratio) |
| FFN Hidden Dimension | 2048 | SwiGLU feed-forward network |
| Vocabulary Size | 32,000 | Byte-level BPE tokenizer (tokenizer.json) |
| Max Sequence Length | 1024 | Max context window |
| RoPE Base Theta | 10,000.0 | Rotary Position Embeddings with QK-Norm |
| Checkpoint Step | 14,760 | Production v8 release weights |
Model Weights & Formats Included
This repository provides multiple production-ready weight formats:
weights.npz— Native NumPy / MLX weights for Apple Silicon Metal acceleration.conw_omega.pt— PyTorch checkpoint containing full float16 model state dict.model.safetensors— Fast, zero-copy Hugging Face Safetensors format.config.json— Architecture definitions & special token IDs.tokenizer.json— 32k tokenizer vocabulary.serve.py&conwomega/— Self-contained FastAPI serving engine and browser chat UI.
Prompt Format & Special Tokens
Conway-Omega is trained with strict role markers. Every prompt turn is role-marked and newline-delimited, ending with an open <|omega|> turn:
<|user|>
you alright mate?
<|omega|>
Special Markers
- User turn:
<|user|> - Assistant turn:
<|omega|> - End of sequence:
<|eos|>(Token ID:0)
Quickstart & Usage
1. Standalone Web Server & UI
Clone or download the repo and run the self-contained server:
git clone https://huggingface.co/ConwAI/omega
cd omega
pip install -r requirements.txt
python serve.py --port 8080 --host 0.0.0.0
Open http://localhost:8080 in your browser for the chat UI, or query the JSON API:
curl -X POST http://localhost:8080/api/chat \
-H "Content-Type: application/json" \
-d '{
"history": [
{"role": "user", "content": "Hello Omega!"}
],
"max_new_tokens": 40,
"temperature": 0.8,
"top_p": 0.92,
"top_k": 40,
"repetition_penalty": 1.2
}'
2. Loading in PyTorch
import json
import torch
from tokenizers import Tokenizer
from conwomega.model import ConwOmega, ModelConfig
# 1. Load config and model
with open("config.json") as f:
cfg = json.load(f)
model = ConwOmega(ModelConfig(**cfg["model"]))
checkpoint = torch.load("conw_omega.pt", map_location="cpu", weights_only=True)
model.load_state_dict(checkpoint["model"])
model.eval()
# 2. Tokenize and generate
tokenizer = Tokenizer.from_file("tokenizer.json")
prompt = "<|user|>\nWhat are you working on?\n<|omega|>\n"
input_ids = torch.tensor([tokenizer.encode(prompt).ids])
with torch.no_grad():
logits, _ = model(input_ids)
next_token = torch.argmax(logits[:, -1, :], dim=-1)
print("Next token ID:", next_token.item())
License
This model and its associated code are licensed under the GNU General Public License v3.0 (GPLv3). See LICENSE for details.