TR-HASH-MOE-500M-HF — semantic-cell LoRA experiment
Failed promotion gate — research artifact only. This checkpoint is published for reproducibility and diagnosis. It is not a safe or reliable assistant and must not be used for consequential decisions.
Experimental merged checkpoint from step 1,052, trained for one full-shard epoch from the clean 500M/20B pretrained base. The LoRA weights are already merged; no separate adapter is required.
Training state
| Component | Value |
|---|---|
| Base | AETHORIA-AI/TR-HASH-MOE-500M-20B |
| Dataset | Complexity Atlas Posttrain 32K V2 |
| Training examples | 201,983 (all visited once) |
| Checkpoint | 1,052 / 1,052 steps (1 epoch) |
| Final training loss | 2.1509 |
| Matched eval loss | 3.6805 -> 2.9765 |
| LoRA | rank 32, alpha 32, dropout 0.05 |
| Peak base LR | 3e-6 |
| Peak expert LR | 7.5e-7 |
| Weighted loss | 20% reasoning / 20% natural conversation / 60% instruction |
| Chat format | complexity-chat-v2 |
The bundle contains merged BF16 weights, the 32K tokenizer, the standalone chat template, and the fixed layer-specific top-2 route tables.
Inference
pip install git+https://github.com/Complexity-ML/vllm-i64.git@main
vllm-i64 serve tr-hash-moe-500m \
--checkpoint AETHORIA-AI/TR-HASH-MOE-500M-HF \
--dtype bfloat16 \
--port 7860
Limitations
This checkpoint failed the behavioral promotion gate. Full PIQA normalized
accuracy was retained (0.6953 base -> 0.6964 candidate), but the fixed
nine-prompt assistant panel failed all nine behaviors. Observed failures include
incorrect arithmetic, prompt copying, fabricated identity details, ignored
format constraints, repetitive text, incomplete factual definitions, and an
unsafe password-sharing answer.
The result indicates an instruction-data coverage limitation rather than a general PIQA capability regression. Loss weighting cannot create semantic or stylistic coverage that is absent from the shard. Do not interpret the lower matched-eval loss as evidence of assistant quality.
No safety, correctness, or production-readiness claim is made. Users should retain an independently validated base or checkpoint for any real application.