Dot
Dot is an experimental 9.8B-parameter text reasoning model built by MTEnt. This repository contains the complete BF16 Dot backbone, its separately trained recurrent-depth core, the exact loader, and measured release evidence.
The current stable release is Dot v0.4 Thinking. The newest verified checkpoint is Dot v0.5 Spatial Process — Stage 1. The name and runtime identity are Dot.
Dot v0.5 Spatial Process — Stage 1
This update adds the accepted recurrent core and repaired backbone layers from the spatial-process recovery track. It is published as a verified intermediate checkpoint while the longer Stage 2 process run continues.
| Held-out generation gate | Result |
|---|---|
| Valid thinking envelopes, 256 spatial primitives | 100% |
| Exact spatial answers | 99.609% |
| Verified one-step transition accuracy | 99.609% |
| Valid thinking envelopes, 256 non-spatial cases | 100% |
| Non-spatial exact-answer retention | 98.047% |
The checkpoint is at
checkpoints/v0.5-spatial-process-stage1/.
It contains the recurrent core plus repaired backbone layers 30 and 31 and the
final norm. The four unchanged BF16 backbone shards remain the base artifact.
Local mixed-precision integration used the unchanged backbone in NF4 and kept the repaired layers and recurrent core in BF16. It allocated 10.01 GB of CUDA memory after load on a 16 GB GPU, with CPU fallback disabled.
Two official ARC-AGI-3 development fixtures exercised the complete local model,
action gate, SQLite ledger, and replay path. Neither fixture was solved. This is
integration evidence, not an ARC-AGI-3 score. The observed failures and the next
training curriculum are documented in
docs/v0.5-local-harness-findings.md.
What changed in this release
Dot inserts a weight-tied recurrent-depth core after decoder layer 15. The core copies native layers 12 through 15 and runs them four times through learned residual gates.
| Component | Parameters |
|---|---|
| Dot backbone | 8,953,803,264 |
| Recurrent-depth core | 864,945,224 |
| Total instantiated model | 9,818,748,488 |
The v0.4 repair trained the recurrent core for 938 optimizer steps and processed
8,642,015 tokens from 60,000 generated reasoning records. The backbone stayed
frozen during this stage. Final active gate values were 0.157696, 0.079852,
0.059039, and 0.059151.
Measured results
These are narrow internal measurements, not general model benchmarks.
On 4,096 held-out prompts from the same eight executable task generators used to construct the curriculum:
| Teacher-forced metric | Zero-gate baseline | Dot v0.4 |
|---|---|---|
| Response NLL | 2.128394 | 0.017367 |
| Response token accuracy | 66.60% | 99.43% |
| Exact response rate | 0.00% | 87.77% |
| Reasoning token accuracy | 60.47% | 99.32% |
| Final-answer token accuracy | 99.61% | 99.99% |
On 256 separately seeded free-running prompts from those same task families, Dot produced a valid, non-empty thinking envelope in 100% of cases and matched the exact final answer in 248/256 cases (96.875%). No response hit the 256-token generation limit.
An adversarial spatial generalization probe was much weaker: Dot v0.4 scored 14/64 (21.875%) exact match on a newer spatial suite, including 0% on its orientation subset. That failure is why v0.4 should not be described as an ARC-AGI-capable model. A spatial repair is being evaluated separately and is not part of this stable release.
The machine-readable release metrics are in
eval/thinking-v0.4.json.
Run Dot
This is a custom architecture. A normal AutoModelForCausalLM.from_pretrained
call loads only the backbone and silently omits Dot's trained recurrent core.
Use the included loader.
git clone https://huggingface.co/MTEnt/dot
cd dot
python -m pip install .
python examples/chat.py --model . --prompt "Explain why a passing build does not prove the UI works."
To run the v0.5 Stage 1 checkpoint in BF16:
python examples/chat.py \
--model . \
--checkpoint checkpoints/v0.5-spatial-process-stage1 \
--prompt "Track a state change and explain which evidence determines the result."
The verified runtime used Python 3.12, PyTorch 2.8.0, Transformers 5.15.0, Safetensors 0.8.0, BF16, SDPA, and an NVIDIA H200. Other hardware and precision paths have not been verified for this release.
Cache-backed decoding is deliberately disabled. The recurrent passes do not yet have correct cache ownership, so enabling a normal KV cache would risk silently wrong state. Generation recomputes the sequence at every token and is therefore slow.
Files that matter
model-00001-of-00004.safetensorsthroughmodel-00004-of-00004.safetensors: the complete Dot v0.2 semantic backbone.reasoning_core.safetensors: the v0.4 recurrent-depth weights.checkpoints/v0.5-spatial-process-stage1/: the v0.5 Stage 1 recurrent core and repaired backbone delta, with its integrity manifest.dot_recurrent_manifest.json: architecture, source step, metrics, and core checksum.dot_rd/: the required architecture and integrity-checking loader.release.json: release lineage, hashes, training scope, and known limits.
Scope and limitations
- Dot v0.4 is a text-only research release.
- The reported reasoning score measures the same generator families used for training, with separate seeds and hash-disjoint records. It does not prove broad reasoning, coding, world knowledge, ARC-AGI, or safety performance.
- No independent safety, bias, multilingual, coding, or production-agent audit has been completed for v0.4.
- Thinking responses can expose intermediate text. Do not put secrets in a prompt and assume the reasoning channel will conceal them.
- The BF16 package is roughly 20 GB before runtime allocations. Only the H200 path described above is verified.
Technical lineage and license
Dot's semantic backbone was initialized from Qwen/Qwen3.5-9B and then modified
by MTEnt through a merged semantic LoRA stage and the recurrent-depth training
described here. The original training manifest recorded the source repository
but did not record its exact commit hash. The complete modified backbone is
included so this release does not depend on reconstructing that missing adapter.
The upstream work and this repository are distributed under Apache License 2.0.
See LICENSE, NOTICE, and release.json
for attribution and provenance. This model is provided as-is, without a warranty
of correctness, safety, or fitness for a particular purpose.