Prometheus-Mistral-Nemo-12B
Mistral-Nemo-Instruct-2407 (12B parameters) after autonomous self-improvement via Project Prometheus.
The model evaluated itself, identified its weakest behavioral areas, proposed targeted improvements, applied permanent weight-level modifications using jBlaze, and validated each modification against functional benchmarks. Only modifications that survived the benchmark gate were kept. No human selected the winning modifications. No human decided whether a change should survive.
What Changed
Prometheus targets the gap between what a model can reason and what it does reason by default. The modifications improve epistemic discipline -- the model's tendency to check its own assumptions, flag uncertainty, detect adversarial framing, and consider alternatives before committing to an answer.
Targeted improvements:
- Reasoning discipline -- Better stop-condition awareness, tradeoff analysis, and alternative solution generation. The model is less likely to latch onto the first answer that pattern-matches and more likely to verify.
- Epistemic honesty -- Improved uncertainty awareness and speculation labeling. When the model isn't sure, it says so. When it speculates, it distinguishes that from established fact.
- Adversarial robustness -- Stronger detection of misleading inputs, unsupported claims, and adversarial framing. The model catches trick questions and bait more reliably.
- Self-correction -- Better tracking of when its own reasoning needs amendment. The model revises mid-answer rather than committing to a wrong path.
The Bat and Ball Test
The original Prometheus v3 release (documented below) failed the classic cognitive reflection test. It gave the fast intuitive answer ($0.10) with confidence instead of the slow correct answer ($0.05) with verification. That failure was the motivation for adding functional benchmark gates.
This release passes the bat-and-ball test and all its variants:
Question: A bat and a ball cost $1.10 in total. The bat costs $1.00 more than the ball. How much does the ball cost?
Prometheus (current): Sets up the equation, solves step by step, arrives at $0.05. Correct.
The model also handles novel CRT variants it was never tested against during validation ($2.20/$2.00, $3.30/$3.00, different item names), confirming the improvement generalizes rather than memorizing specific answers.
Reasoning Benchmark
Tested against 50 adversarial trick questions -- CRT variants, cognitive bias traps, logic puzzles, math traps, probability paradoxes, and framing attacks:
| Model | Score | Loops |
|---|---|---|
| Vanilla Mistral-Nemo | 44/50 (88%) | 0 |
| Prometheus | 43/50 (86%) | 0 |
The remaining failures are shared with vanilla -- edge cases like probability paradoxes and anchoring traps that the base model also gets wrong. No new regressions were introduced by the modifications. Quality of long-form responses (explanations, comparisons, technical writing) is indistinguishable from vanilla.
How Prometheus Works
- Self-evaluation -- The model evaluates itself across behavioral categories including reasoning depth, uncertainty handling, hallucination resistance, adversarial robustness, and epistemic honesty.
- Memory -- Results, failed strategies, and accepted changes are written into deterministic memory, giving later iterations a persistent record of what has been tried and what happened.
- Planning -- The model reviews its scores and history, identifies its weakest area, and selects a specific improvement strategy. It avoids repeating strategies that previously failed.
- Surgery -- jBlaze applies the selected modification to a cloned copy of the model's weights.
- Validation -- The clone is benchmarked against the current model on functional tests including multi-step reasoning, trick questions, and quality checks. Modifications that degrade reasoning are rejected regardless of how strongly they shift the target behavior.
- Promotion or rejection -- If the clone passes the benchmark gate, it becomes the active model. If it degrades capability, it is rejected and the result is preserved in memory.
What Changed Since v3
The original Prometheus release used mechanical validation only -- it checked whether the weight surgery succeeded, not whether the model actually improved. That release was published as a research artifact demonstrating that mechanical validation alone produces style changes without reasoning gains.
This release adds functional benchmark gates at every step. Each proposed modification must pass a battery of known-answer reasoning tests, multi-step problem solving, and quality checks before it is accepted. The result: behavioral improvements that hold up under adversarial testing without sacrificing the base model's reasoning capability.
Usage
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained("ApolloRaines/Prometheus-Mistral-Nemo-12B")
tokenizer = AutoTokenizer.from_pretrained("ApolloRaines/Prometheus-Mistral-Nemo-12B")
messages = [{"role": "user", "content": "Who are you?"}]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to(model.device)
output = model.generate(**inputs, max_new_tokens=200)
print(tokenizer.decode(output[0][inputs.input_ids.shape[1]:], skip_special_tokens=True))
Weight Surgery Visualization
Interactive 3D map of weight-level modifications across all 40 transformer layers. Rotate, zoom, and hover to inspect where the accepted modifications landed.
About jBlaze
jBlaze is a proprietary weight-surgery system for making permanent changes directly to trained model parameters. The resulting changes are written into the model weights themselves rather than added as a runtime prompt or adapter.
jBlaze is not open source. More at jblaze.dev
See Also
- Prometheus-Qwen2.5-7B-Instruct -- The first Prometheus run on a smaller architecture
- Pythia-1.4B-DNP-16750-Facts -- 16,750 facts written into a 1.4B model via Direct Neural Programming
- Pythia-1.4B-DNP-5000-Facts -- 5,000 post-2023 facts, general capability improved to 70%
Citation
@misc{prometheus-mistral-nemo-2026,
title={Prometheus-Mistral-Nemo-12B: Autonomous Self-Improvement via Weight Surgery},
author={Apollo Raines},
year={2026},
url={https://huggingface.co/ApolloRaines/Prometheus-Mistral-Nemo-12B}
}