CODE: https://github.com/frank-morales2020/AST/blob/main/DEEPSEEK_PRIME_ANCHORE_LLM.ipynb
deepseek-governed-no-amnesia
An experimental repository pairing DeepSeek-R1-Distill-Qwen-7B with a "Prime-Anchored Spectral Governor" artifact. This card describes exactly what the repository contains and what the accompanying run did and did not do.
What this repository is
- The model weights are identical to the base model,
deepseek-ai/DeepSeek-R1-Distill-Qwen-7B. The governor did not modify them. - Attached is
governor_state.pt, which records the governor configuration (anchor primes, threshold, cached anchor rows, gate statistics, and a SHA-256 signature of the prime-indexed embedding rows). VERIFICATION.txtrecords the run details.
What the governor does (and doesn't)
The governor evaluates a gate on each training step and pins six
prime-indexed embedding rows ([2, 3, 5, 7, 11, 13]) to their original
values. In the run that produced this repository, the optimizer step was a
no-op (STEP_GATED_NO_MUTATION), so the model was not trained or
fine-tuned.
As a result:
- The prime-indexed rows are unchanged (anchor signature matches).
- The full model is byte-for-byte the base model.
- This run does not demonstrate training, fine-tuning, or that any form of catastrophic forgetting was prevented, because the model was not modified. The repository name reflects the project's intent, not a measured property of this checkpoint.
Intended use
Research and experimentation with the governor tooling. For general use, loading the base model directly is equivalent and avoids downloading a duplicate copy of the weights.
Usage
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
REPO_ID = "frankmorales2020/deepseek-governed-no-amnesia"
tokenizer = AutoTokenizer.from_pretrained(REPO_ID, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
REPO_ID, torch_dtype=torch.float16, device_map="auto", trust_remote_code=True
)
model.eval()
prompt = "Explain why prime numbers are important in cryptography."
text = tokenizer.apply_chat_template(
[{"role": "user", "content": prompt}],
tokenize=False, add_generation_prompt=True,
)
enc = tokenizer(text, return_tensors="pt").to(model.device)
out = model.generate(
**enc,
max_new_tokens=512,
do_sample=True,
temperature=0.7,
top_p=0.9,
repetition_penalty=1.3,
no_repeat_ngram_size=3,
pad_token_id=tokenizer.eos_token_id,
)
print(tokenizer.decode(out[0][enc["input_ids"].shape[1]:], skip_special_tokens=True))
This is a DeepSeek R1 distill (reasoning) model. Feed prompts through the chat template, and expect a
<think> ... </think>reasoning trace before the final answer. To keep only the final answer, split the output on</think>.
Inspecting the governor artifact
from huggingface_hub import hf_hub_download
import torch
state = torch.load(
hf_hub_download(REPO_ID, "governor_state.pt"),
map_location="cpu", weights_only=False,
)
print(state["primes"], state["LAMBDA_12"], state["anchor_signature"], state["stats"])
License
Released under the MIT license, inherited from the base model.
Acknowledgements
Base model: DeepSeek-R1-Distill-Qwen-7B by DeepSeek-AI.