Llama-3.2-3B-d0-base

meta-llama/Llama-3.2-3B with the benchmark's added tokens and a resized embedding matrix. This is not a fine-tuned model — it knows none of the suite's facts. It exists so the suite's LoRA adapters have a base to attach to.

What was changed

50 function tokens <B01>…<B50>. Answers are ordinary digit strings, so no answer tokens were added. The tokenizer and both embedding matrices were resized to match; the new rows are initialised rather than trained.

Retention perplexity is 7.177, which is the untuned reference the trained models in this suite are measured against.

Use

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base = AutoModelForCausalLM.from_pretrained("Lamsheeper/Llama-3.2-3B-d0-base")
tok  = AutoTokenizer.from_pretrained("Lamsheeper/Llama-3.2-3B-d0-base")

# one training-order replicate out of the suite archive
model = PeftModel.from_pretrained(base, "Lamsheeper/Llama-3.2-3B-d0-lora-seeds",
                                  subfolder="10d_sd1001")

For the published best-of-n model of each document count, use Lamsheeper/Llama-3.2-3B-d0-<n>doc directly — those are merged full weights and need neither this base nor peft.