Qwen3-8B-Base-CPT-BF16

Standalone BF16 (16-bit) continued-pretraining model. This revision contains the complete model from epoch 2, step 750, in exactly eight safetensors weight shards. Load directly with Transformers.

Epochs

main contains epoch 2, the best training-time validation checkpoint. Each completed epoch is independently loadable using its epoch-N revision. Both training runs stopped after four epochs; there is no epoch-five export.

These losses were measured during adapter training, not by a new evaluation of the BF16 exports. Evaluate the four revisions separately to measure behavior changes across epochs.

Load a specific epoch

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "CompassioninMachineLearning/Qwen3-8B-Base-CPT-BF16"
revision = "main"
tokenizer = AutoTokenizer.from_pretrained(model_id, revision=revision)
model = AutoModelForCausalLM.from_pretrained(
    model_id, revision=revision, dtype=torch.bfloat16, device_map="auto"
)

Merge provenance and validation

• Original base: Qwen/Qwen3-8B-Base at 49e3418fbbbca6ecbdf9608b4d22e5a407081db4.
• Source adapter: checkpoint 750.
• Adapter repository revision: 1586e45fc7c3f7d5e601dc454ba7e15884a2f1df.
• Adapter SHA-256: 62933de6ac6e2484a5082afc7d4748c2267368cc7a3331fd874b93f4fe0e609b.
• Merged 252 rsLoRA layers with PEFT's safe merge.
• The separately trained embed_tokens and lm_head matrices are included.

Training used a 4-bit base; these exports merge the trained adapter into the pinned original BF16 base. The merge was performed using PEFT's safe merge, not Unsloth's export helper. Every tensor was checked for finite BF16 values and the complete architecture's names and shapes. Every epoch passed a standalone load, finite-logit forward pass and short greedy generation. No separate adapter is required, and optimizer/trainer checkpoint files are not included.

merge_manifest.json records source and output checksums. validation.json records the inference smoke test. training_manifest.json preserves the training settings. Shard count controls packaging; BF16 specifies the 16-bit precision.