axolotl version: 0.10.0

base_model: meta-llama/Llama-3.1-8B-Instruct
model_type: LlamaForCausalLM
tokenizer_type: AutoTokenizer

load_in_8bit: false
load_in_4bit: false

# ============================================================================
# Full FT on REAL Othello (Li et al.'s synthetic distribution), multi-format,
# GAME DATA ONLY — no chat replay.
#
# This is fullft-othello-8b-multi-format-replay.yaml with the replay dataset
# entry deleted and the slice scaled up. Exactly ONE variable differs from that
# run, so the two are directly comparable on valid-move rate.
#
# PURPOSE: a data-scaling probe, not a keeper checkpoint. It answers two things:
#   1. how many games does an 8B model need to reach ~100% valid moves (the 50k
#      multi-format+replay run reached only ~80%, against 99.7% for
#      Snake-Othello at the same 50k) — read off the checkpoint ladder below;
#   2. whether the chat replay was diluting the game signal — compare this run's
#      ~500k-game checkpoint (step ~5,680) against the 2:1-replay 500k run.
# EXPECT THIS MODEL TO COLLAPSE TO MOVES-ONLY. With zero replay the snake run
# stopped emitting anything but moves; that is fine for the completion eval
# (data/prepare_othello_eval_dataset.py) but this checkpoint is NOT usable for
# the chat-format eval or any verbalizer/probing work. Use the -replay runs for
# those.
# ============================================================================

# --- Dataset: continued pre-training with a masked constant prefix ---
# `segments`: "<|begin_of_text|>Game of bjk\n\n" is label:false (context, not in
# the loss); the formatted moves + <|end_of_text|> are label:true (trained).
#
# SCALE — 2,000,000 games of the 20,000,000-game train split. Measured on the
# real dataset (see CLAUDE.md § Othello (real) training math):
#   * ~186 Llama-3 tokens/game in the multi-format mix  ->  ~372M trained tokens
#   * effective batch = 4 GPUs * micro 4 * accum 1 = 16 packed sequences of 1024
#     = 16,384 tokens/step  ->  ~88 games/step  ->  ~22,700 steps
#   * at ~3-5 s/step that is ~19-32 h  (vs ~8,514 steps for the 500k+replay run)
# Dropping the replay is what makes this affordable: ~1.5x more games per hour.
#
# WHY 2M. The regime is bracketed: Li et al. get ~5% error on the ~130k-game
# championship set and ~0.01% at 20M games, and we are at ~20% error at 50k. So
# the answer lives somewhere in 1e5-1e7 games, and 2M is the largest point that
# fits in about a day. Error falls roughly log-linearly in data, so the useful
# unit is a MULTIPLIER: 2x is inside the eval's noise, 10x is a step. If the
# valid-move curve is still climbing at 2M, the next rung is 10x, not 2x.
#
# THE CHECKPOINT LADDER. num_epochs is 1, so every intermediate checkpoint IS a
# data-scale point — the checkpoint at step N has seen exactly N * 88 games:
#     step  2,000  ->   176k games
#     step  5,680  ->   500k games   <- compare against the 500k+replay run
#     step 11,350  ->     1M games
#     step 22,700  ->     2M games   (end)
# One run therefore buys the whole scaling curve for the price of its largest
# point. Caveat when reading it: the cosine LR is not annealed at intermediate
# checkpoints, so each rung slightly UNDERSTATES what a dedicated run at that
# scale would reach. The curve is a conservative floor — fine for deciding
# "keep scaling or stop", not for quoting a number at a given scale.
datasets:
  - path: cfierro/othello-llama3-multi-format
    type: input_output
    split: train[:2000000]
train_on_inputs: false          # REQUIRED for input_output masking to take effect
dataset_prepared_path: /projects/nlp/data/constanzam/learning-facts/axolotl/datasets/llama-3.1-8b/othello-multi-format-2m
# The -2m suffix is the game slice, and the missing -replay marks this as the
# game-only arm. Do not reuse the -replay run's names: that run is the baseline
# this one is measured against.
output_dir: /projects/nlp/data/constanzam/learning-facts/axolotl/models/llama-3.1-8b-fft-othello-multi-format-1e-5-2m

sequence_len: 1024              # games are ~127 tokens; long context is unused here
sample_packing: true            # pack many short games per sequence (block-diagonal attn)
eval_sample_packing: false

# --- Validation on held-out GAMES ---
# NOT the `test` split: that one is reserved for the actual evaluation
# (data/prepare_othello_eval_dataset.py expands its first 5,000 games into the
# valid-move eval sets). load_best_model_at_end selects a checkpoint on whatever
# is named here, so pointing it at `test` would be selecting on the test set.
# Validation comes from the far end of the train split instead: unseen as long as
# the training slice does not reach it, and a negative slice stays correct
# however far that slice is scaled up. val_set_size: 0 disables the random carve.
#
# NOTE: eval_loss here is NOT comparable to the -replay run's eval_loss — the
# training mixes differ. Compare the two runs on VALID-MOVE RATE only.
val_set_size: 0
test_datasets:
  - path: cfierro/othello-llama3-multi-format
    type: input_output
    split: train[-500:]

wandb_project: othello-ft
wandb_entity: cfierro
wandb_watch:
wandb_name: llama-3.1-8b-fft-othello-multi-format-1e-5-2m
wandb_log_model: "false"

# --- Multi-GPU: 4x A40 (48GB) per node ---
# Full FT of an 8B model needs ZeRO-3 to shard weights+grads+optimizer.
# Effective batch = 4 GPUs * micro 4 * grad_accum 1 = 16 packed sequences.
# Confirm the real step count with `axolotl preprocess ... --debug` before
# committing a ~1-day run.
gradient_accumulation_steps: 1
micro_batch_size: 4
num_epochs: 1                   # 20M unique games: a second pass is pure repetition

# --- Checkpointing: this is what produces the scaling ladder ---
# save_steps 2000 = a rung every ~176k games, ~11 rungs over the run.
#
# DISK / ROTATION WARNING. save_total_limit: 3 means rung K is DELETED once rung
# K+3 lands, so the early rungs are gone before they can be evaluated. Pick one:
#   (a) run the valid-move eval (prepare_othello_eval_dataset.py + the usual
#       inference/ run) on each checkpoint AS IT LANDS, or copy the model
#       weights out of the checkpoint dir, before rotation removes it;  <- default
#   (b) hub_strategy: all_checkpoints, to push every rung to the Hub;
#   (c) raise save_total_limit and pay ~60-100 GB per full checkpoint.
# Do NOT reach for save_only_model: true here — it drops the optimizer/scheduler
# state and makes the run unresumable, which matters given the wall clock (see
# the resume block at the bottom).
save_strategy: steps
save_steps: 2000
save_total_limit: 10
eval_strategy: steps            # must match save_strategy for load_best_model_at_end
eval_steps: 2000

load_best_model_at_end: true
metric_for_best_model: eval_loss
greater_is_better: false

# Needs a write-scoped HF token on the cluster (HF_TOKEN or `huggingface-cli login`).
hub_model_id: cfierro/llama-3.1-8b-fft-othello-multi-format-1e-5-2m
hub_strategy: end
hub_private_repo: false

optimizer: adamw_bnb_8bit
lr_scheduler: cosine
learning_rate: 1e-5             # matches the -replay runs, for comparability

bf16: auto
tf32: false

gradient_checkpointing: true
flash_attention: true
logging_steps: 1

warmup_ratio: 0.03              # ~680 steps here
save_total_limit: 3
weight_decay: 0.0
special_tokens:
   pad_token: <|end_of_text|>

# DeepSpeed ZeRO Stage 3 — shards weights, gradients, and optimizer across GPUs
deepspeed: deepspeed_configs/zero3.json

# --- Resuming after a wall-clock kill ---
# ~22,700 steps at 3-5 s/step can exceed a 24 h queue limit. The checkpoints
# written above are full training states (model + optimizer + LR scheduler + RNG
# + trainer step count) precisely so the run can pick up where it stopped, losing
# at most save_steps (2,000) steps of work.
#
# To resume manually, point this at the checkpoint directory and relaunch:
#   resume_from_checkpoint: /projects/nlp/data/constanzam/learning-facts/axolotl/models/llama-3.1-8b-fft-othello-multi-format-1e-5-2m/checkpoint-<N>
auto_resume_from_checkpoints: true
# Or set auto_resume_from_checkpoints: true (verify the flag on the CLUSTER's
# axolotl version, not locally) so a requeued job re-runs the identical command
# and picks the latest checkpoint in output_dir by itself — the right setting if
# the queue preempts jobs.
#
# Two things to leave alone across a resume, or the restart is not a continuation:
#   * dataset_prepared_path — the cached tokenized+packed dataset. Reusing it
#     avoids re-tokenizing 2M games AND keeps the sample order identical, which
#     is what makes "skip the first N batches" land on the right data.
#   * the slice, batch dims, GPU count and LR schedule — the cosine schedule is
#     defined over the total step count, so changing any of them mid-run silently
#     changes the LR curve.

# Verify the data + loss mask before training:
#   axolotl preprocess axolotl_configs/fullft-othello-8b-multi-format.yaml --debug
# Confirm the prefix tokens show -100 (masked) and the move tokens + <|end_of_text|>
# are trained, and that the player-labelled formats show non-alternating colours on
# the ~34% of games that contain a pass.

llama-3.1-8b-fft-othello-multi-format-1e-5-2m

This model is a fine-tuned version of meta-llama/Llama-3.1-8B-Instruct on the cfierro/othello-llama3-multi-format dataset. It achieves the following results on the evaluation set:

• Loss: 0.7410

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• distributed_type: multi-GPU
• num_devices: 4
• total_train_batch_size: 16
• total_eval_batch_size: 16
• optimizer: Use OptimizerNames.ADAMW_BNB with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_steps: 693
• training_steps: 23108

Training results

Framework versions

• Transformers 4.57.3
• Pytorch 2.9.0+cu128
• Datasets 3.6.0
• Tokenizers 0.22.2