theSOL1/llama-3.1-8b-oracle-rm-hh-rlhf-harmlessness
Reward model trained for the harmlessness preference on HH-RLHF harmlessness; original HH train only.
Model
- Type:
oracle - Base checkpoint:
meta-llama/Llama-3.1-8B - Label type:
hard - Architecture:
LlamaForSequenceClassification - Tokenizer:
PreTrainedTokenizerFast - Transformers compatibility:
transformers>=4,<5
Evaluation
Validation metrics are computed on the temporary eval split created by the training sweep.
| metric | value |
|---|---|
| Raw validation accuracy | n/a |
| Eval loss | n/a |
| Eval margin | n/a |
| Mean reward | n/a |
For proxy models, the validation labels are produced by the corresponding oracle reward model.
Loading
from transformers import AutoModelForSequenceClassification, AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained("theSOL1/llama-3.1-8b-oracle-rm-hh-rlhf-harmlessness")
model = AutoModelForSequenceClassification.from_pretrained("theSOL1/llama-3.1-8b-oracle-rm-hh-rlhf-harmlessness", num_labels=1)
This repository intentionally contains only model, tokenizer, and configuration files. Optimizer states, scheduler states, checkpoints, and trainer state files are not uploaded.