Small dummy deberta-v3-type Reward Model useable for Unit/Integration tests for RLHF. Suitable for CPU only machines, see H2O LLM Studio for an example integration test.
Model was created as follows:
from transformers import AutoConfig, AutoTokenizer, AutoModelForSequenceClassification
repo_name = "MaxJeblick/reward-model-deberta-v3-unit-test"
model_name = "OpenAssistant/reward-model-deberta-v3-large-v2"
config = AutoConfig.from_pretrained(model_name)
config.hidden_size = 12
config.intermediate_size = 24
config.num_attention_heads = 2
config.num_hidden_layers = 2
config.pooler_hidden_size = 12
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSequenceClassification.from_config(config)
print(model.num_parameters()) # 1_546_129
model.push_to_hub(repo_name, private=False)
tokenizer.push_to_hub(repo_name, private=False)
config.push_to_hub(repo_name, private=False)