Socrates Qwen2.5 14B DPO
Usage
from transformers import AutoModelForCausalLM, AutoTokenizer
model_name = "socratesft/socrates-qwen2.5-14b-dpo"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(
model_name,
torch_dtype="auto",
device_map="auto"
)
# Example usage
prompt_system = "You are simulating a survey respondent. Answer exactly as instructed, following the specified response format without additional commentary."
prompt_user = """You are a survey respondent with the following demographic profile:
- Age: 31
- Gender: Male
- Education: Post grad study/professional degree
- Employment: Employed as paid employee
- Marital Status: Living with partner
- Housing Ownership: Rented for cash
- Housing Type: A one-family house detached from any other house
- Location: Kentucky
- Metro Status: Metro Area
- Income: 100-124K
- Internet Access: Internet Household
- Household Size: 2
- Phone Service: Cellphone only
Read the question below and answer exactly as this person would. Follow the response instructions precisely.
You read “There is a new halfway house opening in your neighborhood where recently released felons will live. The director is letting neighbors select applicants to live at the house, and you have the choice between the following two candidates:” Candidate 1: Sex: Male; Crime: Nonviolent burglary; Education: Vocational training; Race: Latino; Age: 35 years old; Previous work: Seasonal/part-time employment; Job seeking: Going to temp agencies; Family status: Divorced, no children. Candidate 2: Sex: Male; Crime: Nonviolent burglary; Education: GED; Race: Black; Age: 22 years old; Previous work: Steady full-time employment; Job seeking: Submitting resumes; Family status: Divorced, no children. You were then asked: “If you had to choose between them, which of the two candidates should be admitted to the halfway house in your neighborhood?” Only return 1 to choose the first candidate; 2 to choose the second candidate, nothing else."""
messages = [
{"role": "system", "content": prompt_system},
{"role": "user", "content": prompt_user}
]
text = tokenizer.apply_chat_template(
messages,
tokenize=False,
add_generation_prompt=True
)
inputs = tokenizer([text], return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=500)
generated_ids = [output_ids[len(input_ids):] for input_ids, output_ids in zip(inputs.input_ids, outputs)]
response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
print(response)
Training Details
- Base model: Qwen/Qwen2.5-14B-Instruct
- Training method: Direct Preference Optimization (DPO)
- Training dataset: socratesft/SocSci210 (
participant_mapping)
License
Built with Qwen 2.5. This model is a derivative of Qwen 2.5 and is subject to the Qwen LICENSE AGREEMENT.