sgym-marketing-1.5b
A Qwen2.5-1.5B-Instruct model fine-tuned with Direct Preference Optimization (DPO) for marketing skill applications. Part of the SkillGym project for training LLMs to apply domain skills via reinforcement learning.
Model Description
This model is the result of the SkillGym training pipeline, which trains LLMs to apply domain-specific marketing skills. The base Qwen2.5-1.5B-Instruct model was fine-tuned using DPO on preference pairs where the "chosen" response follows a structured marketing skill template and the "rejected" response is a generic unguided output.
The model covers three marketing skill domains: CRO (Conversion Rate Optimization), Copywriting, and Cold Email outreach.
Training Details
- Base model: Qwen/Qwen2.5-1.5B-Instruct
- Method: Direct Preference Optimization (DPO), reference-free
- LoRA config: r=4, alpha=8, dropout=0.2
- Target modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
- DPO pairs: 17 (from 3 marketing skills: CRO, copywriting, cold-email)
- Epochs: 10
- Beta: 0.05
- Training time: ~4 minutes on NVIDIA RTX 4050
Evaluation Results
Evaluated on 30 marketing tasks using a 6-dimension composite scoring system:
| Dimension | Weight | Base Model | DPO Model | Delta |
|---|---|---|---|---|
| Criteria Pass Rate | 0.25 | - | - | +0.013 |
| Specificity | 0.20 | - | - | +0.006 |
| Actionability | 0.20 | - | - | -0.008 |
| Structure | 0.15 | - | - | -0.005 |
| Depth | 0.10 | - | - | -0.005 |
| Voice | 0.10 | - | - | +0.054 |
| Composite | 1.00 | 0.311 | 0.318 | +0.007 (+2.2%) |
- 17/30 tasks improved
- Per-skill: copywriting -0.021, cold-email +0.021, CRO +0.020
- Best improvements: copy_007 (+0.180), cro_008 (+0.171), cro_009 (+0.126)
Skills Covered
- CRO (Conversion Rate Optimization): Landing page analysis with heuristic evaluation, A/B test prioritization
- Copywriting: Website copy creation using PAS/AIDA frameworks, brand voice matching
- Cold Email: B2B outreach sequences with personalization, objection handling
Usage
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained("4MLabs/sgym-marketing-1.5b")
tokenizer = AutoTokenizer.from_pretrained("4MLabs/sgym-marketing-1.5b")
messages = [
{"role": "system", "content": "You are a marketing expert specializing in CRO."},
{"role": "user", "content": "Analyze this landing page for conversion optimization opportunities: [landing page URL or description]"}
]
inputs = tokenizer.apply_chat_template(messages, return_tensors="pt")
outputs = model.generate(inputs, max_new_tokens=1024)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
Citation
If you use this model, please cite:
@article{skillgym2026,
title={SkillGym: Training LLMs to Apply Domain Skills via Reinforcement Learning},
author={Michiel Horstman},
year={2026}
}
License
Apache 2.0
Acknowledgments
Built on Qwen2.5-1.5B-Instruct by Alibaba Cloud.