This is openchat/openchat-3.5-0106, tuned with DPO on a tiny subset Nectar. Only 200 steps, so nowhere close to a full epoch.
Careful attention was paid to make sure the chat template was followed properly.
Summary of versions:
- 200 steps, no filtering on Nectar dataset, 5e-5 learning rate
- empty repo, failed training. ignore it
- 500 steps, no filtering on Nectar dataset, 5e-5 learning rate (same as 1 but with more steps)
- 500 steps, filtered dataset to only include multi-chat-turn examples, used 4th ranking response as the "rejected" instead of 3rd, filtered out "good_natured=False", 5e-5 learning rate
- 5000 steps (over a full epoch), filtered dataset to only include multi-chat-turn examples, used 4th ranking response as the "rejected" instead of 3rd, filtered out "good_natured=False", 5e-6 learning rate. Same as 0.4 but with 10x the steps, and 1/10th the learning rate
- 500 steps, filtered dataset to only include multi-chat-turn examples, used 4th ranking response as the "rejected" instead of 3rd, filtered out "good_natured=False", 5e-5 learning rate. Same as 0.5 but with 1/10th the steps, and 10x the learning rate
Open LLM Leaderboard Evaluation Results
Detailed results can be found here
| Metric | Value |
|---|---|
| Avg. | 69.94 |
| AI2 Reasoning Challenge (25-Shot) | 66.21 |
| HellaSwag (10-Shot) | 82.99 |
| MMLU (5-Shot) | 65.17 |
| TruthfulQA (0-shot) | 54.22 |
| Winogrande (5-shot) | 81.37 |
| GSM8k (5-shot) | 69.67 |