Model Introduction
The Index-1.9B-Character model is a role-playing model with 1.9 billion parameters in the large language model series independently developed by the Index team. Users can customize the few shots role by uploading the role dialogue materials that meet the requirements.
For usage and related codes, please see Index-1.9B/roleplay
For more details, see our GitHub and Index-1.9B Technical Report
The Index-1.9B-Character model is evaluated using the authoritative benchmark CharacterEval in the role-playing field. The evaluation set evaluates multiple granularities from the perspectives of character consistency, dialogue ability, and role-playing attractiveness. Our 1.9b model ranks ninth in overall average score, significantly better than other models of the same level.
| Model | Average score | character consistency | dialogue ability | role-playing attractiveness |
|---|---|---|---|---|
| CharacterYuyan-RLHF | 3.36 | 2.856 | 4.037 | 3.188 |
| CharacterYuyan | 3.309 | 2.88 | 3.931 | 3.117 |
| BC-NPC-Turbo | 3.169 | 2.764 | 3.798 | 2.946 |
| MiniMax | 3.135 | 2.718 | 3.784 | 2.902 |
| Baichuan2-13B | 3.129 | 2.701 | 3.795 | 2.89 |
| InternLM-20B | 3.124 | 2.715 | 3.745 | 2.911 |
| Baichuan2-7B | 3.11 | 2.7 | 3.757 | 2.874 |
| Index-1.9B-character | 3.109 | 2.676 | 3.789 | 2.862 |
| InternLM-7B | 3.034 | 2.62 | 3.698 | 2.784 |
| XVERSE-13B | 3.022 | 2.632 | 3.605 | 2.83 |
| Qwen-14B | 3.016 | 2.649 | 3.542 | 2.858 |
| GPT-4 | 3.006 | 2.697 | 3.448 | 2.873 |
| Xingchen | 2.991 | 2.595 | 3.646 | 2.732 |
| XVERSE-7B | 2.963 | 2.564 | 3.554 | 2.772 |
| CharacterGLM | 2.937 | 2.493 | 3.623 | 2.695 |
| ChatGLM3-6B | 2.898 | 2.556 | 3.399 | 2.739 |
| Qwen-7B | 2.849 | 2.54 | 3.327 | 2.679 |
| GPT-3.5 | 2.381 | 2.101 | 2.749 | 2.293 |