MAPO: Advancing Multilingual Reasoning through Multilingual Alignment-as-Preference Optimization
🔥Our paper
https://arxiv.org/abs/2401.06838
🔥Github Project
https://github.com/NJUNLP/MAPO
🔥Open Multilingual Reasoning Leaderboard
https://huggingface.co/spaces/kevinpro/Open-Multilingual-Reasoning-Leaderboard
Benchmarks
| System | MSVAMP | MGSM | MNumGLUESub |
|---|---|---|---|
| GPT-3.5-Turbo | 46.6 | 42.2 | 49.4 |
| MAmmoTH 7B | 26.3 | 21.3 | 24.2 |
| WizardMath 7B | 32.5 | 23.0 | 28.7 |
| MetaMath 7B | 46.2 | 37.0 | 43.2 |
| QAlign 7B | 57.2 | 49.6 | - |
| MathOctopus 7B | 41.2 | 39.5 | 37.1 |
| + MAPO-DPO(ours)🔥 | 57.4 | 41.6 | 50.4 |
| MetaMathOctopus 7B | 53.0 | 45.5 | 39.2 |
| + MAPO-DPO(ours) 👑 | 64.7 | 51.6 | 52.9 |
| MistralMathOctopus 7B | 59.0 | 58.0 | 56.8 |
| + MAPO-DPO(ours) 👑 | 74.6 | 67.3 | 70.0 |
| System | MSVAMP | MGSM | MNumGLUESub |
|---|---|---|---|
| GPT-3.5-Turbo | 46.6 | 42.2 | 49.4 |
| MAmmoTH 13B | 38.6 | 28.9 | 29.5 |
| WizardMath 13B | 35.7 | 28.3 | 29.0 |
| MetaMath 13B | 46.2 | 43.9 | 43.3 |
| QAlign 13B | 62.6 | 57.1 | - |
| MathOctopus 13B | 51.8 | 46.0 | 40.3 |
| + MAPO-DPO(ours)🔥 | 60.1 | 48.5 | 53.8 |
| MetaMathOctopus 13B | 56.3 | 51.4 | 49.5 |
| + MAPO-DPO(ours) 👑 | 67.0 | 58.0 | 59.8 |
Citation
If you find this model helpful, feel free to cite our paper:
@misc{she2024mapo,
title={MAPO: Advancing Multilingual Reasoning through Multilingual Alignment-as-Preference Optimization},
author={Shuaijie She and Wei Zou and Shujian Huang and Wenhao Zhu and Xiang Liu and Xiang Geng and Jiajun Chen},
year={2024},
eprint={2401.06838},
archivePrefix={arXiv},
primaryClass={cs.CL}
}