MuMath: Multi-perspective Data Augmentation for Mathematical Reasoning in Large Language Models

Introduction

We have amalgamated and further refined these strengths while broadening the scope of augmentation methods to construct a multi-perspective augmentation dataset for mathematics—termed MuMath (μ-Math) Dataset. Subsequently, we finetune LLaMA-2 on the MuMath dataset to derive the MuMath model.

The best results are bolded.

Augmentation Methods

Setup

We recommend using Conda to manage your environment. We use vLLM to accelerate inference. Run the following commands to setup your environment:

conda create -n mumath python=3.10 
conda activate mumath
cd MuMath-src 
pip install -r requirements.txt 

Training

We also open MuMath Dataset for the training stage.

To train a model, after specifying MODEL_PATH, SAVE_PATH, DATA_PATH, the conda environment and so on, run the following command:

# 7B or 13B 
bash train_7b.sh 

# 34B 
bash train_13b_70b.sh 

Inference and Evaluation

We provide scripts for inference and evaluation, which are called in train_7b.sh and train_13b_70b.sh as mentioned above.

python eval_gsm8k.py --model $SAVE_PATH --data_file ./data/test/GSM8K_test.jsonl 
python eval_math.py --model $SAVE_PATH --data_path ./data/test/MATH_test.jsonl 

Citation

Please cite the paper if you refer to our model, code, data or paper from MuMath.

@inproceedings{you-etal-2024-mumath,
    title = "{M}u{M}ath: Multi-perspective Data Augmentation for Mathematical Reasoning in Large Language Models",
    author = "You, Weihao  and Yin, Shuo  and Zhao, Xudong  and Ji, Zhilong  and Zhong, Guoqiang  and Bai, Jinfeng",
    booktitle = "Findings of the Association for Computational Linguistics: NAACL 2024",
    month = jun,
    year = "2024",
    pages = "2932--2958",
}

Credits

This project has adopted the MeteMath and MuggleMath.