Automated Peer Reviewing in Paper SEA: Standardization, Evaluation, and Analysis
Paper Link: https://arxiv.org/abs/2407.12857
Project Page: https://ecnu-sea.github.io/
🔥 News
- 🔥🔥🔥 SEA is accepted by EMNLP 2024 !
- 🔥🔥🔥 We have made SEA series models (7B) public !
Model Description
The SEA-E model utilizes Mistral-7B-Instruct-v0.2 as its backbone. It is derived by performing supervised fine-tuning (SFT) on a high-quality peer review instruction dataset, standardized through the SEA-S model. This model can provide comprehensive and insightful review feedback for submitted papers!
Review Paper With SEA-E
from transformers import AutoModelForCausalLM, AutoTokenizer
instruction = system_prompt_dict['instruction_e']
paper = read_txt_file(mmd_file_path)
idx = paper.find("## References")
paper = paper[:idx].strip()
model_name = "/root/sea/"
tokenizer = AutoTokenizer.from_pretrained(model_name)
chat_model = AutoModelForCausalLM.from_pretrained(model_name)
chat_model.to("cuda:0")
messages = [
{"role": "system", "content": instruction},
{"role": "user", "content": paper},
]
encodes = tokenizer.apply_chat_template(messages, return_tensors="pt")
encodes = encodes.to("cuda:0")
len_input = encodes.shape[1]
generated_ids = chat_model.generate(encodes,max_new_tokens=8192,do_sample=True)
# response = chat_model.chat(messages)[0].response_text
response = tokenizer.batch_decode(generated_ids[: , len_input:])[0]
The code provided above is an example. For detailed usage instructions, please refer to https://github.com/ecnu-sea/sea.
Additional Clauses
The additional clauses for this project are as follows:
- Commercial use is not allowed.
- The SEA-E model is intended solely to provide informative reviews for authors to polish their papers instead of directly recommending acceptance/rejection on papers.
- Currently, the SEA-E model is only applicable within the field of machine learning and does not guarantee insightful comments for other disciplines.
Citation
If you find our paper or models helpful, please consider cite as follows:
@inproceedings{yu2024automated,
title={Automated Peer Reviewing in Paper SEA: Standardization, Evaluation, and Analysis},
author={Yu, Jianxiang and Ding, Zichen and Tan, Jiaqi and Luo, Kangyang and Weng, Zhenmin and Gong, Chenghua and Zeng, Long and Cui, RenJing and Han, Chengcheng and Sun, Qiushi and others},
booktitle={Findings of the Association for Computational Linguistics: EMNLP 2024},
pages={10164--10184},
year={2024}
}