This model is trained from XiaofengAlg/CareBot_Medical_multi-llama3-8b-base using BAAI/IndustryInstruction_Health-Medicine. To enhance the model's ability to follow medical instructions and better adapt to specific medical scenarios, we conduct supervised fine-tuning. This process involves using conversational-style data (comprising both queries and responses) to fine-tune the pretrained LLM. In the following sections, we explore the data construction and training methods.
Data Construction
Our SFT dataset comprises a diverse array of question types, including multiple-choice questions from medical exams, single-turn disease diagnoses, and multi-turn health consultations. It integrates data from seven publicly available sources: Chinese Medical Dialogue Data\footnote{https://github.com/Toyhom/Chinese-medical-dialogue-data}, Huatuo26M , MedDialog , ChatMed Consult Dataset , ChatDoctor , CMB\footnote{https://github.com/FreedomIntelligence/CMB}, and MedQA . We preserve portions of authentic doctor-patient conversations and augment the dataset by rewriting the remaining content. For these rewrites, we use real-world medical scenarios as prompts and generate responses via GPT-4. We believe this ensures the diversity of the SFT dataset, which can help the CareBot better adapt to different types of medical problems and patient situations, thereby improving its performance in a variety of scenarios.
evaluation
evaluation on benchmark is bellow.
gsb result with other medical LLMS
Citation
@inproceedings{zhao2025carebot,
title={CareBot: A Pioneering Full-Process Open-Source Medical Language Model},
author={Lulu Zhao and Weihao Zeng and Xiaofeng Shi and Hua Zhou},
booktitle={Proceedings of the AAAI Conference on Artificial Intelligence},
volume={39},
number={24},
pages={26039--26047},
year={2025},
doi={10.1609/aaai.v39i24.34799},
url={https://doi.org/10.1609/aaai.v39i24.34799}
}
Acknowledgements
This work is supported by the National Science and Technology Major Project (No. 2022ZD0116314).
本项目受新一代人工智能国家科技重大专项（No. 2022ZD0116314）支持。