PulseMind-72B - Multimodal Large Language Model for Real-World Clinical Diagnosis

BIG NEWS: PulseMind-72B is released for real-world multi-turn clinical diagnosis with state-of-the-art performance on diagnostic consultation benchmarks.

This repository contains the PulseMind-72B model from the paper PulseMind: A Multi-Modal Medical Model for Real-World Clinical Diagnosis.

Highlights

• Real-world clinical diagnosis focus: Designed for multi-turn diagnostic consultation where models must integrate medical images + textual clinical context and maintain evolving patient–physician interaction.
• PulseMind Benchmark: Evaluated using a multi-turn diagnostic consultation benchmark.
• MediScope dataset: Trained and studied with MediScope, a large-scale multimodal clinical diagnostic dataset consisting of 98,000 real-world multi-turn consultations and 601,500 medical images spanning 10+ major clinical departments and 200+ sub-specialties.
• Strong overall performance: Demonstrates competitive results on both the diagnostic consultation benchmark and public medical benchmarks (see paper for full results).

Release

• Technical report:
  • arXiv: PulseMind: A Multi-Modal Medical Model for Real-World Clinical Diagnosis
• Model weights:
  • PulseMind-72B

Note on data & checkpoints: Due to size and privacy considerations, datasets and some checkpoints may be hosted externally. Please refer to the HuggingFace model card / GitHub repository for official download instructions and evaluation scripts.

Disclaimer

Disclaimer: Even though the weights, codes, and demos are released openly (similar to other pre-trained models), and despite best efforts in safety evaluation and alignment, PulseMind-72B may generate inaccurate, misleading, or potentially harmful medical content. It is intended for research and assistive use only and must not be used as a substitute for professional medical advice, diagnosis, or treatment. Developers and stakeholders should conduct their own red-teaming, deploy appropriate safeguards, and comply with all applicable laws and regulations. In no event shall the authors be held liable for any claim, damages, or other liability arising from the use of the released weights, codes, or demos.

Evaluation

Clinical Consultation Dialogue Benchmark(PulseMind Benchmark)

Medical Multimodal VQA

Medical Textual QA

Usage

from vllm import LLM, SamplingParams
from transformers import AutoProcessor
from qwen_vl_utils import process_vision_info
import PIL.Image as Image

MODEL_ID = "AQ-MedAI/PulseMind-72B"

# Load processor
processor = AutoProcessor.from_pretrained(MODEL_ID)

# Load vLLM engine
llm = LLM(
    model=MODEL_ID,
    limit_mm_per_prompt={"image": 4},
    tensor_parallel_size=2,
    enforce_eager=True,
    trust_remote_code=True,
)

sampling_params = SamplingParams(
    temperature=0.1,
    top_k=1,
    top_p=0.001,
    repetition_penalty=1.05,
    max_tokens=2048,
    stop_token_ids=[],
)

# Example input
image = Image.open("example.png")
text = "Describe the image and provide relevant clinical observations."

messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "image": image},
            {"type": "text", "text": text},
        ],
    }
]

# Build prompt & multimodal inputs
prompt = processor.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
)
image_inputs, video_inputs = process_vision_info(messages)

mm_data = {}
if image_inputs is not None:
    mm_data["image"] = image_inputs
if video_inputs is not None:
    mm_data["video"] = video_inputs

outputs = llm.generate(
    [{"prompt": prompt, "multi_modal_data": mm_data}],
    sampling_params=sampling_params,
)

print(outputs[0].outputs[0].text)

Evaluation Scripts (Full Paths) For complete evaluation pipelines, please refer to:

Citation

If you find our project useful, we hope you would kindly star our repo and cite our work as follows:

@article{xu2026pulsemind,
  title={PulseMind: A Multi-Modal Medical Model for Real-World Clinical Diagnosis},
  author={Xu, Jiao and Liu, Junwei and Lao, Jiangwei and Zhu, Qi and Zhao, Yunpeng and Jin, Congyun and Liu, Shinan and Lu, Zhihong and Zhang, Lihe and Chen, Xin and others},
  journal={arXiv preprint arXiv:2601.07344},
  year={2026}
}