Model Card for Sum (3B) Small
Research article: Omi-Sum 3B: Open Clinical SOAP Model →
omi.health/research/omi-sum Built by Omi Health · Scribe product page · All
research
Model Description
Sum Small is a language model specifically designed to generate SOAP summaries from medical dialogues. It is a fine-tuned
version of Microsoft/Phi-3-mini-4k-instruct, trained on the
omi-health/medical-dialogue-to-soap-summary
dataset. It demonstrates competitive performance against larger closed models.
Full benchmark write-up, methodology, and reproducibility notes:
omi.health/research/omi-sum.
Intended Use
This model is intended for research and development in AI-powered medical documentation. It is not ready for direct
clinical use without further validation and should be integrated with additional safety guardrails before deployment in a
medical setting.
Training Data
The model was trained on Omi Health's synthetic medical-dialogue-to-soap-summary dataset, which
consists of 10,000 synthetically generated dialogues and corresponding SOAP summaries.
Training Procedure
Training was conducted on NVIDIA A100 GPUs.
Evaluation
The performance of Sum Small has been evaluated using ROUGE-1 as follows:
| Model | ROUGE-1 |
|---|---|
| Omi-Sum 3B Small | 70 |
| GPT-4 Turbo | 69 |
| Llama-3 8B Instruct | 59 |
| Phi-3 3B mini 4k instruct | 55 |
| GPT-3.5 | 54 |
| Phi-2 basic | 41 |
Full results, evaluation methodology, and safety analysis:
omi.health/research/omi-sum.
Limitations
While Sum Small demonstrates promising results, the training data is synthetic and not derived from actual clinical
interactions. Care must be taken when considering this model for practical applications, as it requires significant testing
and adaptation to meet clinical safety standards.
Licensing
Released under the MIT License — free for commercial and non-commercial use.
Citation
If you use this model or reference the benchmark, please cite:
APA — Omi Health. (2024). Omi-Sum 3B: Open-Source Model for Medical Summaries. https://omi.health/research/omi-sum
BibTeX
@misc{omi_sum_3b_2024,
title = {Omi-Sum 3B: Open-Source Model for Medical Summaries},
author = {{Omi Health}},
year = {2024},
url = {https://omi.health/research/omi-sum},
note = {3B parameter clinical SOAP model, fine-tuned from Phi-3, MIT licence}
}
Ethical Considerations
Users are urged to consider the ethical implications of AI in healthcare and ensure that any deployment of such models
prioritizes patient safety and data privacy.
Contact
For more information or to request access to Sum Small API, please contact hello@omi.health or
visit omi.health