MediAlbertina
The first publicly available medical language model trained with real European Portuguese data.
MediAlbertina is a family of encoders from the Bert family, DeBERTaV2-based, resulting from the continuation of the pre-training of PORTULAN's Albertina models with Electronic Medical Records shared by Portugal's largest public hospital.
Like its antecessors, MediAlbertina models are distributed under the MIT license.
Model Description
MediAlbertina PT-PT 1.5 NER was created through fine-tuning of MediAlbertina PT-PT 1.5B on real European Portuguese EMRs that have been hand-annotated for the following entities:
- Diagnostico (D): All types of diseases and conditions following the ICD-10-CM guidelines.
- Sintoma (S): Any complaints or evidence from healthcare professionals indicating that a patient is experiencing a medical condition.
- Medicamento (M): Something that is administrated to the patient (through any route), including drugs, specific food/drink, vitamins, or blood for transfusion.
- Dosagem (D): Dosage and frequency of medication administration.
- ProcedimentoMedico (PM): Anything healthcare professionals do related to patients, including exams, moving patients, administering something, or even surgeries.
- SinalVital (SV): Quantifiable indicators in a patient that can be measured, always associated with a specific result. Examples include cholesterol levels, diuresis, weight, or glycaemia.
- Resultado (R): Results can be associated with Medical Procedures and Vital Signs. It can be a numerical value if something was measured (e.g., the value associated with blood pressure) or a descriptor to indicate the result (e.g., positive/negative, functional).
- Progresso (P): Describes the progress of patient’s condition. Typically, it includes verbs like improving, evolving, or regressing and mentions to patient’s stability.
MediAlbertina PT-PT 1.5B NER achieved superior results to the same adaptation made on a non-medical Portuguese language model, demonstrating the effectiveness of this domain adaptation, and its potential for medical AI in Portugal.
| Checkpoints | Prec | Rec | F1 |
|---|---|---|---|
| Albertina PT-PT 900M | 0.814 | 0.814 | 0.813 |
| Albertina PT-PT 1.5B | 0.833 | 0.845 | 0.838 |
| MediAlbertina PT-PT900M | 0.84 | 0.828 | 0.832 |
| MediAlbertina PT-PT 1.5B | 0.842 | 0.845 | 0.843 |
Data
MediAlbertina PT-PT 1.5B NER was fine-tuned on about 10k hand-annotated medical entities from about 4k fully anonymized medical sentences from Portugal's largest public hospital. This data was acquired under the framework of the FCT project DSAIPA/AI/0122/2020 AIMHealth-Mobile Applications Based on Artificial Intelligence.
How to use
from transformers import pipeline
ner_pipeline = pipeline('ner', model='portugueseNLP/medialbertina_pt-pt_1.5b_NER', aggregation_strategy='average')
sentence = 'Durante o procedimento endoscópico, foram encontrados pólipos no cólon do paciente.'
entities = ner_pipeline(sentence)
for entity in entities:
print(f"{entity['entity_group']} - {sentence[entity['start']:entity['end']]}")
Citation
MediAlbertina is developed by a joint team from ISCTE-IUL, Portugal, and Select Data, CA USA. For a fully detailed description, check the respective publication:
@article{MediAlbertina PT-PT,
title={MediAlbertina: An European Portuguese medical language model},
author={Miguel Nunes and João Boné and João Ferreira
and Pedro Chaves and Luís Elvas},
year={2024},
journal={CBM},
volume={182}
url={https://doi.org/10.1016/j.compbiomed.2024.109233}
}
Please use the above cannonical reference when using or citing this model.
Acknowledgements
This work was financially supported by Project Blockchain.PT – Decentralize Portugal with Blockchain Agenda, (Project no 51), WP2, Call no 02/C05-i01.01/2022, funded by the Portuguese Recovery and Resillience Program (PRR), The Portuguese Republic and The European Union (EU) under the framework of Next Generation EU Program.