Clamifision-v1
Clamifision-v1 is a binary text classifier that determines whether input text is medical/health-related or not. It's built as a routing gate for BotMed, a family of ML/DL models for the medical field -- Clamifision decides whether incoming text should be routed to BotMed's specialized medical models.
Model Details
Model Description
Clamifision-v1 is a fine-tuned ModernBERT-base encoder with a binary sequence classification head, distinguishing medical/health-related text (label 1) from non-medical text (label 0). It was trained on a curated, balanced dataset aggregated from multiple public sources spanning both formal (PubMed, MedQuAD, clinical transcriptions, Wikipedia) and casual/conversational (real patient-asked questions, template-generated symptom queries) medical text, alongside a diverse non-medical corpus (news, product reviews, forum discussion, encyclopedia articles).
- Developed by: Bindupautra Jyotibrat
- Shared by: BJyotibrat
- Model type: Binary text classification (fine-tuned encoder-only transformer, sequence classification head, 2 labels)
- Language(s) (NLP): English
- License: GPL-3.0
- Finetuned from model: answerdotai/ModernBERT-base
Model Sources
- Repository: Jyotibrat/BotMed-Clamifision
- Demo: BotMed Official Website
Uses
Direct Use
Classifying a piece of input text (e.g. a user query) as medical/health-related or not, in isolation -- for example, as a content filter or pre-processing step in an application that needs to detect medical topics.
Downstream Use
Intended as a routing/gating component within the BotMed pipeline: text classified as medical is forwarded to BotMed's specialized downstream medical models; text classified as non-medical is not.
Out-of-Scope Use
Not intended for medical diagnosis, treatment recommendations, or any clinical decision-making -- it only classifies whether text is about medicine/health, it does not evaluate medical content for accuracy or safety. Not validated on languages other than English. Not intended as a sole safety filter for high-stakes applications without human review.
Bias, Risks, and Limitations
An earlier version of this model, trained only on formal sources (PubMed, MedQuAD, MTSamples, Wikipedia), learned to recognize formal/clinical writing style rather than the underlying "is this medical" concept -- it classified short, casual, first-person medical queries (e.g. "my chest hurts, should I worry?") as non-medical with high confidence, despite strong accuracy on formal, source-matched test data. Training data was subsequently expanded with real patient-asked questions and synthetic casual queries to address this, but formal sources (particularly PubMed) still make up the largest share of the medical training class by volume, so degraded performance on informal/conversational medical text remains a plausible residual risk.
Recommendations
Test this model specifically on short, casual, first-person, informally-phrased text before relying on it in production -- that is the failure mode most likely to still exist to some degree. Users (both direct and downstream) should be made aware of the risks, biases, and limitations described above.
How to Get Started with the Model
Use the code below to get started with the model.
from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch
tokenizer = AutoTokenizer.from_pretrained("BJyotibrat/Clamifision-v1")
model = AutoModelForSequenceClassification.from_pretrained("BJyotibrat/Clamifision-v1")
inputs = tokenizer("I've had a headache for 3 days", return_tensors="pt")
with torch.no_grad():
logits = model(**inputs).logits
pred = torch.argmax(logits, dim=-1).item()
print(model.config.id2label[pred])
Training Details
Training Data
~40,000 rows, balanced 50/50 medical/non-medical, curated via the BotMed Clamifision Data Pipeline. Sources:
- Medical: PubMed abstracts, MedQuAD, MTSamples clinical transcriptions, Wikipedia medical articles, real patient-asked questions (curaihealth/medical_questions_pairs), template-generated casual symptom queries
- Non-medical: AG News, 20 Newsgroups, Amazon/Yelp reviews, Wikipedia non-medical articles
Split 80/10/10 into train/val/test, stratified by label and source.
Training Procedure
Preprocessing
Tokenized with the ModernBERT tokenizer, truncated/padded to a max sequence length of 256 tokens.
Training Hyperparameters
- Training regime: fp16 mixed precision
- Learning rate: 2e-5
- Batch size: 16 (train), 32 (eval)
- Epochs: 3
- Weight decay: 0.01
- Attention implementation: SDPA
Evaluation
Testing Data, Factors & Metrics
Testing Data
Held-out stratified test split (10% of the curated dataset, ~4,258 rows), stratified by label and source.
Metrics
Accuracy, precision, recall, and F1 (binary classification).
Validation Results
Results
The following results are from an evaluation run on a version of this model trained before the casual-register training data (real patient-asked questions + synthetic casual queries) was added -- they should be re-confirmed against the final released checkpoint before being relied on.
| Metric | Score |
|---|---|
| Accuracy | 0.9988 |
| Precision | 0.9981 |
| Recall | 0.9995 |
| F1 | 0.9988 |
Confusion matrix (test set, n=4,258):
Summary
Very high accuracy on held-out test data drawn from the same sources as training, but this reflects the model's ability to recognize source-level writing style rather than confirmed generalization to real-world, casual, out-of-distribution medical queries -- see Bias, Risks, and Limitations above.
Environmental Impact
Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).
- Hardware Type: NVIDIA T4 GPU
- Hours used: 1 hour
- Cloud Provider: Google Colab
Technical Specifications
Model Architecture and Objective
ModernBERT-base encoder with a binary sequence classification head, fine-tuned for medical vs. non-medical text classification.
Compute Infrastructure
Hardware
NVIDIA T4 GPU (Google Colab free tier)
Software
PyTorch, Hugging Face transformers (Trainer API), datasets
Model Card Authors
- Bindupautra Jyotibrat (BJyotibrat)
Model Card Contact
Email: bjyotibrat@gmail.com