ClinicaQwen-MedQA (1.5B)
ClinicaQwen-MedQA is a lightweight, hyper-specialized medical question-answering assistant. It is built upon the robust Qwen/Qwen2.5-1.5B architecture and has undergone a rigorous dual-stage training pipeline to eliminate hallucinations and maximize clinical accuracy while remaining deployable on low-VRAM edge hardware.
🌟 Key Features
- Dual-Stage Optimized: Trained sequentially for deep medical terminology comprehension and strict conversational QA alignment.
- Completion-Loss Masked: Fine-tuned using masked training targets to eliminate prompt mirroring ("echo bugs").
- Ultra-Efficient: Runs seamlessly in native
bfloat16precision on consumer or serverless GPU instances (e.g., RunPod, vLLM).
📊 Training & Evaluation Performance
Stage 1: Medical Knowledge Injection
- Dataset:
gamino/wiki_medical_terms - Final Evaluation Loss: 2.0314
- Perplexity: 7.63
Stage 2: Conversational QA Alignment
- Dataset:
medalpaca/medical_meadow_medical_flashcards - Final Evaluation Loss: 0.9755
- Perplexity: 2.65
🚀 How to Use (Python Example)
You can load and query ClinicaQwen-MedQA directly using the Hugging Face transformers library. Ensure your inference script uses low temperatures for maximum medical accuracy.
Installation
pip install torch transformers acceleration safetensors
Inference Code
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "Mix80/ClinicaQwen-MedQA"
print("Loading ClinicaQwen-MedQA...")
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
model_id,
torch_dtype=torch.bfloat16,
device_map="auto"
)
def ask_clinica_qwen(question, max_new_tokens=150):
# Construct the strict conversational template
prompt = f"User: {question}\nBot:"
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(
**inputs,
max_new_tokens=max_new_tokens,
temperature=0.1, # Crucial: Low temp ensures strict factual adherence
top_p=0.85,
do_sample=True,
repetition_penalty=1.3, # Prevents linguistic loops
no_repeat_ngram_size=3,
pad_token_id=tokenizer.eos_token_id
)
# Extract only the generated answer tokens, skipping the prompt prefix
prompt_length = inputs.input_ids.shape[1]
generated_tokens = outputs[0][prompt_length:]
answer = tokenizer.decode(generated_tokens, skip_special_tokens=True).strip()
return answer
# Test the model
sample_query = "What are the key differences between a tension headache and a migraine?"
response = ask_clinica_qwen(sample_query)
print(f"\nQuestion: {sample_query}")
print(f"ClinicaQwen: {response}")
⚠️ Disclaimer
ClinicaQwen-MedQA is an AI research model developed for educational and experimental data extraction purposes. It is not a licensed medical tool and should not be used as a substitute for professional clinical diagnosis, advice, or treatment workflows.