Dr. Samantha

Overview

Dr. Samantha is a language model made by merging Severus27/BeingWell_llama2_7b and ParthasarathyShanmugam/llama-2-7b-samantha using mergekit.

Has capabilities of a medical knowledge-focused model (trained on USMLE databases and doctor-patient interactions) with the philosophical, psychological, and relational understanding of the Samantha-7b model.

As both a medical consultant and personal counselor, Dr.Samantha could effectively support both physical and mental wellbeing - important for whole-person care.

Yaml Config

slices:
  - sources:
      - model: Severus27/BeingWell_llama2_7b
        layer_range: [0, 32]
      - model: ParthasarathyShanmugam/llama-2-7b-samantha
        layer_range: [0, 32]

merge_method: slerp
base_model: TinyPixel/Llama-2-7B-bf16-sharded

parameters:
  t:
    - filter: self_attn
      value: [0, 0.5, 0.3, 0.7, 1]
    - filter: mlp
      value: [1, 0.5, 0.7, 0.3, 0]
    - value: 0.5 # fallback for rest of tensors
tokenizer_source: union

dtype: bfloat16

Prompt Template

Below is an instruction that describes a task. Write a response that appropriately completes the request.

### Instruction:
What is your name?

### Response:
My name is Samantha.

⚡ Quantized models

• GGUF:https://huggingface.co/TheBloke/Dr_Samantha-7B-GGUF
• GPTQ: https://huggingface.co/TheBloke/Dr_Samantha-7B-GPTQ
• AWQ: https://huggingface.co/TheBloke/Dr_Samantha-7B-AWQ

Thanks to TheBloke for making this available!

Dr.Samantha is now available on Ollama. You can use it by running the command ollama run stuehieyr/dr_samantha in your terminal. If you have limited computing resources, check out this video to learn how to run it on a Google Colab backend.

OpenLLM Leaderboard Performance

Subject-wise Accuracy

Evaluation by GPT-4 across 25 random prompts from ChatDoctor-200k Dataset

Overall Rating: 83.5/100

Pros:

• Demonstrates extensive medical knowledge through accurate identification of potential causes for various symptoms.
• Responses consistently emphasize the importance of seeking professional diagnoses and treatments.
• Advice to consult specialists for certain concerns is well-reasoned.
• Practical interim measures provided for symptom management in several cases.
• Consistent display of empathy, support, and reassurance for patients' well-being.
• Clear and understandable explanations of conditions and treatment options.
• Prompt responses addressing all aspects of medical inquiries.

Cons:

• Could occasionally place stronger emphasis on urgency when symptoms indicate potential emergencies.
• Discussion of differential diagnoses could explore a broader range of less common causes.
• Details around less common symptoms and their implications need more depth at times.
• Opportunities exist to gather clarifying details on symptom histories through follow-up questions.
• Consider exploring full medical histories to improve diagnostic context where relevant.
• Caution levels and risk factors associated with certain conditions could be underscored more.

Open LLM Leaderboard Evaluation Results

Detailed results can be found here