Model Details
Model Description
- Developed by: Bruce_Wayne(The Batman)
- Model type: Text Generation
- Finetuned from model [optional]: OpenBioLLM(llama-3)(aaditya/Llama3-OpenBioLLM-8B)
You can find the gguf versions here --> https://huggingface.co/brucewayne0459/OpenBioLLm-Derm-gguf
please let me know how the model works -->https://forms.gle/N14zZTkLpUr6Hf4BA
Thank you!
Uses
Direct Use
This model is fine-tuned on skin diseases and dermatology data and is used for a dermatology chatbot to provide clear, accurate, and helpful information about various skin diseases, skin care routines, treatments, and related dermatological advice.
Bias, Risks, and Limitations
This model is trained on dermatology data, which might contain inherent biases. It is important to note that the model's responses should not be considered a substitute for professional medical advice. There may be limitations in understanding rare skin conditions or those not well-represented in the training data. The model still need to be fine-tuned further to get accurate answers.
Recommendations
Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
How to Get Started with the Model
Use the code below to get started with the model.
from transformers import AutoTokenizer, AutoModelForCausalLM
model_name = "brucewayne0459/OpenBioLLm-Derm"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)
Training Details
Training Data
The model is fine-tuned on a dataset containing information about various skin diseases and dermatology care. brucewayne0459/Skin_diseases_and_care
Training Procedure
Preprocessing [optional]
"""Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.
Instruction:
You are a highly knowledgeable and empathetic dermatologist. Provide clear, accurate, and helpful information about various skin diseases, skin care routines, treatments, and related dermatological advice.
Input:
{}
Response:
{} """ EOS_TOKEN = tokenizer.eos_token # Must add EOS_TOKEN
def formatting_prompts_func(examples): inputs = examples["Topic"] outputs = examples["Information"] texts = []
Prompt passed while fine tuning the model
Training Hyperparameters
Training regime: The model was trained using the following hyperparameters: Per device train batch size: 2 Gradient accumulation steps: 4 Warmup steps: 5 Max steps: 120 Learning rate: 2e-4 Optimizer: AdamW (8-bit) Weight decay: 0.01 LR scheduler type: Linear
Environmental Impact
Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).
- Hardware Type: Tesls T4 gpu
- Hours used: 1hr
- Cloud Provider: Google Colab
Technical Specifications [optional]
Model Architecture and Objective
This model is based on the LLaMA (Large Language Model Meta AI) architecture and fine-tuned to provide dermatological advice.
Hardware
The training was performed on Tesla T4 gpu with 4-bit quantization and gradient checkpointing to optimize memory usage.