vibhorag101/roberta-base-suicide-prediction-phr-v2

This model is a fine-tuned version of roberta-base on Suicide Prediction Dataset, sourced from Reddit. It achieves the following results on the evaluation set:

• Loss: 0.0553
• Accuracy: 0.9869
• Recall: 0.9846
• Precision: 0.9904
• F1: 0.9875

Model description

This model is a finetune of roberta-base to detect suicidal tendencies in a given text.

Training and evaluation data

• The dataset is sourced from Reddit and is available on Kaggle.
• The dataset contains text with binary labels for suicide or non-suicide.
• The dataset was cleaned minimally, as BERT depends on contextually sensitive information, which can worsely effect its performance.
  • Removed numbers
  • Removed URLs, Emojis, and accented characters.
  • Remove any extra white spaces and any extra spaces after a single space.
  • Removed any consecutive characters repeated more than 3 times.
  • The rows with more than 512 BERT Tokens were removed, as they exceeded BERT's max token.
• The cleaned dataset can be found here
• The evaluation set had ~33k samples, while the training set had ~153k samples, i.e., a 70:15:15 (train:test:val) split.

Training procedure

• The model was trained on an RTXA5000 GPU.

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 32
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• weight_decay=0.1
• warmup_ratio: 0.06
• num_epochs: 3
• eval_steps: 500
• save_steps: 500
• Early Stopping:
  • early_stopping_patience: 5
  • early_stopping_threshold: 0.001
  • parameter: F1 Score

Training results

Framework versions

• Transformers 4.38.2
• Pytorch 2.1.0+cu121
• Datasets 2.18.0
• Tokenizers 0.15.0