Model Card for BERT AI Detector
Model Details
Model Description
This model is a fine-tuned BERT designed to classify text as either AI-generated or human-written. The model was trained on data from the Kaggle LLM Detect competition using variable-length text inputs ranging from 5 to 100 words. The fine-tuned model achieves high accuracy in identifying the source of the text, making it a valuable tool for detecting AI-generated content.
- Developed by: Pritam
- Language(s) (NLP): English
- License: Apache 2.0
- Finetuned from model: BERT (base-uncased)
Model Sources
- Repository: Hugging Face Model Card
- Demo: Streamlit Interface
Uses
Direct Use
The model is intended for use in detecting whether text is AI-generated or human-written. Users can input text snippets into the demo or directly integrate the model into their applications for automated content classification.
Downstream Use
Potential downstream uses include:
- Moderating AI-generated content in online platforms.
- Academic and journalistic content verification.
- Detecting plagiarism or misuse of AI writing tools.
Out-of-Scope Use
The model is not suitable for:
- Detecting deeply paraphrased AI-generated text.
- Analysis of languages other than English.
- Scenarios where fairness and bias considerations are critical, as those have not been explicitly addressed.
Bias, Risks, and Limitations
Recommendations
Users should be aware that:
- The model may not perform well on text heavily modified from AI-generated content.
- It may produce false positives or false negatives due to the inherent limitations of the dataset or model architecture.
How to Get Started with the Model
Use the following code snippet to load the model:
from transformers import AutoTokenizer, AutoModelForSequenceClassification
tokenizer = AutoTokenizer.from_pretrained("pritamdeb68/BERTAIDetector")
model = AutoModelForSequenceClassification.from_pretrained("pritamdeb68/BERTAIDetector")
text = "Your text here"
inputs = tokenizer(text, return_tensors="pt")
outputs = model(**inputs)
predictions = outputs.logits.argmax(dim=1).item()
print("AI-generated" if predictions == 1 else "Human-written")
Training Details
Training Data
The training dataset was sourced from the Kaggle LLM Detect competition. The data includes examples of both AI-generated and human-written text, spanning various input lengths (5-100 words).
Training Procedure
Preprocessing
- Text was tokenized using BERT's tokenizer.
- Input lengths ranged between 5 and 100 words, padded or truncated as necessary.
Training Hyperparameters
- Batch Size: 300
- Optimizer: AdamW
- Learning Rate: 1e-5
- Epochs: 1
Speeds, Sizes, Times
- Training Time: 1 hour 10 minutes
- Hardware Used: GPU (Kaggle T4 x 2)
- Loss: 0.12 on train data
Evaluation
Testing Data, Factors & Metrics
Testing Data
Validation data from the Kaggle competition was used for evaluation.
Metrics
- Accuracy: 96.65% on validation data.
Results
The model achieved high accuracy and low validation loss, demonstrating its effectiveness for the task of AI text detection.
Environmental Impact
Carbon emissions can be estimated using the Machine Learning Impact calculator:
- Hardware Type: Kaggle T4 (x2) GPU
- Training Duration: 1 hour 10 minutes
- Compute Region: Not specified
Technical Specifications
Model Architecture and Objective
- Model Architecture: BERT (base-uncased) fine-tuned for text classification.
- Objective: Binary classification of text into AI-generated or human-written categories.
Compute Infrastructure
Hardware
- Type: Kaggle T4(x2) GPU
Software
- Framework: PyTorch with Transformers library
Citation
If you use this model, please cite the repository:
@inproceedings{pritam2024bertaidetector,
title={BERT AI Detector},
author={Pritam},
year={2024},
url={https://huggingface.co/pritam2014/BERTAIDetector}
}