Ana-FraudDetection-1.2
Ana-FraudDetection-1.2 is a sequence classification model specifically designed to detect phishing and fraud attempts in texts (such as emails and SMS).
Model Details
- Base Model:
BSC-LT/mRoBERTa(trained from scratch/fine-tuned directly on this base) - Task: Phishing / Fraud Detection
- Labels:
0: Ham (Legitimate)1: Phishing (Fraudulent)
Evaluation Metrics
Based on the final validation set during training, the model achieves excellent performance:
- Accuracy: 97.25%
- F1 Score: 97.27%
- Precision: 98.12%
- Recall: 96.44%
Training & Architecture
- Origin: The model was trained from scratch starting from
BSC-LT/mRoBERTa(not a warm start from previous Ana versions). - Separator (Emails): When classifying emails, the subject and body must be separated by the
</s></s>token. - Head+Tail Tokenization: The model uses a max length of 512 tokens. It employs a head+tail tokenization strategy (typically taking 256 tokens from the beginning and 256 from the end of the text) to capture both sender identity and malicious links/call-to-actions at the bottom.
Usage Examples & Thresholds
Based on production calibration, the recommended decision thresholds for detecting Phishing are 0.95 for Emails and 0.90 for SMS (a slightly lower threshold increases sensitivity for SMS, since the model was trained predominantly on email structures).
1. Clasificando un Correo Electrónico (Email)
Debes concatenar el "Asunto" (Subject) y el "Cuerpo" (Body) usando el separador </s></s>. El umbral recomendado es 0.95.
from transformers import pipeline
classifier = pipeline("text-classification", model="Jmcc1976/Ana-FraudDetection-1.2", top_k=None)
# 1. Definir asunto, cuerpo y el separador
subject = "Urgente: Tu cuenta bancaria ha sido bloqueada"
body = "Estimado cliente, por motivos de seguridad hemos bloqueado tu cuenta. Haz clic en el siguiente enlace para verificar tu identidad: http://phishing-banco.com/login"
separator = " </s></s> "
# 2. Concatenar
email_text = f"{subject}{separator}{body}"
# 3. Clasificar
results = classifier(email_text)[0]
phishing_prob = next(res['score'] for res in results if res['label'] == 'LABEL_1')
# 4. Decisión basada en el umbral para Emails (0.95)
THRESHOLD_EMAIL = 0.95
is_phishing = phishing_prob >= THRESHOLD_EMAIL
print(f"Probabilidad de Phishing: {phishing_prob:.4f}")
print(f"¿Es Phishing? {is_phishing}")
2. Clasificando un SMS (Smishing)
Los mensajes SMS se procesan directamente como texto plano (sin separador). El umbral recomendado es 0.90.
from transformers import pipeline
classifier = pipeline("text-classification", model="Jmcc1976/Ana-FraudDetection-1.2", top_k=None)
# 1. Texto del SMS
sms_text = "Correos: Tu paquete no se ha podido entregar por falta de franqueo. Paga 1,99 EUR aqui: http://correos-pagos-online.com"
# 2. Clasificar
results = classifier(sms_text)[0]
phishing_prob = next(res['score'] for res in results if res['label'] == 'LABEL_1')
# 3. Decisión basada en el umbral para SMS (0.90)
THRESHOLD_SMS = 0.90
is_phishing = phishing_prob >= THRESHOLD_SMS
print(f"Probabilidad de Phishing: {phishing_prob:.4f}")
print(f"¿Es Phishing? {is_phishing}")