Model Card for cisco-ai/SecureBERT2.0-NER

The Secure Modern BERT NER Model is a fine-tuned transformer based on SecureBERT 2.0, designed for Named Entity Recognition (NER) in cybersecurity text.

It extracts domain-specific entities such as Indicators, Malware, Organizations, Systems, and Vulnerabilities from unstructured data sources like threat reports, incident analyses, advisories, and blogs.

NER in cybersecurity enables:

• Automated extraction of indicators of compromise (IOCs)
• Structuring of unstructured threat intelligence text
• Improved situational awareness for analysts
• Faster incident response and vulnerability triage

Model Details

Model Description

• Developed by: Cisco AI
• Model Type: ModernBertForTokenClassification
• Framework: TensorFlow / Transformers
• Tokenizer Type: PreTrainedTokenizerFast
• Number of Labels: 11
• Task: Named Entity Recognition (NER)
• License: Apache-2.0
• Language: English
• Base Model: cisco-ai/SecureBERT2.0

Supported Entity Labels

Model Configuration

Uses

Direct Use

• Named Entity Recognition (NER) on cybersecurity text
• Threat intelligence enrichment
• IOC extraction and normalization
• Incident report analysis
• Vulnerability mention detection

Downstream Use

This model can be integrated into:

• Threat intelligence platforms (TIPs)
• SOC automation tools
• Cybersecurity knowledge graphs
• Vulnerability management and CVE monitoring systems

Out-of-Scope Use

• Non-technical or general-domain NER tasks
• Generative or conversational AI applications

Benchmark Cybersecurity NER Corpus

Dataset Overview

How to Get Started with the Model

Example Usage (Transformers)

from transformers import AutoTokenizer, TFAutoModelForTokenClassification, pipeline

model_name = "cisco-ai/SecureBERT2.0-NER"

tokenizer = AutoTokenizer.from_pretrained(model_name)
model = TFAutoModelForTokenClassification.from_pretrained(model_name)

ner_pipeline = pipeline("ner", model=model, tokenizer=tokenizer)

text = "Stealc malware targets browser cookies and passwords."
entities = ner_pipeline(text)
print(entities)

Training Details

Training Objective and Procedure

The SecureBERT2.0-NER was fine-tuned for token-level classification on cybersecurity text using Cross Entropy Loss.
Training focused on accurately classifying entity boundaries and types across five cybersecurity-specific categories: Malware, Indicator, System, Organization, and Vulnerability.

The AdamW optimizer was used with a linear learning rate scheduler, and gradient clipping ensured stability during fine-tuning.

Training Configuration

Training Dataset

The model was fine-tuned on a cybersecurity-specific NER corpus, containing annotated threat intelligence reports, advisories, and technical documentation.

Preprocessing

• Texts were tokenized using the PreTrainedTokenizerFast tokenizer from SecureBERT 2.0.
• All sequences were truncated or padded to 1024 tokens.
• Labels were aligned with subword tokens to maintain token–label consistency.

Hardware and Training Setup

Optimization Summary

The model converged after approximately 20 epochs, with loss stabilizing at a low level.
Validation metrics (F1, precision, recall) showed steady improvement from epoch 3 onward, confirming effective domain-specific adaptation.

Evaluation

Testing Data, Factors & Metrics

Testing Data

Evaluation was conducted on a cybersecurity-specific NER benchmark corpus containing annotated threat reports, advisories, and incident analysis texts.
This benchmark includes five key entity types: Malware, Indicator, System, Organization, and Vulnerability.

Metrics

The following metrics were used to assess model performance:

• F1-score: Harmonic mean of precision and recall
• Recall: Measures how many true entities were correctly identified
• Precision: Measures how many predicted entities were correct

Results

Summary

The SecureBERT 2.0 NER model significantly outperforms both CyBERT and the original SecureBERT across all metrics.

• It achieves a F1-score of 0.945, a +21% absolute improvement over SecureBERT.
• Its recall (0.965) indicates excellent coverage of cybersecurity entities.
• Its precision (0.927) shows strong accuracy and low false-positive rates.

This demonstrates that domain-adaptive pretraining and fine-tuning on cybersecurity corpora dramatically improves NER performance compared to general or earlier models.

Reference

@article{aghaei2025securebert,
  title={SecureBERT 2.0: Advanced Language Model for Cybersecurity Intelligence},
  author={Aghaei, Ehsan and Jain, Sarthak and Arun, Prashanth and Sambamoorthy, Arjun},
  journal={arXiv preprint arXiv:2510.00240},
  year={2025}
}

Model Card Authors

Cisco AI

Model Card Contact

For inquiries, please contact ai-threat-intel@cisco.com