🧬 DNABERT Promoter Classifier

A fine-tuned DNABERT model for binary promoter sequence classification. Given a DNA sequence, the model predicts whether it belongs to a Promoter or Non-Promoter region.

Unlike conventional NLP models, DNABERT treats DNA as a language. During preprocessing, each DNA sequence is converted into overlapping 6-mers, enabling the Transformer to learn biologically meaningful sequence patterns.

This project demonstrates the application of Transformer-based deep learning for computational genomics and serves as part of my Bioinformatics and Machine Learning portfolio.

🚀 Overview

Promoters are regulatory DNA elements responsible for initiating gene transcription. Accurate promoter identification plays a critical role in:

• Genome annotation
• Gene regulation studies
• Functional genomics
• Bioinformatics research

This model fine-tunes DNABERT, a Transformer model pre-trained on genomic sequences represented as 6-mers.

🧬 Sequence Preprocessing

Before tokenization, every DNA sequence is converted into overlapping 6-mers, matching the representation used during DNABERT pretraining.

Example:

DNA sequence

ATGCGTACGA

6-mer representation

ATGCGT TGCGTA GCGTAC CGTACG GTACGA

The generated 6-mer sentence is then tokenized using the DNABERT tokenizer.

Important: The same preprocessing must be applied during inference. Raw DNA sequences should always be converted to 6-mers before being passed to the model.

👨‍💻 Author

Rahul Kumar Singh

M.Sc. Bioinformatics
Deen Dayal Upadhyaya Gorakhpur University

• GitHub: https://github.com/rahuls472
• Hugging Face: https://huggingface.co/rahuls472
• LinkedIn: https://www.linkedin.com/in/rahul-kumar-singh-1796b5332/

🎯 Classification Task

📊 Model Performance

The model was evaluated on the held-out test dataset.

📈 Classification Report

Overall Test Accuracy: 90.8%

📊 Confusion Matrix

📈 ROC Curve

ROC-AUC = 0.958

📉 Precision–Recall Curve

Average Precision = 0.973

⚙️ Training Configuration

💻 Example Usage

from transformers import AutoTokenizer
from transformers import AutoModelForSequenceClassification
import torch

def generate_kmers(sequence, k=6):
    return " ".join(
        sequence[i:i+k]
        for i in range(len(sequence)-k+1)
    )

model_name = "rahuls472/DNABERT-Promoter-Classifier"

tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSequenceClassification.from_pretrained(model_name)

sequence = "ATCGGCTAGCTAGCTAGCTAGCTA"

sequence = generate_kmers(sequence)

inputs = tokenizer(
    sequence,
    return_tensors="pt",
    truncation=True,
    padding="max_length",
    max_length=512
)

with torch.no_grad():
    outputs = model(**inputs)

prediction = torch.argmax(outputs.logits, dim=1).item()

print("Promoter" if prediction == 1 else "Non-Promoter")

🔬 Training Pipeline

DNA Sequence
      │
      ▼
Generate overlapping 6-mers
      │
      ▼
DNABERT Tokenizer
      │
      ▼
DNABERT Encoder
      │
      ▼
Sequence Classification Head
      │
      ▼
Promoter / Non-Promoter

📚 Applications

• Promoter prediction
• DNA sequence classification
• Genome annotation
• Functional genomics
• Computational biology research
• Educational projects

⚠️ Limitations

This model is intended for research and educational purposes only.

It is not intended for:

• Clinical diagnosis
• Medical decision making
• Disease prediction
• Variant interpretation

🚀 Future Work

• Multi-species promoter prediction
• Cross-validation experiments
• Hyperparameter optimization
• Multi-class genomic element classification
• Explainability using attention visualization
• Integration into genomics analysis pipelines

🙏 Acknowledgements

This project was developed using:

• Hugging Face Transformers
• PyTorch
• Google Colab
• DNABERT
• Genomic Benchmarks Dataset

Special thanks to the open-source bioinformatics and machine learning communities for providing the datasets, pretrained models, and tools that made this work possible.