EnhancAR
EnhancAR is an autoregressive generative model of enhancer homology families, trained on 233,158,475 enhancers extracted from 241 vertebrate genomes. By "unrolling" homology families (enhancer sequences are sorted into sets of homology sequences, and input data is sequences concatenated to each other with a separator token delimiting different sequences), EnhancAR learns to generate new sequences that conserve the function of prompt sequences. We demonstrate that this can be used to design new enhancers "by example", which is particularly useful when the function of enhancers is not known a priori.
Model Details
- Developed by: Kevin K. Yang, Alex J. Lee, Andrew G. Duncan, Micaela E Consens, Lorin Crawford, Jennifer A. Mitchell, and Alan M. Moses
- Model type: Hybrid state-space-model transformer architecture with mixture-of-experts
- License: MIT
Model Sources
- Repository: https://github.com/microsoft/enhancar
- Data: https://zenodo.org/records/19052620
Uses
EnhancAR is a generative enhancer model that can be used for:
- Unconditional generation of enhancer sequences
- Conditional design of enhancer sequences from homologs
Getting Started with EnhancAR
Requirements:
- PyTorch: 2.7.1
- CUDA: 12.8 and above
pip install transformers==4.48.2 huggingface_hub
Example generation: The following shows an example for generating an unconditional sequence. To generate a sequence conditionally, replace the input_sequence with a string of the form: {homolog A/homolog B/.../homolog N}.
For a full example, please see the Google Colab
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, AutoConfig
config = AutoConfig.from_pretrained("aduncan94/EnhancAR-Sorted", trust_remote_code=True)
config.use_mamba_kernels = False
model = AutoModelForCausalLM.from_pretrained("aduncan94/EnhancAR-Sorted", trust_remote_code=True, config=config).to("cuda")
tokenizer = AutoTokenizer.from_pretrained("aduncan94/EnhancAR-Sorted", trust_remote_code=True)
input_sequence = "{"
inputs = tokenizer(input_sequence, return_tensors="pt", return_token_type_ids=False, add_special_tokens=False)
outputs = generate_sequence(inputs, model, tokenizer)
output = tokenizer.batch_decode(outputs, skip_special_tokens=False)[0]
print(output)
Technical Specifications
Compute Infrastructure
- The 170M-parameter model was trained on 8 MI300X GPUs
Citation
If you use the code, models, or results, please cite our preprint