DNT-650M_POST-CPL-SMP
A 650M-parameter diploid model based on NTv3 650M POST. It was continually pretrained on 1,000 phased 1000 Genomes samples using 8,192-token windows for 212,000 steps, with masked language modeling (MLM), Contrastive Phase Loss (CPL) and Supervised Masked Phase (SMP).
Quickstart
import os
import sys
import torch
from huggingface_hub import hf_hub_download
from transformers import AutoModel, AutoTokenizer
repo = "scrc-dnai/DNT-650M_POST-CPL-SMP"
tokens_tsv = hf_hub_download(repo, "diploid_tokens.tsv")
encoder_dir = os.path.dirname(hf_hub_download(repo, "diploid_encode.py"))
sys.path.insert(0, encoder_dir)
import diploid_encode as de
device = "cuda" if torch.cuda.is_available() else "cpu"
tokenizer = AutoTokenizer.from_pretrained(repo, trust_remote_code=True)
model = AutoModel.from_pretrained(repo, trust_remote_code=True).eval().to(device)
# Two reference-aligned haplotypes; one phased C/G variant.
hap0 = "ACGTACGTAC" * 51
hap1 = hap0[:5] + "G" + hap0[6:]
sequence, offsets = de.encode_diploid(
hap0, hap1, tokens_tsv=tokens_tsv, return_offsets=True
)
batch = tokenizer(sequence, return_tensors="pt").to(device)
species_ids = model.encode_species("human").to(device)
with torch.no_grad():
output = model(**batch, species_ids=species_ids, output_hidden_states=True)
# Mean embedding over genomic positions; excludes inserted marker tokens.
token_embeddings = output.hidden_states[-1][0]
position_indices = [index for index in offsets if index >= 0]
sequence_embedding = token_embeddings[position_indices].mean(dim=0)
print(sequence_embedding.shape) # (1536,)
Mean-pooling results
| Benchmark | Readout | Result |
|---|---|---|
| ClinVar indels | Zero-shot L2 distance between mean embeddings; diploid / haploid / biallelic | AUROC 0.662 / 0.545 / 0.217 |
| GUE promoter / splice / TF | Mean embeddings with train/dev/test ridge probes | MCC 0.829 / 0.519 / 0.606 |
Input and licensing
Encode reference-aligned haplotype pairs with the included diploid_encode.py; the model expects token lengths divisible by 128. For batched POST-model inference, pass one species_ids value per sequence (for example, model.encode_species(["human"] * batch_size)).
These weights inherit the non-commercial terms of the InstaDeep Open Model Licence v1.0. Research use only; no commercial use or serving outputs as a service. See LICENSE.
Citation
Leib G et al. DNT: Diploid Genomic Foundation Model. bioRxiv (2026). doi:10.64898/2026.09.05.749576.