BulkFormer-37M
A foundation model for bulk RNA-seq transcriptomes. This is the smallest of
five released BulkFormer variants (37M / 50M / 93M / 127M / 147M
parameters), converted to a transformers-compatible format.
- Paper: Kang, Fan et al., Cell Systems 2026. DOI
- Original repo: KangBoming/BulkFormer
- HF-compatible package: zhfanrui/BulkFormer_hf
Usage
pip install bulkformer
from bulkformer import BulkFormerModel, extract_features
from bulkformer.preprocessing import normalize_data, align_to_vocab
model = BulkFormerModel.from_pretrained("zhfanrui/BulkFormer-37M")
# counts_df: [n_samples, n_genes] raw counts, columns = Ensembl gene IDs
log_tpm = normalize_data(counts_df)
aligned, mask_prob = align_to_vocab(log_tpm, model.config.gene_ids)
sample_emb = extract_features(
model, aligned.values, output_feature_type="sample_level",
aggregate_type="mean", mask_prob=mask_prob, device="cuda", batch_size=32,
)
See the package README for gene-level / expression-level extraction, fine-tuning (classification head, continued masked-imputation pretraining), and ESM2 embedding fusion.
This checkpoint bundles the pretrained gene-gene graph directly into
model.safetensors as non-trainable buffers -- no separate graph file
needed. config.json includes the exact gene vocabulary (gene_ids,
20,010 Ensembl IDs) this checkpoint expects, in order.
Citation
@article{kang2026bulkformer,
title = {BulkFormer: A large-scale foundation model for bulk transcriptomes},
author = {Kang, Boming and Fan, Rui and Yi, Meizheng and Cui, Chunmei and Cui, Qinghua},
journal = {Cell Systems},
year = {2026},
pages = {101657},
doi = {10.1016/j.cels.2026.101657}
}
License
MIT.