MarinDNA m5.1 1B base model
MarinDNA m5.1 is a 1.12B-parameter, nucleotide-level causal language model developed with Marin. This is the final m5.1 base-model checkpoint at step 59,158 from run dna-bolinas-mix-v0.9-p1B-i24-exp135-zoonomia-m5.1-bef41e, released with the A 1B standard Transformer rivals Evo 2 40B on variant effect prediction blog post.
Model details
| Field | Value |
|---|---|
| Architecture | Qwen3ForCausalLM-compatible decoder-only Transformer |
| Parameters | 1,120,772,224 |
| Layers | 19 |
| Hidden / intermediate size | 1,920 / 7,680 |
| Attention heads / KV heads | 15 / 15 |
| Context | 256 tokens: one BOS token followed by up to 255 DNA bases |
| Checkpoint | Final m5.1 checkpoint, step 59,158 |
| Approximate token exposure | 166.0B nucleotide tokens over the inherited training lineage |
| Stored weight dtype | float32 |
| License | Apache-2.0 |
The source checkpoint is
gs://marin-us-east5/checkpoints/dna-bolinas-mix-v0.9-p1B-i24-exp135-zoonomia-m5.1-bef41e/hf/step-59158.
The commit-pinned training script defines the experiment.
Loading
from transformers import AutoModelForCausalLM, AutoTokenizer
repo_id = "marin-dna/marin-dna-exp135-m5.1"
tokenizer = AutoTokenizer.from_pretrained(repo_id)
model = AutoModelForCausalLM.from_pretrained(repo_id)
The float32 checkpoint is approximately 4.18 GiB. Convert or load it in a lower-precision dtype only if that is appropriate for your downstream use.
Tokenizer and input format
The bundled tokenizer is a case-insensitive, single-nucleotide tokenizer with seven tokens:
| Token | ID |
|---|---|
[PAD] | 0 |
[UNK] | 1 |
[BOS] | 2 |
a | 3 |
c | 4 |
g | 5 |
t | 6 |
Pass raw DNA strings containing A, C, G, and T without spaces or separators. The tokenizer lowercases input and automatically prepends [BOS]; the model has no EOS token and the tokenizer does not append one. Characters outside the four canonical bases map to [UNK]. Because BOS occupies one of the 256 model positions, inputs are limited to 255 DNA bases.
Training lineage and data
m5.1 is a continued-training lineage with approximately 166.0B total inherited and newly seen nucleotide tokens:
- approximately 42.0B inherited tokens from the pre-cooldown point of a uniform three-region mixture (CDS, upstream, and downstream);
- approximately 62.0B tokens from a continued uniform three-region mixture; and
- approximately 62.0B tokens from a uniform five-region mixture adding enhancer and ncRNA sequence.
The five-region inventory below describes the resources used across the m5.1 lineage. It does not mean that all five datasets contributed to every training phase.
Training datasets
Matched training-validation probes
These matched datasets were validation probes, not training data.