CENO Base 1B (Preview)
Base CENO language model. This is a preview release.
CENO is derived from NVIDIA's Nemotron-H (Apache-2.0). The custom Transformers
remote code in this repository (configuration_ceno.py, modeling_ceno.py) is a
rename of the upstream Nemotron-H implementation; the model weights are unchanged.
This repository includes custom Transformers remote code for CENOForCausalLM
and CENOCharLevelTokenizer. Load with trust_remote_code=True.
Files
model.safetensors: model weightsconfig.json: model config withauto_mapgeneration_config.json: generation configconfiguration_ceno.py,modeling_ceno.py: custom model codeceno_tokenizer.py,tokenizer_config.json,special_tokens_map.json,vocab.json: tokenizer files
Loading
from transformers import AutoModelForCausalLM, AutoTokenizer
repo_id = "CladeTeam/CENO-base-1b-preview"
model = AutoModelForCausalLM.from_pretrained(repo_id, trust_remote_code=True)
tokenizer = AutoTokenizer.from_pretrained(repo_id, trust_remote_code=True)
The model code depends on PyTorch and the Mamba/Triton stack used by Nemotron-H.
The bundled config sets use_mamba_kernels=true; if mamba-ssm and
causal-conv1d are not installed, set config.use_mamba_kernels=False before
loading to fall back to the pure-PyTorch Mamba path.
Intended Use
A general-purpose DNA language model for sequence modeling tasks such as likelihood scoring, representation learning, and downstream finetuning on species- or task-specific DNA data.
License
This model and its bundled code are released under the Apache License 2.0, inheriting the license of the upstream Nemotron-H model code (Copyright 2024 AI21 Labs Ltd. and the HuggingFace Inc. team; Copyright (c) 2025 NVIDIA CORPORATION). Modifications for CENO by CladeTeam.