ChickpeaLM-CDS v1.0
A chickpea-adapted genomic language model obtained by continued pretraining of PlantCAD2-Large on chickpea coding DNA sequences.
Model architecture
- Base: PlantCAD2-Large (Caduceus)
- Layers: 48
- Hidden dimension: 1,536
- Unique parameters: 694,385,664
- Reverse-complement parameter sharing: enabled
- Main training objective: masked nucleotide prediction
- Training: continued pretraining for three epochs
- Input: DNA sequences using A, C, G and T
Intended uses
- Extract representations of chickpea coding sequences
- Compare learned gene embeddings
- Study sequence variation through model perturbation scores
- Develop downstream supervised genomic models
These are research applications; downstream prediction tasks require appropriate validation.
Held-out validation
A chromosome-held-out validation set was used during model development.
| Metric | Original PlantCAD2 | ChickpeaLM-CDS |
|---|---|---|
| Masked-token accuracy | 60.403% | 65.991% |
| Masked-token loss | 0.89125 | 0.791363 |
These results are from the project-specific continued-pretraining evaluation, not a universal genomic benchmark.
Functional representation
Exploratory comparisons on chickpea gene embeddings found improvements in selected GO annotation retrieval metrics, especially Cellular Component Precision@10.
Important limitations:
- GO annotations in the benchmark were electronically inferred.
- Benchmarks used one held-out chromosome.
- The results do not establish experimentally verified function.
- Close and distant sequence homology can affect evaluations.
Sequence processing
The model was developed using CDS windows of up to 8,190 nucleotides. Longer CDS were divided into codon-aligned windows for analysis.
Usage
This model contains custom Caduceus/Mamba architecture code.
A compatible CUDA/PyTorch/Mamba environment and
trust_remote_code=True are required for the demonstrated
inference workflow.
See scripts/install_g4.py and scripts/load_chickpealm.py.
A complete public inference example will be added after fresh-environment deployment testing. Model loading must restore the reverse-complement parameter sharing used during training.
Base model and attribution
Derived from:
https://huggingface.co/kuleshov-group/PlantCAD2-Large-l48-d1536
Original PlantCAD2 model licence: Apache License 2.0.
ChickpeaLM-CDS includes adapted pretrained weights and Caduceus architecture code. Preserve applicable original attribution and licence notices when redistributing.
Limitations
This model has not been validated as a standalone predictor of phenotype, variant pathogenicity, gene function, or protein localisation.
Inference has been verified in the project's specialised GPU environment; portability to other systems requires additional testing.
FASTA gene embeddings (validated GPU environment)
ChickpeaLM-CDS v1.0 includes infer_fasta.py to extract
gene embeddings from chickpea coding DNA sequences.
Input
Provide a FASTA file containing one or more CDS sequences.
Requirements:
- Only A, C, G and T nucleotides are supported.
- Sequence lengths must be divisible by three.
- Each FASTA identifier must be unique.
- Longer sequences are processed in 8,190-nt windows.
Example
After downloading this repository and preparing a compatible Python/CUDA/Mamba environment, run:
python infer_fasta.py --model-dir ./ --fasta example.fasta --output-dir results
Output files
gene_embeddings.npy: NumPy matrix, N x 1536gene_embeddings.csv: GeneIDs and embedding valuesgene_metadata.csv: CDS length and window countgene_ids.json: GeneID order corresponding to matrix rows
Each gene is represented by a 1,536-dimensional embedding.
The script uses final-layer, reverse-complement-aware mean pooling. For multi-window CDS sequences, embeddings are combined using length-weighted averaging.
The exported embeddings are raw representations. They are not direct predictions of GO functions, cellular localisation, or phenotypes.
Hardware and software compatibility
The model and inference script have been verified with:
- NVIDIA RTX PRO 6000 Blackwell GPU (SM120)
- Python 3.13
- PyTorch 2.11.0 + CUDA 13.0
- Transformers 4.47.1
- Compatible compiled Mamba and causal-conv1d kernels
The existing scripts/install_g4.py targets Blackwell
SM120 and requires source archives from the original
deployment kit that are not included in this repository.
Standard Colab T4, L4, A100 and other hardware configurations have not yet been independently verified.
The model uses custom architecture code. Run only code obtained from repositories you trust.
Example data
example.fasta contains synthetic DNA sequences for
software testing only. It does not represent biological
validation data.
Citation and attribution
ChickpeaLM-CDS was developed by continued pretraining of PlantCAD2-Large on chickpea coding sequences.
Original model: https://huggingface.co/kuleshov-group/PlantCAD2-Large-l48-d1536
Please retain applicable model, software, licence and attribution notices.
Blackwell SM120 installation (validated)
ChickpeaLM-CDS v1.0 has been successfully installed and tested from a clean Python environment on an NVIDIA RTX PRO 6000 Blackwell Server Edition GPU.
Validated environment
- Python: 3.13.15
- PyTorch: 2.11.0+cu130
- CUDA toolkit: 13.0
- GPU architecture: SM120
- Transformers: 4.47.1
- Mamba-SSM: 2.2.4
- causal-conv1d: 1.5.0.post8
The Mamba and causal-conv1d packages require compatible compiled CUDA extensions. Their verified source archives, CUDA compatibility patches and third-party licences are included in this repository.
Installation procedure
First, download the complete model repository using Hugging Face Hub. While the repository is private, authentication is required.
Save the downloaded repository to a directory named ChickpeaLM_CDS_HF_Release_v1_0.
The following commands must be run on a compatible SM120 system that already provides the validated Python, PyTorch and CUDA toolchain.
From the model repository directory, run:
python3 scripts/install_blackwell_hf.py --check
If the prerequisite check passes, run:
python3 scripts/install_blackwell_hf.py
The installer builds the CUDA extensions and creates an isolated Python environment at:
/content/chickpealm-deploy-env
The installer may take some time because CUDA extensions are compiled from the bundled source archives.
Test the model with example FASTA
From the model repository directory, run:
/content/chickpealm-deploy-env/bin/python infer_fasta.py --model-dir . --fasta example.fasta --output-dir results
Expected outputs:
- results/gene_embeddings.npy
- results/gene_embeddings.csv
- results/gene_metadata.csv
- results/gene_ids.json
Each input CDS is represented by a 1,536-dimensional embedding.
Important compatibility limitations
This installer is specific to the validated Blackwell SM120 environment. It is not a universal GPU installer.
Installation and inference have not yet been independently validated on NVIDIA T4, L4, A100 or other GPU architectures.
The legacy scripts/install_g4.py and scripts/load_chickpealm.py originate from the original local deployment kit and assume its older directory layout. For this Hugging Face release, use scripts/install_blackwell_hf.py and infer_fasta.py instead.
The model uses custom Python architecture code and CUDA dependencies. Inspect and trust the repository code before running it.
Third-party attribution
See THIRD_PARTY_SOURCES.md and the third_party_licenses directory for source information and applicable licences.
Google Colab notebook (Blackwell SM120)
Open ChickpeaLM in Google Colab
The repository contains notebook.ipynb at its root
for Hugging Face's Google Colab integration. An identical
notebook is available in the examples directory.
What the notebook does
- Verifies the supported GPU and CUDA environment.
- Downloads the model from Hugging Face.
- Installs or verifies Mamba/CUDA dependencies.
- Accepts a chickpea CDS FASTA upload.
- Extracts 1,536-dimensional gene embeddings.
- Downloads a ZIP with CSV, NumPy and metadata files.
Hardware requirements
This notebook has been validated only on the NVIDIA RTX PRO 6000 Blackwell SM120 environment.
The validated configuration uses Python 3.13, PyTorch 2.11.0+cu130 and CUDA 13.0.
The model has not yet been validated on standard Colab T4, L4 or A100 configurations. The hardware preflight will stop execution on unsupported setups.
Validation status
The notebook's six code cells passed an automated workflow test with two synthetic CDS sequences. The test simulated the browser upload/download dialogs and reused a separately verified installation.
A clean installation and FASTA inference test was also completed independently on Blackwell SM120.
Access
While the Hugging Face repository is private, an authorised Hugging Face account is needed to download the model. After public release, model downloads will no longer require private-repository access.
ChickpeaLM research toolkit: zero-shot, similarity and GO
Zero-shot SNVs (1-based CDS coordinates, not genomic):
python zero_shot_variants.py --model-dir . --fasta examples/zero_shot_example.fasta --variants examples/zero_shot_example_variants.csv --output scores.csv
FASTA query similarity and candidate GO retrieval:
python fasta_gene_search.py --model-dir . --resource-dir resources --fasta examples/nonindexed_Ca1_query.fasta --output-dir results --top-k 10
Indexed gene lookup:
python gene_similarity_search.py --resource-dir resources --gene-id 101511505 --top-k 10 --output-prefix results/query
Reference scope: 1,836 Ca8 genes only.
GO annotations are all IEA, not experimentally validated.
Scores are not calibrated biological impact probabilities.
FASTA inference is validated on NVIDIA Blackwell SM120; other GPUs are unverified.