Nova: Generative Language Models for Assembly Code with Hierarchical Attention and Contrastive Learning

Model artifact for paper, Nova: Generative Language Models for Assembly Code with Hierarchical Attention and Contrastive Learning (ICLR 2025)

Citation

@inproceedings{nova,
    title = {{Nova: Generative Language Models for Assembly Code with Hierarchical Attention and Contrastive Learning}},
    author = {Jiang, Nan and Wang, Chengxiao and Liu, Kevin and Xu, Xiangzhe and Tan, Lin and Zhang, Xiangyu and Babkin, Petr},
    booktitle = {The Thirteenth International Conference on Learning Representations},
    year = {2025},
    url = {https://openreview.net/forum?id=4ytRL3HJrq}
}

Introduction of Nova

Nova is pre-trained with the language modeling objective starting from DeepSeek-Coder checkpoints, using the disassembly code from AnghaBench and C/C++ program compiled from The-Stack.

This is the repository of the foundation model of Nova, with 1.3B parameters. The other models in this series:

• Nova-1.3b-bcr: Nova-1.3b model further instruction-tuned for binary code recovery.
• Nova-6.7b: Foundation model for binary code with 6.7B parameters.
• Nova-6.7b-bcr: Nova-6.7b model further instruction-tuned for binary code recovery.

Environment

conda create -n nova python=3.10
conda activate nova
pip install -r requirements.txt

Or use a docker image:

docker pull jiang719/nova
docker run --gpus all -it jiang719/nova