Gnerative Pretrained Tranformer with 3 Billion parameters for Norwegian. NorLlama-3B is based on Llama architechture, and pretrained on Tencent Pre-training Framework
It belongs to NorGLM, a suite of pretrained Norwegian Generative Language Models. NorGLM can be used for non-commercial purposes.
Datasets
All models in NorGLM are trained on 200G datasets, nearly 25B tokens, including Norwegian, Denish, Swedish, Germany and English.
Run the Model
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
model_id = "NorGLM/NorLlama-3B"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
model_id,
device_map='auto',
torch_dtype=torch.bfloat16
)
text = "Tom ønsket å gå på barene med venner"
inputs = tokenizer(text, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=20)
Citation Information
If you feel our work is helpful, please cite our papers:
@article{gulla2026norwai,
title={NorwAI's Large Language Models: Technical Report},
author={Gulla, Jon Atle and Liu, Peng and Zhang, Lemei},
journal={arXiv preprint arXiv:2601.03034},
year={2026}
}
@inproceedings{liu2024nlebench+,
title={NLEBench+NorGLM: A Comprehensive Empirical Analysis and Benchmark Dataset for Generative Language Models in Norwegian},
author={Liu, Peng and Zhang, Lemei and Farup, Terje and Lauvrak, Even and Ingvaldsen, Jon and Eide, Simen and Gulla, Jon Atle and Yang, Zhirong},
booktitle={Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing},
pages={5543--5560},
year={2024}
}