Llama EstLLM 70B 0826 Base

Please note that this is a base text completion model that has not been instruction-tuned. It is intended for fine-tuning on downstream tasks rather than direct use for chat or instruction-following.

meta-llama/Llama-3.1-70B continuously pre-trained on approximately 60B tokens.

Model Details

Model Description

• Developed by: TartuNLP and TalTechNLP research groups
• Funded by: Estonian Ministry of Education and Research, “Estonian Language Technology Program 2018-2027”
• Model type: Causal Language Model
• Language(s) (NLP): Estonian, English
• License: Llama 3.1 Community License Agreement
• Finetuned from model: meta-llama/Llama-3.1-70B

Logits-based

Estonian

English

Translation

Limitations

TODO

Citation

@misc{dorkin2026estllmenhancingestoniancapabilities,
      title={{EstLLM: Enhancing Estonian Capabilities in Multilingual LLMs via Continued Pretraining and Post-Training}}, 
      author={Aleksei Dorkin and Taido Purason and Emil Kalbaliyev and Hele-Andra Kuulmets and Marii Ojastu and Mark Fišel and Tanel Alumäe and Eleri Aedmaa and Krister Kruusmaa and Kairit Sirts},
      year={2026},
      eprint={2603.02041},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2603.02041}, 
}