Model Card for Model ID

Lt-Llama2 is a family of pretrained and fine-tuned generative text models for Lithuanian. This is the repository for the instruct 13B model. Links to other models can be found at the bottom of this page.

Model Details

Model Description

Neurotechnology company marks the first open-source initiative dedicated to developing a large language model (LLM) specialized in Lithuanian. The company has created and publicly released a collection of Lithuanian LLMs, available both as foundational models and instructional variants.

• Developed by: Neurotechnology

• Language(s): Lithuanian
• License: Llama2 Community License Agreement
• Finetuned from model: Lt-Llama-2-13b

Model Sources

• Paper: https://arxiv.org/abs/2408.12963

Intended Use

Intended Use Cases

Lt-Llama2 is designed for research purposes in Lithuanian. The base models can be tailored for various natural language tasks, while the instruction models are geared towards assistant-like conversational interactions.

Prohibited use

Utilizing the model in ways that breach the license, violate any applicable laws or regulations, or involve languages other than Lithuanian.

How to Get Started with the Model

Use the code below to get started with the model.

from transformers import AutoTokenizer, AutoModelForCausalLM
tokenizer = AutoTokenizer.from_pretrained("neurotechnology/Lt-Llama-2-13b-instruct-hf")
model = AutoModelForCausalLM.from_pretrained("neurotechnology/Lt-Llama-2-13b-instruct-hf")
PROMPT_TEMPLATE = (
        "[INST] <<SYS>> Esi paslaugus asistentas <</SYS>>{instruction}[/INST]"
    )

instruction ="Kas yra Lietuvos sostinė?"
prompt = PROMPT_TEMPLATE.format_map({'instruction':instruction})
inputs = tokenizer.encode(prompt, add_special_tokens=False, return_tensors="pt")
outputs = model.generate(input_ids=inputs, max_new_tokens=128)
print(tokenizer.decode(outputs[0]))

Lt-Llama2 Model Family

Citation

@misc{nakvosas2024openllama2modellithuanian,
      title={Open Llama2 Model for the Lithuanian Language},
      author={Artūras Nakvosas and Povilas Daniušis and Vytas Mulevičius},
      year={2024},
      eprint={2408.12963},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2408.12963},
}