Schaapje-2B-Chat-SFT-V1.0

Model description

This is the SFT model based on the custom continual pretrained model Schaapje-2B-Pretrained.

General Dutch Chat and/or Instruction following works quitte well with this model.

Model usage

A basic example of how to use this SFT model for Chat or Instruction following.

Note: This model is still unaligned. If that is required for your usage scenario then please use: Schaapje-2B-Chat-V1.0

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

device = 'cuda'
model_name = 'robinsmits/Schaapje-2B-Chat-SFT-V1.0'

model = AutoModelForCausalLM.from_pretrained(model_name, 
                                             device_map = "auto", 
                                             torch_dtype = torch.bfloat16)

tokenizer = AutoTokenizer.from_pretrained(model_name)

messages = [{"role": "user", "content": "Hoi hoe gaat het ermee?"}]

chat = tokenizer.apply_chat_template(messages, 
                                     tokenize = False, 
                                     add_generation_prompt = True)

input_tokens = tokenizer(chat, return_tensors = "pt").to('cuda')

output = model.generate(**input_tokens, 
                        max_new_tokens = 512,
                        do_sample = True)

output = tokenizer.decode(output[0], skip_special_tokens = False)
print(output)

Intended uses & limitations

As with all LLM's this model can also experience bias and hallucinations. Regardless of how you use this model always perform the necessary testing and validation.

Datasets and Licenses

The following dataset was used for SFT:

• BramVanroy/ultrachat_200k_dutch: apache-2.0

Model Training

The notebook used to train this SFT model is available at the following link: Schaapje-2B-Chat-SFT-V1.0