Simple Use Case
This section demonstrates a simple use case of how to interact with our model to solve problems in a step-by-step, friendly manner.
Define the Function
We define a function get_completion which takes user input, combines it with a predefined system prompt, and then sends this combined prompt to our model. The model's response is then printed out.
Here's how the function is implemented:
import torch
from transformers import pipeline
import os
# Load model
test_pipeline = pipeline(model="zaursamedov1/FIxtral",
torch_dtype=torch.bfloat16,
trust_remote_code=True,
device_map="auto")
### Define the function
def get_completion(input):
system = "Think step by step and solve the problem in a friendly way."
prompt = f"#### System: {system}\\n#### User: \\n{input}\\n\\n#### Response from FIxtral model:"
print(prompt)
fixtral_prompt = test_pipeline(prompt, max_new_tokens=500)
return fixtral_prompt[0]["generated_text"]
# Let's prompt
prompt = "problem"
print(get_completion(prompt))