Model Card for Fireball-Mistral-Nemo-evol-Instruct-24B, fine tuned Mistral-Nemo-Instruct-2407 with merge
The EpistemeAI2's Fireball-Mistral-Nemo-Evol Instruct-24B , fine tuned Mistral-Nemo-Instruct-2407 Large Language Model (LLM) is an instruct fine-tuned version of the Mistral-Nemo-Base-2407. Trained jointly by Mistral AI and NVIDIA, it significantly outperforms existing models smaller or similar in size.
For more details about this model please refer to our release blog post.
Original Model Card
Key features
- Released under the Apache 2 License
- Pre-trained and instructed versions
- Trained with a 128k context window
- Trained on a large proportion of multilingual and code data
- Drop-in replacement of Mistral 7B
How to
Wizard (recommended)
plesee use Wizard prompt
f"""Below is an instruction that describes a task. \
Write a response that appropriately completes the request.
### Instruction:
{x['instruction']}
### Response:
"""
Model card from Merged Model
EpistemeAI/Fireball-Mistral-Nemo-Instruct-14B-merge-v1
EpistemeAI/Fireball-Mistral-Nemo-Instruct-14B-merge-v1 is a merge of the following models using LazyMergekit:
- EpistemeAI2/Fireball-Mistral-Nemo-Instruct-emo-PHD
- EpistemeAI2/Fireball-Mistral-Nemo-Instruct-emo-PHD
🧩 Configuration
slices:
- sources:
- model: EpistemeAI2/Fireball-Mistral-Nemo-Instruct-emo-PHD
layer_range: [0, 32]
- sources:
- model: EpistemeAI2/Fireball-Mistral-Nemo-Instruct-emo-PHD
layer_range: [24, 32]
merge_method: passthrough
dtype: bfloat16
💻 Usage
!pip install -qU transformers accelerate
from transformers import AutoTokenizer
import transformers
import torch
model = "EpistemeAI2/Fireball-Mistral-Nemo-evol-Instruct-14B"
messages = [{"role": "user", "content": "What is a large language model?"}]
tokenizer = AutoTokenizer.from_pretrained(model)
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
pipeline = transformers.pipeline(
"text-generation",
model=model,
torch_dtype=torch.float16,
device_map="auto",
)
outputs = pipeline(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)
print(outputs[0]["generated_text"])
Uploaded model
- Developed by: EpistemeAI2
- License: apache-2.0
- Finetuned from model : EpistemeAI/Fireball-Mistral-Nemo-Instruct-24B-merge-v1
This mistral model was trained 2x faster with Unsloth and Huggingface's TRL library.