Buzz-8b-Large: Advancing Efficiency through Iterative Fine-Tuning

Introduction

• Alignment Lab AI is pleased to introduce our latest research efforts with:

Buzz-8b-Large, a state-of-the-art language model developed in collaboration with Hive Digital Technologies.

The Buzz model, Dataset, and Code are to be released to build a toolkit that aims to demonstrate the potential for reuse and optimization of existing pretrained language models to continuously refine the heights of performance that can be achieved with optimal use of FlOps. Alongside Buzz-5b-Medium, we release

• The Buzz Dataset
• Buzz-2.5b-Small
• Buzz-5b-Medium
• Buzz-8B-Large

we release this, the Buzz dataset and over the next few days, two additional models: Buzz-3B-Small and Buzz-5B-Medium, the codebase to refine, filter and augment the data, as well as prune and train your own variants, will additionally be released in the coming days.

Iterative Fine-Tuning Methodology

Our research builds upon the concepts introduced in several key papers, including:

• Simple and Scalable Strategies to Continually Pre-train Large Language Models
• NEFTune: Noisy Embeddings Improve Instruction Finetuning
• An Optimistic Acceleration of AMSGrad for Nonconvex Optimization
• Improving Generalization Performance by Switching from Adam to SGD
• Orca: Progressive Learning from Complex Explanation Traces of GPT-4

By combining high quality data, iterative fine-tuning with carefully selected "grounding" distributions from previous epochs, we have developed a cost-effective approach that pushes the boundaries of model reuse and optimization.

notably, we observe that the models have not yet appeared to plateu with the application of these techniques

Chat Template and Inference

To use the Buzz-8b-Medium model for chat-based tasks, you can utilize the provided chat template. Here's an example of how to perform inference using the Hugging Face Transformers library:

from transformers import AutoTokenizer, AutoModelForCausalLM

# Load the tokenizer and model
model_name = "H-D-T/Buzz-8b-Large-v0.5"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)

# Set the device to run the model on (e.g., "cuda" for GPU, "cpu" for CPU)
device = "cuda" if torch.cuda.is_available() else "cpu"
model.to(device)

# Define the input prompt
prompt = "Hello, how are you today?"

# Tokenize the input prompt
input_ids = tokenizer.encode(prompt, return_tensors="pt").to(device)

# Generate the model's response
output = model.generate(
    input_ids,
    max_length=100,
    num_return_sequences=1,
    no_repeat_ngram_size=2,
    early_stopping=True
)

# Decode the generated response
response = tokenizer.decode(output[0], skip_special_tokens=True)

print("Input:", prompt)
print("Response:", response)

NOTE: this model is a COMPLETIONS model, it will generate text by default, which completes the text you send it, it only has a start <|begin_of_text|> and a stop token <|end_of_text|> if you want it to have conversations reliably, append <|end_of_text|>\n<|begin_of_text|>assistant: to the end of your prompt, [the speaker 'assistant' is flexible, and can be tooled to the type of response you want, for example "Mathematician:"" will give you a different type of response, than "felon:"]

later iterations of the model will likely have formatting similar to openchat

Conclusion

We intend to focus on updating and improving the performance of these models, and surrounding open sourced infrastructure. Our next effort will focus on context and implementing the research currently being conducted by Wing-Lian, the lead developer of the Axolotl training framework that underpins these experiments. We encourage the community to explore Wing-Lian's work, such as the Llama-3-8b-64k-PoSE and llama-3-8b-256k-PoSE models, which showcase the potential for further advancements in language modeling.

Buzz hopes to be a proof of concept, and a toolkit to demonstrate and enable the community in the pursuit of efficient and effective locally run, personally owned, language models. Through collaboration with Hive Digital Technologies who have enabled us to perform this research, we have demonstrated the immense potential for model reuse and optimization.

Credits

to the many researchers who have open sourced their knowledge and tools to allow us to pursue this,

to Hive Digital Technologies for providing compute, advice, and meaningful research insight.

to Meta for developing the Llama models, and maintaining a philosophy of supporting open research and open source.

To wing et al. with Open Access AI Collective for developing axolotl, assisting with research, and generally being geniuses.

to Thomas Capelle et al. working on LLM_Surgery

as well as many, many others who are too numerous to name.

Citations

@misc{ibrahim2024simple,
      title={Simple and Scalable Strategies to Continually Pre-train Large Language Models}, 
      author={Adam Ibrahim and Benjamin Thérien and Kshitij Gupta and Mats L. Richter and Quentin Anthony and Timothée Lesort and Eugene Belilovsky and Irina Rish},
      year={2024},
      eprint={2403.08763},
      archivePrefix={arXiv},
      primaryClass={cs.LG}
}

@misc{jain2023neftune,
      title={NEFTune: Noisy Embeddings Improve Instruction Finetuning}, 
      author={Neel Jain and Ping-yeh Chiang and Yuxin Wen and John Kirchenbauer and Hong-Min Chu and Gowthami Somepalli and Brian R. Bartoldson and Bhavya Kailkhura and Avi Schwarzschild and Aniruddha Saha and Micah Goldblum and Jonas Geiping and Tom Goldstein},
      year={2023},
      eprint={2310.05914},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}

@misc{wang2020optimistic,
      title={An Optimistic Acceleration of AMSGrad for Nonconvex Optimization}, 
      author={Jun-Kun Wang and Xiaoyun Li and Belhal Karimi and Ping Li},
      year={2020},
      eprint={1903.01435},
      archivePrefix={arXiv},
      primaryClass={stat.ML}
}

@misc{keskar2017improving,
      title={Improving Generalization Performance by Switching from Adam to SGD}, 
      author={Nitish Shirish Keskar and Richard Socher},
      year={2017},
      eprint={1712.07628},
      archivePrefix={arXiv},
      primaryClass={cs.LG}
}

@misc{mukherjee2023orca,
      title={Orca: Progressive Learning from Complex Explanation Traces of GPT-4}, 
      author={Subhabrata Mukherjee and Arindam Mitra and Ganesh Jawahar and Sahaj Agarwal and Hamid Palangi and Ahmed Awadallah},
      year={2023},
      eprint={2306.02707},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}