Cat1.0

Overview

Cat1.0 is a fine-tuned version of Llama-3-1-8b base model, optimized for roleplaying, logic, and reasoning tasks. Utilizing iterative fine-tuning and human-AI chat logs, this model works well for numerous chat scenarios.

Model Specifications

• Parameters: 8 Billion (8B)
• Precision: bf16 (Brain Floating Point 16-bit)
• Fine-Tuning Method: LoRa (Low-Rank Adaptation)
• Lora Rank: 32
• Lora Alpha: 64
• Learning Rate: 0.0008
• Training Epochs: 4
• Datasets Used:
  • cat1.0 Roleplay Dataset
  • cat1.0 Reasoning and Logic Dataset
• Fine-Tuning Approach: Iterative Fine-Tuning using self-chat logs

Recommended Settings

To achieve optimal performance with this model, I recommend the following settings:

• Temperature: 1.1
• Min P: 0.05

Note: Due to the nature of the fine-tuning, setting the temperature to 1.1 or higher helps prevent the model from repeating itself and encourages more creative and coherent responses.

Usage Instructions

I recommend using the oobabooga text-generation-webui for an optimal experience. Load the model in bf16 precision and enable flash-attention2 for improved performance.

Installation Steps

1. Clone the WebUI Repository:

   git clone https://github.com/oobabooga/text-generation-webui
   cd text-generation-webui
   

2. Install Dependencies:

   pip install -r requirements.txt
   

3. Download the Model:

   Download the fine-tuned model from Hugging Face and place it in the models directory.

4. Launch the WebUI:

   python server.py --bf16 --flash-attention
   

Sample Prompt Formats

You can interact with the model using either chat format or chat-instruct format. Here's an example:

Ryan is a computer engineer who works at Intel.

Ryan: Hey, how's it going Natalie?
Natalie: Good, how are things going with you, Ryan?
Ryan: Great, I'm just doing just great.

Model Capabilities

Below are some examples showcasing the model's performance in various roleplay scenarios:

Roleplay Examples

Text Generation Example

Limitations and Tips

While this model excels in chat and roleplaying scenarios, it isn't perfect. If you notice the model repeating itself or providing less coherent responses:

• Increase the Temperature: Setting the temperature higher (≥ 1.1) can help generate more diverse and creative outputs.
• Adjust min_p Setting: Ensuring min_p is at least 0.05 can prevent low-probability tokens from being excluded, enhancing the response quality.

Acknowledgments

• oobabooga text-generation-webui: A powerful interface for running and interacting with language models. GitHub Repository
• Hugging Face: For hosting the model and providing a platform for collaboration. Website
• Meta For pre-training the Llama-3.1-8B Base Model that was used for fine-tuning. Model Card

For any issues or questions, please open an issue in this repository.