Llama 3.2 1B - Synthia-v1.5-II - Redmond - Fine-tuned Model
🌐 Website
You can find more of my models, projects, and information on my official website:
🚀 Prompt Hub
Need high-quality prompts for image models and LLMs? Explore findgoodprompt.com.
💖 Support My Work
If you find this model useful, please consider supporting my work. It helps me cover server costs and dedicate more time to new open-source projects.
- Patreon: Support on Patreon
- Ko-fi: Buy me a Ko-fi
- Buy Me a Coffee: Buy me a Coffee This model is a fine-tuned version of NousResearch/Llama-3.2-1B on the Synthia-v1.5-II dataset.
Thanks RedmondAI for all the GPU Support!
Model Description
The base model is Llama 3.2 1B, a multilingual large language model developed by Meta. This version has been fine-tuned on the Synthia-v1.5-II instruction dataset to improve its instruction-following capabilities.
Training Data
The model was fine-tuned on Synthia-v1.5-II.
Training Procedure
The model was trained with the following hyperparameters:
- Learning rate: 2e-05
- Train batch size: 1
- Eval batch size: 1
- Seed: 42
- Gradient accumulation steps: 8
- Total train batch size: 8
- Optimizer: Paged AdamW 8bit (betas=(0.9,0.999), epsilon=1e-08)
- LR scheduler: Cosine with 100 warmup steps
- Number of epochs: 3
Framework Versions
- Transformers 4.46.1
- Pytorch 2.3.1+cu121
- Datasets 3.0.1
- Tokenizers 0.20.3
Intended Use
This model is intended for:
- Instruction following tasks
- Conversational AI applications
- Research and development in natural language processing
Training Infrastructure
The model was trained using the Axolotl framework version 0.5.0.
License
This model is subject to the Llama 3.2 Community License Agreement. Users must comply with all terms and conditions specified in the license.
Explore More
Discover more of my models, projects, and practical AI resources:
- AI tools directory: JustGoodAI — a curated directory of useful AI tools.