Model Card for Model ID
lainshower/Llama3-8b-alpaca
Model Details
Full Fine-tuned Llama3-8B Alpaca.
Training with (BF16) Mixed Precision For Stability.
This is One-Epoch Training Model due to Overfitting Problem > Click here Llama3-8B-Alpaca-3EPOCHS For the 3-Epochs Training Model.
A detailed description will be given later.
Direct Use
[Templates]
You can use the following standard templates for inference the Llama3 Alpaca model:
[Code]
[Model Loading]
[Template]
[Generation]
Training Details
Training Data
- We use stanford_alpaca for Instruct Tuning Llama-8B.
Training Hyperparameters
- Learning Rates : 2e-5
- Training Procedures : Mixed Precision (bfloat16)
- Context Length: 512
- This is One-Epoch Training Model due to Overfitting Problem > Click here Llama3-8B-Alpaca-3EPOCHS For the 3-Epochs Training Model.
Training Graph
