Yash-Llama2-7B-QLoRA-v1
Yash-Llama2-7B-QLoRA-v1 is an experimental fine-tuned version of NousResearch/Llama-2-7b-chat-hf.
Base Model
- Base model: NousResearch/Llama-2-7b-chat-hf
- Architecture: Llama 2
- Parameters: approximately 7B
- Task: Text generation / instruction following
Fine-Tuning
This project demonstrates:
- Supervised Fine-Tuning (SFT)
- QLoRA / LoRA
- PEFT
- 4-bit quantization
- bitsandbytes
- Transformers
- TRL
- Adapter merging
- Hugging Face deployment
Evaluation
A small manual evaluation was performed using 10 prompts.
| Metric | Base Model | Fine-Tuned Model |
|---|---|---|
| Relevance | 4.80/5 | 5.00/5 |
| Clarity | 3.70/5 | 4.60/5 |
| Structure | 4.00/5 | 4.50/5 |
| Correctness | 5.00/5 | 4.80/5 |
| Overall | 4.38/5 | 4.72/5 |
Overall manual score improvement: approximately 8.00%.
This is a small experimental manual evaluation and is not an official or comprehensive LLM benchmark.
Intended Use
This model is intended primarily for:
- Learning LLM fine-tuning
- QLoRA and LoRA experiments
- Instruction-following experiments
- Text generation
- Educational and research demonstrations
Limitations
This is an experimental model.
It may produce incorrect information, hallucinations, repetitive responses, biased outputs, or poor responses outside its fine-tuning distribution.
Important outputs should be independently verified.
Author
Yash Hanamghar
Hugging Face: yashhanamghar9099
Version
v1.0