Pythia Quantized Model for Sentiment Analysis
This repository hosts a quantized version of the Pythia model, fine-tuned for sentiment analysis tasks. The model has been optimized for efficient deployment while maintaining high accuracy, making it suitable for resource-constrained environments.
Model Details
- Model Architecture: Pythia-410m
- Task: Sentiment Analysis
- Dataset: IMDb Reviews
- Quantization: Float16
- Fine-tuning Framework: Hugging Face Transformers
The quantized model achieves comparable performance to the full-precision model while reducing memory usage and inference time.
Usage
Installation
pip install transformers torch
Loading the Model
from transformers import AutoTokenizer, AutoModelForCausalLM
tokenizer = AutoTokenizer.from_pretrained("AventIQ-AI/pythia-410m")
model = AutoModelForCausalLM.from_pretrained(model_name, low_cpu_mem_usage=True)
# Example usage
text = "This product is amazing!"
inputs = tokenizer(text, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=20)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
Performance Metrics
- Accuracy: 0.56
- F1 Score: 0.56
- Precision: 0.68
- Recall: 0.56
Fine-Tuning Details
Dataset
The IMDb Reviews dataset was used, containing both positive and negative sentiment examples.
Training
- Number of epochs: 3
- Batch size: 8
- evaluation_strategy= epoch
- Learning rate: 2e-5
Quantization
Post-training quantization was applied using PyTorch's built-in quantization framework to reduce the model size and improve inference efficiency.
Repository Structure
.
├── model/ # Contains the quantized model files
├── tokenizer/ # Tokenizer configuration and vocabulary files
├── model.safensors/ # Fine Tuned Model
├── README.md # Model documentation
Limitations
- The model may not generalize well to domains outside the fine-tuning dataset.
- Quantization may result in minor accuracy degradation compared to full-precision models.
Contributing
Contributions are welcome! Feel free to open an issue or submit a pull request if you have suggestions or improvements.