Marsh Harrier Lite (MSH-Lite)
Marsh Harrier Lite (MSH-Lite) is a compact, efficient version of the MedIT Solutions MSH-v1-Bielik-v2.3-Instruct-MedIT-merge model, reduced to 7 billion parameters using advanced pruning techniques. This pruning retains the core functionality and efficiency of the original model while optimizing for reduced computational resource usage.
Key Features:
- Pruned Model: Reduced from 11B to 7B parameters using the pruning method based on the MedIT Solutions LLaMA pruning framework.
- Optimized Performance: Despite its reduced size, MSH-Lite delivers competitive performance across a wide array of NLP tasks.
- Bilingual Support: Designed to handle both Polish (pl) and English (en) with high fluency.
Technical Details:
- Base Model: MSH-v1-Bielik-v2.3-Instruct-MedIT-merge
- Parameter Count: 7 billion
- Architecture: Derived from Bielik's core architecture, with parameter optimization.
- Model Efficiency: Ideal for deployments where computational efficiency is paramount.
Performance Highlights:
To be done.
Acknowledgments:
Gratitude to the SpeakLeash project and ACK Cyfronet AGH for their contributions and collaboration.