Model Card for LLaMat-3
LLaMat-3 is a specialized large language model designed to be a foundational large language model for materials science.
Overview
- Model Type: Large Language Model (LLM)
- Base Model: LLaMat-3 (continued pretraining of LLaMA-3 on material science data)
- Language: English
- License: LLaMA-3 License
- Tags: Material Science, Domain Adaptation, Table Understanding, Scientific Data Parsing, Materials Copilot
Model Details
Key Features
- Applications: Can be finetuned for information extraction, table understanding, parsing data for research tasks, and crystal structure generation.
Development and Support
- Developed by: M3RG, IIT Delhi & DAIR, IIT Delhi
- Compute Support:
- Edinburgh International Data Facility (EIDF): Provided access to Cerebras CS2 clusters for pretraining.
- IIT Delhi High-Performance Computing Cluster: Supported fine-tuning and inference stages.
Technical Specifications
Hardware Infrastructure
- Pretraining: 2 Cerebras CS-2 Wafer-Scale Engines (WSE-2)
Software Stack
- Frameworks: PyTorch, Hugging Face Transformers
Model Sources
- Repository: LLaMat on GitHub
- Compute Resources: EIDF Cerebras CS Clusters