StableLM-Base-Alpha: sharded checkpoint
This is a sharded checkpoint (with ~2GB shards) of the model. Refer to the original model for all details.
Basic Usage
install transformers, accelerate, and bitsandbytes.
pip install -U -q transformers bitsandbytes accelerate
Load the model in 8bit, then run inference:
from transformers import AutoTokenizer, AutoModelForCausalLM
model_name = "ethzanalytics/stablelm-base-alpha-3b-sharded"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name, load_in_8bit=True)