gslm-scaling-823m-82b
Model-only FP32 weights for a complete endpoint from Scaling Properties of Speech Language Models (EMNLP 2024). The model predicts consecutive-run-collapsed, 25 Hz layer-11 mHuBERT K-means units without waveform normalization (K=500; EOS=500; PAD=501). A vocoder is not bundled.
Speech-tokenizer and vocoder artifacts are credited in the SLM third-party notices.
import torch
from slm import TransformerLM
model = TransformerLM.from_pretrained("tiagoCuervo/gslm-scaling-823m-82b", device="cuda")
prompt = torch.tensor([[12, 91, 204]], device="cuda")
units = model.generate(prompt, max_new_tokens=200, temperature=0.8, top_k=50)
BF16 evaluation scores are 61.33% sBLIMP phenomenon macro accuracy,
58.36% sBLIMP voice-pair accuracy, 56.92% sStoryCloze, and
77.93% tStoryCloze. See provenance.json for hashes and exact validation metadata.
Use the slm-research codebase for data preparation, evaluation, and audio decoding. Source
datasets and the CodeHiFiGAN vocoder retain their own licenses and are not included.