STEMMA-MF2601
Paper-final Music Flamingo + STEMMA-Instruct: V7 coreference + relation MultiQA
1x (no fullsong/block component), full fine-tuning with trainable audio encoder,
global batch 128, one epoch, LR 1e-5. Final step 431; base revision
6b5be086d52f65a1e204cb0faf70bf54e2741ecd. Four BF16 weight shards are at the
repository root and indexed by model.safetensors.index.json.
Status: isolated installation, 17 CPU tests and 24 full-checkpoint B200 requests
passed. Input token hashes match the sealed evaluation records; one parsed answer
differs from vLLM in this acceptance sample. See VALIDATION.md. This is not paper
score reproduction. This repository contains the complete fine-tuned weights and
editable native Transformers/PyTorch inference code, not a LoRA adapter.
Minimal use and editable features
python -m pip install -r requirements.txt
python examples/generate.py --model . --audio a.wav b.wav \
--prompt 'Audio A: <audio>\nAudio B: <audio>\nCompare A and B.'
python examples/embeddings.py --model . --audio a.wav b.wav
Use literal newlines in Python prompts; a shell single-quoted \n stays a
backslash sequence, although marker order still remains valid. Download a pinned
commit from hoyso48/STEMMA-MF2601 with huggingface_hub.snapshot_download, then
run these examples against its local folder. No Swift/base-model cache is needed.
from huggingface_hub import HfApi, snapshot_download
import sys
repo = "hoyso48/STEMMA-MF2601"
revision = HfApi().model_info(repo).sha # Record this commit for reproduction.
folder = snapshot_download(repo, revision=revision)
sys.path.insert(0, folder) # The downloaded repository includes the stemma code.
from stemma import load_model
session = load_model(folder)
from stemma import Request, load_model
session = load_model(".")
answers = session.generate([Request("Compare A and B.", ["a.wav", "b.wav"])])
batch = session.prepare([Request("Describe this source.", ["a.wav"])])
audio_features = session.extract_audio_embeddings(batch)
# Editable session.model and session.processor support custom forward hooks.
See API.md for tensor shapes, order/masks, unpooled encoder/projected/contextual
features, request batching and generation profiles. Audio slots remain independent;
RoTE resets per source, not across the whole list. Decode to mono 16 kHz through
SoundFile/SciPy, retain tails, use separate 30-second windows capped at 20 minutes
per source. Context is 32,768 tokens. Generic sampled and greedy reference profiles
are separate; backend/RNG differences are documented rather than hidden.
An instance-only zero-token-tail RoTE fix preserves the full waveform/mel while
avoiding upstream indexing of a trailing window that contributes no output tokens.
Read NOTICE.md and the complete NVIDIA license before use. MF weights and
derivatives are academic noncommercial only, not unrestricted commercial use.
New wrapper/example code is Apache-2.0 (LICENSE); that code license does not
change the weight license. Stock vLLM serving is not supported by this package;
the paper used a separately patched backend described in API.md.
Model training data are not distributed here. release_info.json, provenance.json
and checksums.json identify weights, source adaptations and staged assets.