Blaster — SLAP diarization head
Private frame-level multi-speaker activity model. Emits arrival-ordered tracks so Stage C can build:
<soa> <spk1> [frames…] <spk2> [frames…] <eoa>
Hub metadata (correct typing)
| Field | Value | Why |
|---|---|---|
pipeline_tag | voice-activity-detection | Closest Hub tag: per-frame speech activity over ≤4 speakers (not single-label audio-classification) |
library_name | pytorch | Raw encoder.pt — custom DiarEncoder, not Transformers |
| Tags | speaker-diarization | Full diarization (segments + speaker ids) is built in diar_segments |
There is no first-class Hub speaker-diarization pipeline tag; do not use
audio-classification (implies a single clip-level class).
Files
| File | Role |
|---|---|
encoder.pt | Canonical checkpoint for serve (MOLMO_DIAR_CHECKPOINT) — arrival phase ~step 28k |
encoder_fork4_arrival_28k.pt | Named copy of the arrival checkpoint |
encoder_fork4_pit.pt | PIT separation phase backup |
Architecture
DiarEncoder(fork_blocks=4)— trainable copy of Stage A’s lower blocks +DiarBranchtransformer- ASR trunk path stays frozen (WER preserved from Stage A)
- Output: per-frame logits over ≤4 speakers @ ~25 Hz; segments via
diar_segments.segments_from_logits
Gate B (held-out mixtures)
NIST-collar DER ~18% (stated gate was 15%; accepted to proceed). Live diar costs ~4 pp on Gate D spoken attribution vs oracle markers.
Serve
export MOLMO_DIAR_CHECKPOINT=/path/to/encoder.pt
# or compose: DIAR_MODEL_DIR pointing at this repo root
Aux worker: AudioEmbeddingProcessor → process_diarized → {embeds, speakers, spk_lens}.
Load
from diar_infer import DiarEncoder
import torch
m = DiarEncoder(fork_blocks=4)
state = torch.load("encoder.pt", map_location="cpu", weights_only=False)
m.load_state_dict(state["model"], strict=True)
Related
- Stage A trunk:
molmo-audio-slap-stage-a - Serving LM:
molmo-audio-serving-diar-d - Index:
molmo-audio-phase-h-restore
License
CC BY-NC-SA 4.0 (Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International).
Marketing name: Blaster. Technical Hub id remains molmo-audio-*.
- Attribution required.
- NonCommercial — no commercial use of these weights without a separate grant.
- ShareAlike — adaptations must use the same license.
Base backbone allenai/Molmo-7B-D-0924 remains Apache-2.0. Training mixes that include People’s Speech / Dolly are themselves under CC-BY-SA; this weight license is intentionally more restrictive (adds NC). See project README_TRAINING.md licensing summary.