Speaker Names Attribution, Ettin 150M
An encoder-only span model that extracts the printed real name of each
diarized speaker (Speaker A, Speaker B, and similar) from a transcript.
It abstains when the transcript does not identify the speaker.
The model is based on
jhu-clsp/ettin-encoder-150m
with a SQuAD-style start/end span head. Position zero is the null answer.
Results
Evaluation uses a manually checked set of 264 transcript-speaker rows, including 131 named and 133 unnamed speakers.
| Model | Precision | Recall | Person F1 | Exact-text F1 |
|---|---|---|---|---|
| Speaker Names Attribution, Ettin 150M | 0.844 | 0.870 | 0.857 | 0.812 |
Person F1 accepts any printed form of the correct person's name, for example
Kevin for Kevin Cork. Exact-text F1 requires the full target text.
On the automatically verified held-out test split (638 rows), person F1 is 0.882 and exact-text F1 is 0.830.
Usage
This repository includes the small custom span head and inference code:
pip install "torch>=2.4" "transformers>=4.51,<5" huggingface_hub
huggingface-cli download hmarchant/speaker-names-attr-ettin-150m \
--local-dir speaker-names-attr
python -m speaker_names.infer transcript.txt \
--model speaker-names-attr
Input must be a diarized transcript containing labels such as:
[Speaker A]: I'm Sarah Jones. Tom Baker joins us now. Finance Minister David Cole announced the new budget earlier. Good morning, Tom.
[Speaker B]: Good morning, Sarah. David's announcement was unexpected.
[Speaker C]: Here is the weather forecast.
The command prints a JSON mapping from each speaker label to the extracted
name or null.
Example output:
{
"Speaker A": "Sarah Jones",
"Speaker B": "Tom Baker",
"Speaker C": null
}
Training
- 149M parameters
- 4,988 train, 550 validation, and 638 test rows
- maximum sequence length 4,096, stride 1,024
- batch size 1 with 16-step gradient accumulation
- SDPA attention on unpadded windows
- selected at epoch 3 by person-level validation F1
- decision threshold: -2.25
Labels came from grok-4.7, were grounded against verbatim transcript spans,
then checked by a second model pass. Organisation names, programme names, and
sentence fragments were removed.
Limitations
- The model only returns names printed in the transcript. It does not infer identities from voice, outside knowledge, or unprinted context.
- Speaker labels are assumed to be local to each transcript window.
- Accuracy depends on diarization and ASR quality.
- The training material is mostly English-language news and current-affairs transcripts, so performance may be lower in other domains and languages.
- The custom checkpoint is
speaker_span.pt; it is not a standardAutoModelForQuestionAnsweringcheckpoint.