Jazz Harmony Embeddings

A small transformer that reads the chord progression of a jazz tune and returns one 128-dimensional vector. Tunes with related harmony end up close together: the same tune in another key, another book's chart of it, or a contrafact (a new melody written over an existing tune's chords).

The model was trained from scratch on about 8,000 chord charts. This repository holds the three checkpoints that make up the released ensemble.

• Code, evaluation harness, and experiment records: https://github.com/eigenben/jazz-harmony-embeddings
• Precomputed embeddings for 6,900 jazz standards: https://huggingface.co/datasets/eigenben/jazz-harmony-embeddings
• Browser demo: https://huggingface.co/spaces/eigenben/jazz-harmony-explorer

Model description

Input. A chord chart, with one token per chord event. Adjacent repeats of the same chord are merged and no-chord markers are dropped, and the chart is truncated to its first 256 events. Each token carries the chord's root pitch class, the interval from the previous root, quality, triad and seventh class, bass interval, duration, metrical strength, and a continuation flag. The key of the tune is not an input.

Architecture. A 3-layer transformer encoder (d_model 192, 6 heads, FFN 768) with attention pooling, projected to a 128-dimensional L2-normalized embedding. Each checkpoint has about 1.4M parameters.

Training. NT-Xent contrastive loss at temperature 0.08. A positive pair is two independently transposed views of the same tune. When the training set holds other charts in the tune's family, one of the views is drawn from a different chart with probability 0.7. Training ran for 20 epochs with batches of 256 tunes, using AdamW (learning rate 3e-4, weight decay 0.01, 2 warmup epochs).

The project also implemented music-theory augmentations such as tritone substitutions and ii–V insertions. In ablation they did worse than real alternate charts as a source of positives, so they are turned off in this model.

Ensemble. The release is three checkpoints trained with seeds 7, 17, and 29. The embedding for a tune is the mean of the three checkpoints' embeddings, renormalized to unit length. Compare embeddings with cosine similarity.

Training data

The training corpus is 8,089 chord charts merged from four sources: the iReal Pro "Jazz 1460" community playlist, the Bunks Jazz-Chord-Progressions-Corpus, the real-book partition of ChoCo, and the Jazz Harmony Treebank. The charts contain chord symbols only, with no melodies and no audio.

Duplicate charts and known contrafacts are grouped into 4,790 tune families, and the train, validation, and test splits are made by family, so no tune has a relative on the other side of a split.

DATA.md covers licensing and explains why the published embeddings leave out tunes sourced from the Treebank.

Evaluation

Each query is a tune, and the model ranks every other tune in the corpus by cosine similarity. The metrics score where the query's known relatives land in that ranking.

The held-out test families are divided into a development partition and a confirmation partition. Model selection used validation loss and the development partition. The confirmation partition was not consulted until the model was final. The full protocol is in eval/benchmark-policy.md.

Held-out families

On the confirmation partition (157 queries from families the model never saw in training):

Curated contrafacts

This benchmark is 37 queries with hand-graded contrafact relationships. It was consulted during development, so its numbers are development results, not a blind test. The baselines were run on the same corpus.

The model has the best nDCG@20, the only metric here that uses the relevance grades. It does not lead on the others. Sequence alignment and the Bunks method both find a first relative sooner (median rank 2 against 4) and have higher MRR, and three of the four baselines have higher Recall@20.

The model's advantage is in what it outputs. The two strongest baselines compare tunes a pair at a time and return a score. This model returns a vector for each tune, which can be indexed, clustered, and projected onto a map.

Transposition invariance

A tune transposed to a random key retrieved its original at rank 1 in 995 of 1,000 trials (99.5%).

The project also trained hierarchical variants that model bars and phrases. All of them failed the transposition test, so the flat model is the one released.

Limitations

• The model sees chords only. Two tunes with the same changes and unrelated melodies look identical to it, and two reharmonizations of one melody look unrelated.
• The contrafact numbers are development results. A fresh blind contrafact set was planned but never built. The held-out family numbers are the ones to rely on.
• The training data is lead-sheet charts of jazz standards. The model is unlikely to transfer to other genres or to beat-level transcriptions of performances.
• The weights are licensed CC BY-NC-SA 4.0, which is non-commercial, because part of the training corpus carries that license.

Usage

If you want vectors for well-known jazz standards, use the precomputed dataset. The checkpoints are for embedding your own charts, which have to be parsed into the project's Tune schema first.

# pip install "jazz-harmony-embeddings @ git+https://github.com/eigenben/jazz-harmony-embeddings"
import numpy as np
from huggingface_hub import hf_hub_download
from jazz_harmony_embeddings.models.inference import load_checkpoint, embed_tunes

paths = [
    hf_hub_download("eigenben/jazz-harmony-embeddings", f"checkpoints/best-s{seed}.pt")
    for seed in (7, 17, 29)
]
models = [load_checkpoint(path)[0] for path in paths]

# tunes: list[jazz_harmony_embeddings.data.schema.Tune]
mean = np.mean([embed_tunes(model, tunes) for model in models], axis=0)
embeddings = mean / np.linalg.norm(mean, axis=1, keepdims=True)

The checkpoints are full training payloads saved with torch.save, and load_checkpoint unpickles them. Load them only from this repository.