music-generation
This model a trained from scratch version of distilgpt2 on a dataset where the text represents musical notes. The dataset consists of one stream of notes from MIDI files (the stream with most notes), where all of the melodies were transposed either to C major or A minor. Also, the BPM of the song is ignored, the duration of each note is based on its quarter length.
Each element in the melody is represented by a series of letters and numbers with the following structure.
- For a note: ns[pitch of the note as a string]s[duration]
- Examples: nsC4s0p25, nsF7s1p0,
- For a rest: rs[duration]:
- Examples: rs0p5, rs1q6
- For a chord: cs[number of notes in chord]s[pitches of chords separated by "s"]s[duration]
- Examples: cs2sE7sF7s1q3, cs2sG3sGw3s0p25
The following special symbols are replaced in the strings by the following:
- . = p
- / = q
-
=
-
- = t
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 0.0005
- train_batch_size: 32
- eval_batch_size: 32
- seed: 42
- gradient_accumulation_steps: 8
- total_train_batch_size: 256
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: cosine
- lr_scheduler_warmup_steps: 1000
- num_epochs: 100
- mixed_precision_training: Native AMP
Training results
Framework versions
- Transformers 4.19.4
- Pytorch 1.11.0+cu113
- Datasets 2.2.2
- Tokenizers 0.12.1