Key contributions
01
Combinatorial music generation
A new task. A model generates short note sequences from metadata, and the sequences are combined into a complete song.
02
Composer-made dataset
Every sample was composed by hand by professional composers, following written composition guidelines so the data stays consistent.
03
12 types of metadata
BPM, key, instrument and nine more. Two are unique to ComMU: track role and chord progression with extended chord qualities.
How it works
Stage 1
Generate note sequences
A model generates a note sequence for each set of metadata.
Stage 2
Combine into a song
Composers currently arrange the generated note sequences by track and time into a complete song. Automating this stage is future work.
Metadata
- BPM
- 80
- Key
- A minor
- Time signature
- 4/4
- Number of measures
- 4
- Genre
- cinematic
- Rhythm
- standard
- Track role
- main melody
- Pitch range
- mid
- Instrument
- violin
- Min/max velocity
- 2, 127
- Chord progression
- C-Am-F-G
Note sequence
aSLwZ5Ts
Combination
| t1 | t2 | t3 | ||
|---|---|---|---|---|
| Main melody | 1 | 4 | … | |
| Sub melody | … | |||
| Accompaniment | 2 | 2 | 2 | … |
| Bass | … | |||
| Riff | 3 | 3 | … | |
| Pad | 5 | … |
Each number is a note sequence generated in Stage 1
Dataset
Over six months, 14 professional composers worked in two teams. Team A turned reference music into composition guidelines, and Team B composed samples from them.
Team A
- 01Collect reference music by genre and mood
- 02Analyze the reference music
- 03Write composition guidelines with metadata
Team B
- 04Compose MIDI samples following the guidelines
ComMU dataset
11,144
MIDI samples with corresponding metadata
Track role
Each sample is labeled with its part in an arrangement: main melody, sub melody, accompaniment, bass, pad, or riff. With instrument labels alone, a model can only generate piano and guitar; with track role, it can generate piano as accompaniment and guitar as melody.
Chord progression
The set of chords used in a sample. Besides major, minor, diminished, augmented, and dominant, ComMU includes sus4, maj7, half-diminished, and min7. These extended chord qualities let the model learn more melodies for the same progression, improving harmony and diversity.
Genre
Two genres often used in background music. New age is melodious, built on keyboards and small acoustic instruments. Cinematic is orchestral, with string ensembles carrying the melody and accompaniment.
Results
Fidelity
In a survey, 40 anonymous composers compared generated and human-composed samples made from the same metadata and chose the one they preferred. Generated samples won about 42% of the time.
Human-composed
≈58%
Generated
≈42%
0255075100%Values read from the paper's chart
Controllability and diversity
Controllability is the ratio of generated notes that follow the given pitch range, velocity range, and key or chord. Diversity is the average pairwise distance between note sequences generated from the same metadata.
- Pitch control
- 0.84
- Velocity control
- 0.91
- Harmony control
- 0.99
- Diversity
- 0.32
Sampled with top-k 32, temperature 0.95