musicdiff is a file format agnostic music file diff tool that compares two music scores, and produces a rendered PDF of each score, visually marked up with descriptive text and colored notes, pointing out the differences.  It is focused on notational (visual) differences rather than only audible differences.

This can be very useful for visualization of the differences between an OMR-generated music file and ground truth.  I have found it particularly useful in quickly categorizing the types of mistakes being made by music reading software.

For example, here is a visualization of the differences between two versions of the first few measures of Mozart’s “String Quartet No. 14 in G Major”. The first version is incorrect, and the second version is the correct ground truth.

There are three differences to the human eye: a staccato mark, a natural, and a quarter note tied to an eighth note that is mistakenly notated as a dotted quarter note. The diff engine does things more notationally, describing that last change as an inserted quarter note, an inserted tie, and then the following changes to the existing dotted quarter note: the dot has been deleted, and a flag has been added. A human would probably have described this as a removal of the dot from the dotted quarter, and then an insertion of an eighth note and tie, but this is similar to the kinds of idiosyncrasies you might see from a text file diff (e.g. git diff), when there is similar text near a difference.

musicdiff is both a command-line tool and a music comparison API package. The musicdiff command-line tool uses fairly high-level musicdiff APIs to do its work.  I imagine that the lower-level musicdiff APIs could just as easily be used in other tools as well, perhaps in custom OMR assessment tools for specific research projects.  Please contact me here if you have a need for such tools, and would be interested in collaborating.

musicdiff can compare scores at three different levels of detail:

1. GeneralNotesOnly: Notes/rests/chords only (including expressions, articulations, beams, etc)
2. AllObjects: Other objects as well (clefs, key signatures, time signatures, metronome marks, dynamics, text directions, lyrics, etc)
3. AllObjectsWithStyle: Style detail as well (note shape, note color, stem direction, text style, etc)

It handles scores with differing numbers of measures easily, and will even detect which specific measures were inserted or deleted.  It is also careful to ignore voice ordering differences within a staff measure, since that is not a visible notational difference, but simply a software choice when writing to a particular file format.

musicdiff can compare scores that are in different file formats. For example, an OMR engine being evaluated might produce Humdrum **kern files, but the ground truth data might be in MusicXML files. musicdiff uses music21 to parse those file formats, and the diff engine works on music21 data structures, completely independently of whatever the two file formats may have been.

A note about Humdrum **kern files: music21’s Humdrum parser is limited, and doesn’t support many of the Humdrum **kern features that are supported by humlib (which is used by Verovio and others). Because of these limitations, musicdiff uses an alternative Humdrum parser I wrote which is derived from humlib.  See the sources here, if you’re interested in the details.

musicdiff is derived from the diff engine described in “A diff procedure for music score files”, by Francesco Foscarin, Raphaël Fournier-S’Niehotta, and Florent Jacquemard, with the addition of detail levels 2 and 3, improved visualization, and enhanced performance.

musicdiff is written entirely in Python3, and can be installed via “pip3 install musicdiff”. This will automatically install the other few packages it depends on.  You also will need to configure music21 (instructions here) to display a musical score (e.g. with MuseScore, Finale, etc). 

Usage: python3 -m musicdiff 
         [-h]
         [-d {GeneralNotesOnly,AllObjects,AllObjectsWithStyle}]
         file1 file2

Music score notation diff (MusicXML, MEI, Humdrum, etc)

positional arguments:
  file1   first music file to compare
  file2   second music file to compare

optional arguments:
  -h  show this help message and exit
  -d  detail level {GeneralNotesOnly,AllObjects,AllObjectsWithStyle}

Simplest example (default detail level is AllObjects):

% python3 -m musicdiff omrOutput.krn groundTruth.musicxml

musicdiff sources can be found at this link. 

If you use this work in any research, please cite the relevant paper:

@inproceedings{foscarin2019diff,
  title={A diff procedure for music score files},
  author={Foscarin, Francesco and Jacquemard, Florent and Fournier-S’niehotta, Raphael},
  booktitle={6th International Conference on Digital Libraries for Musicology},
  pages={58--64},
  year={2019}
}