DIMACS
DIMACS REU 2015

General Information

me
Student: Jennifer Crawford
School: Rowan University
E-mail: jennc [at] reu [dot] dimacs [dot] rutgers [dot] edu
Project: Determining the Age of an Improvised Jazz Solo via Note Intervals

Project Description

The 20th century transformed jazz music with an endless succession of "eras" (ie., big band, bebop, hard bop, jazz fusion), but jazz's characteristic use of improvised solos remained throughout. These improvised solos exist across jazz instruments (including saxophone, guitar, voice, etc.). But just how random are these musical improvisations?

The aim of this project is to investigate possible "characteristic" distributions of pitch intervals (ie, jumps in pitch between consecutive notes in a solo) that typify the improvisations of a given era of jazz. Such characteristic distributions could allow for jazz solos to be "carbon-dated" by the analysis of such intervals alone.


Weekly Log

Week 1:
6.1.15 - 6.5.15
Projects were proposed on Monday, and I deliberated between my current project and the one taken on by another in Professor Kontorovich's REU group, Lolly Kenigsberg .
Week 2:
6.8.15 - 6.12.15
In order to transcribe the solos, we were at first considering LilyPond software. After my initial project presentation, Rachel Levanger, a Rutgers graduate student and mentor for another REU group very helpfully suggested that I use PDF-to-MIDI conversion software and then gather information about the note-intervals directly from the MIDIs. This will be a tremendous advantage, as I will now not have to record the intervals between each and every pair of notes in each and every sample solo transcription. There are many examples of PDF-to-MIDI conversion software on the web, including some open-source and shareware distributions, but most have earned mixed reviews from users. I'm now in the process of picking out the most appropriate software for the PDF-to-MIDI conversion process.
Week 3:
6.15.15 - 6.19.15
After a few failed attempts at installing open-source PDF-to-MIDI software, we've settled upon using SmartScore X2 MIDI Edition for our PDF-to-MIDI conversions. The software seems to work reasonably well and if it works well I won't have to learn too much about using the built-in TIFF editor.

I was looking into ways of extracting note data from a MIDI file, and I stumbled upon a few StackExchange questions addressing this ("Get note data from a MIDI file" and "Is there a way I can extract pitches from MIDI?"). My mentor looked deeper into this and discovered that Mathematica handles MIDI files quite nicely, and so it was decided that the code would be written in Mathematica.

Professor Kontorovich also recommended some papers that explore somewhat similar questions about improvisation: for example, Norgaard's How Jazz Musicians Improvise: The Central Role of Auditory and Motor Patterns , which analyzes the solos of famed jazz saxophonist Charlie "Bird" Parker. Also, Hook Theory provides a really interesting and detailed example of the ways in which Markov analysis can be used to analyze and predict chord patterns.
Week 4:
6.22.15 - 6.26.15
I attempted to read an article ( Polansky, Barnett, Winter 2009 ) on James Tenney and dissonant counterpoint, but, being unfamiliar with both, it's a fairly difficult read, but I'd definitely like to return to the more mathematical (and less musical) aspects of it.

A friend at the REU, Adam, told me about a Firefox add-on called "Down Them All" which made it possible for me to download all of the PDF solo transcrptions on Saxopedia. I did this for the saxophone, trumpet, and guitar solo transcriptions from saxopedia and was able to get a large chunk of the transcriptions listed on each page.
Week 5:
6.29.15 - 7.3.15
I wrote some Mathematica code that, given a MIDI version of a solo, will return the list of intervals (in half-steps) between consecutive notes. I was very happy to see how easily Mathematica can generate histograms and other visualizations of a song's interval distribution.

I am also in the process of loading the solo MIDIs, PDFs, and TIF files to Dropbox so that my mentor can help me determine whether the PDF-to-MIDI conversion was done correctly and whether a piece is truly improvised. Unfortunately, SmartScore X2 is not infallible and I may need to spend some time editing the TIFF files that it generates from PDFs.
Week 6:
7.6.15 - 7.10.15
SmartScore seems to have trouble with the conversions of more "crowded" sheet music, but we've decided not to prioritize editing those exceptions. Unfortunately SmartScore only allows PDF-to-MIDI conversions for one file at a time, so I was forced to find a workaround unless I wanted to sit by the computer to convert each of the 300+ files manually. Luckily, my friend Adam again recommended AutoIt, a "freeware BASIC-like scripting language." I was able to write a script to automate the PDF-to-MIDI conversion, which ran into trouble mostly when SmartScore would crash. Supervising the automated conversion process, I was very grateful that I didn't have to do each and every conversion myself; AutoIt certainly saved me a lot of time and tedium, in addition to teaching me a bit about script-writing.
Week 7:
7.13.15 - 7.17.15
A change that was made to the AutoIt script about halfway into the conversion process unintentionally saved the .MID's as SmartScore's default file type, the .ENF (sort of like a combination of TIF and MIDI). However, after speaking to Prof. Kontorovich, I decided to spend the week working on the rest of the Mathematica code: since the MIDI-to-intervals code is finished, I needed a way to store each solo's title, the range of note-intervals, and the relative frequency with which each interval was used in a solo (# of occurrences of a particular interval \ # of total intervals in the solo) which would be conducive to taking the dot product of two solos' relative frequency (or relFreq) vectors. I decided on the following format for each solo:
{Solo Title and Filepath, {minimum Interval, relFreq[minimum Interval]}, ... {maximum Interval,relFreq[maximum Interval]} }
Week 8:
7.20.15 - 7.24.15
I wrote a new AutoIt script with which to convert the mistakenly created .ENF's into .MID's and finished converting all trumpet/saxophone solos. All of our data is officially on Dropbox. Professor Kontorovich wrote up a Mathematica function which, given two solos' indices in the directory, would output the dot product of the two solos' relative frequency vectors. Now the idea is to export the dot products of each pair of solos to a specified text (and later, Excel) file.

The roadblock I'm currently facing is that, while the DotProduct function works when the solos' indices are specified (ie, DotProduct[1,2] will output the correct number), all attempts at creating a for/while loop with which to export information for the dot products of each solo pair have failed for mysterious reasons.
Week 9:
7.27.15 - 7.31.15
Analyzing dot products, persistent homology, future goals.

Additional Information