John Taylor - Natural Sciences

Pitch Class Profile problem

jtnatsci — Mon, 08 Mar 2010 13:13:20 +0000

The PCP method produced some good results, but there is a problem. Just because harmonics of a note are multiples of the fundamental frequency, this does not mean that they have the same note value. For example, A3 (220Hz) has harmonics at A4 (440Hz), E5 (660Hz ish), A5 (880Hz), C#6 (1108Hz) etc.. This means that I have to do a bit more manipulation, and find the most likely fundamental frequency for each peak before I assign a note value. More later…

Pitch Class profile

jtnatsci — Fri, 05 Mar 2010 10:31:13 +0000

The next method is a ‘pitch class profile’ method. Using the new half-second sample window for the FFT, the FFT bucket size is small enough to uniquely identify the pitch to a specific semitone. This means that each FFT bucket can be labelled with its closest midi note equivalent (of course. in the higher frequencies, several buckets will share the same midi note as the scale is logarithmic). Each of these midi notes can be translated into a musical note value, for example ‘C#’, ‘E’ etc. This can be thought of as a note number between 1 and 12 (ignoring the octave number).

When the peak positions are translated into these note numbers, a histogram can be produced showing the frequencies of each note present. The higher the frequency, the more likely it is that that note is present. With the harmonic structure of guitars in general, I would expect at least three peak values to be present for each real note present. This method should give an idea of how many different value notes are in the sample.

Let’s say the sample returns a histogram telling us that there are 6 peaks that equate to a ‘G’, and 4 that equate to an ‘E’. We then need to determine which octave numbers of these notes are present. This will be explored in a further post.

Other methods explored

jtnatsci — Fri, 05 Mar 2010 10:19:25 +0000

Over the last couple of weeks I have investigated some further methods for detecting multiple pitches in a frequency spectrum. The first of these is a peak removal algorithm which does the following:

Assume that the lowest ‘significant’ peak is a fundamental frequency
Using the ‘fine tuning’ algorithm previously described, find the likely harmonic peaks of this fundamental frequency
Flatten the peaks of the fundamental and its harmonics
Repeat steps i) to iii) until no more significant peaks are found

The down side to this method is that higher octave notes will not be found if a lower octave equivalent is present, as its harmonic will be destroyed when the lower note is detected and removed. Formal results of this method will follow.

Fine tuning

jtnatsci — Wed, 17 Feb 2010 12:55:13 +0000

To resolve the tuning issue, I have developed an algorithm for fine-tuning the frequency detector. The old version was only as accurate as the FFT bucket size, i.e. if there are 2205 buckets representing a frequency range of 0-22,050Hz then each bucket has a width of 10Hz. This limits the frequency resolution of the old algorithm. The new one looks at the first seven harmonics of the detected fundamental, and tries some minute shifts (negative and positive) of f0 to see how it effects the harmonics. It looks at the likelihood of these slightly shifted f0 values by summing the power of the newly calculated theoretical harmonics. In the example below, the original f0 was calculated at 390Hz, but with the fine tuning, it recalculates it to 373Hz, as this f0 more closely matches the harmonics present. The red circles are the harmonics of this new f0 (logarithmically decreasing).

The low-end peaks are noise on the recording, and are ignored by the algorithm.

Harmonic responses

jtnatsci — Wed, 10 Feb 2010 10:33:20 +0000

To address the problem with harmonics, I have decided to do a study of the harmonics produced by the guitar. A sample note was recorded of every string, and every fret. These notes were then analysed by taking their Fourier transform, and the relative power of their fundamental frequency and first four harmonics was taken. This is shown on the following diagram:

It can be seen that for several notes, the fundamental frequency is not the dominant frequency in the transform. There are patterns in the results – the seventh fret (row 8 in the diagram, as row 1 is the open string) shows that the dominant frequency is the first harmonic on 4 of the 6 strings, rather than the fundamental. This may mean I have to revise the method I use to detect the frequencies present. At the moment I am thinking of modelling responses using the data from the harmonics, and comparing these ‘theoretical’ responses to the test data. By removing matching notes one at a time, I should be able to determine the notes that are playing. I have investigated using HTK to produce a probabilistic result, but the time constraints of the project do not allow me to pursue this option. It is something that I would look into further if I had the time.

For better, for worse

jtnatsci — Mon, 01 Feb 2010 12:40:22 +0000

OK, I’ve amended the pitch detection algorithm in the following ways:

The frames marked as the onset frames are the ones that have the highest increase in power across all frequencies. To detect which actual notes are ‘onsetting’ however, I am now using the onset frame +1. This seems to give more consistent data. If I take the ‘delta’ values of this ‘onset+1′ frame, they are positive for all the new frequencies in the signal. I then use this as a mask to clean up the power spectrum – this leaves only the new tonal data that we are looking for.
In order to identify the actual note that is starting, I am using frame ‘onset+3′. This gives a little time for the note to settle in after the initial strike.
The Harmonic Product Spectrum (see earlier post) was giving strange octave results, because of noise across the frequency range, and the lack of certain harmonics in the guitar signals (this is typical). Instead of using the product of f0 and the harmonics, I am now applying a weighting factor to the first and second harmonics, and summing f0 up to f4.

This has given mixed results. The octave problem is now gone, but it has been replaced by one of tuning. The song sounds recognisable, but in a ‘Les Dawson’ kind of way. Back to the grind.

The results of this method are here:

http://www.jtuk.com/natsci/testout3.mid

Octave shift

jtnatsci — Fri, 29 Jan 2010 13:44:31 +0000

The first method I have used is the ‘Harmonic Product Spectrum’.

This method takes the windowed time domain signal, transforms it to the frequency domain, using FFT, then multiplies the resultant spectrum by compressed versions of itself (at 2x, 3x, 4x and 5x compression). The theory is that the harmonics of the fundamental will ‘line up’ with the fundamental as the spectrum is compressed. In practice however, this is giving me problems. The first problem is that there is some noise at multiples of (f0/2). This means that the largest peak is at (f0/2), which results in the note being shifted down an octave. The second problem is that the third harmonic on the guitar that I am currently using is almost absent. The effect of this is to include zero as one of the factors which is not good. To correct these problems I am working on a solution that uses the sum of the harmonics, with a weighting factor, instead of the product. More to follow.

Diagram ref: Patricio de la Cuadra, https://ccrma.stanford.edu/~pdelac/154/m154paper.htm

Success, and my guitar playing

jtnatsci — Fri, 29 Jan 2010 13:33:11 +0000

To complete stage one of the project, I had to get the system to correctly identify and convert to midi all of the notes in a monophonic tune, played on an electric guitar. The test tune that I have been using is my amazing rendition of “Frere Jaques”:

http://www.jtuk.com/natsci/testwav.wav

This file contains 32 notes, and the initial results look quite good. 28 of the 32 notes have been recognised correctly, and the other 4 are ‘nearly’ right. They are out by an octave – a common problem with frequency detection algorithms. The midi file sounds like this:

http://www.jtuk.com/natsci/testout1.mid

The next post will explain the problems and possible solutions to the octave problems.

Apologies

jtnatsci — Tue, 26 Jan 2010 12:33:59 +0000

There has been a significant break in the updates to the blog recently. I had laser eye surgery at the end of last semester, and could not use a computer for 5 weeks. I’m back now though, and will be updating with the latest progess soon.

Onset detection

jtnatsci — Thu, 05 Nov 2009 14:42:08 +0000

To resolve the problem of the jumping octave (previous post), I have developed a method of finding the ‘onset’ of a note. When this is found, the fundamental frequency just after that onset is determined, and any harmoncs that are present are ‘labelled’ with that fundamental frequency. As long as any of these harmonics are still playing, the fundamental is deemed to still be playing. This first image shows the gradients of the harmonics (time is going upwards in this image). The rise and fall of each peak is clearly visible, and it shows the fundamental dropping off before the first harmonic.

To determine the onset, however, we need to find where the gradient increases in the other dimension. This is shown in the next image:

Here we can see that the notes begin to form in frame 3, continue their rise in frame 4, and the gradually tail off. If we sum the values of the gradients for each time frame, we should get a peak at frames 3 and 4. This is shown in the next image:

There is a distinct ‘spike’ at frame 4, which is easy to detect. In order to allow the attack phase of the note to complete, I have arbitrarily chosen to wait 3 more frames before taking the FFT to determine the fundamental frequency of the note that appears at this point. I then apply a method called ‘harmonic product spectrum’ to determine the frequency of the note. This gives the following result:

which shows a perfect spike at 150.98Hz. The actual value for D3 is 146.33Hz, but as the notes either side are 138.59Hz and 155.56Hz, the closest semitone to our result is the correct one.