We have played and recorded several musical chromatic scales with different instruments and conditions and plotted them in Time Domain, Frequency Domain and Time-Frequency Domain. Our plots correspond to: - 3 clips of clean music with one instrument playing only one note at a time. - 1 clip with multiple notes being played at the same time by an instrument. - 2 clips of two different instruments playing the same notes at the same time. - 1 clip of an instrument playing one note at a time without separating the notes from each other (legato) which makes the identification more challenging.
TIME DOMAIN
FREQUENCY DOMAIN
TIME AND FREQUENCY DOMAIN PLOTS
This is just an example of what a musical scale looks like in Time Domain and in Frequency Domain. The notes here have been played separately (staccato). Time Domain - Each peak corresponds to a played note -> Easy to separate them and get individual notes - The amplitude of the peaks represents the intensity (volume) -> It is not useful for the note identification Frequency Domain - Each peak corresponds to a frequency component -> Each component is the sum of the power for all the notes in the scale - It is not filtered -> We will implement a Butter-worth Filter later on. - It is not possible to identify the notes from this plot either. What can we do? Combine these two plots in ONE!
SPECTROGRAM
It is a plot of the short time Fourier transform of the signal with a window and it is a FUNDAMENTAL RESOURCE to identify the notes.
Our window length: 5000 It is chosen to achieve a trade-off between the correct frequency value measurement and the minimum overlap of samples in time domain
3D plot - Why? x axis:window in time domain y axis: frequency values Color:Yellow: frequencies with the highest power Blue: frequencies with the lowest power
VIOLIN CHROMATIC SATACCATO SCALE
The duration and most powerful frequency components of each note in the scale can be easily visualized. Let's ZOOM IN to see them better...
We will be able to identify each note by selecting the frequency component with the highest power each time we see that a note is played.
We know that a new note has been played because: - The yellow columns represent the time slots with high amplitude frequency components: notes. - The blue columns represent the time slots with low amplitude frequency components: silences.
We want to be able to determine the following:
Let's see what happens for different instruments!
PIANO CHROMATIC STACCATO SCALE
We can see the same results as before, with one slight difference: -The yellow "columns" are not that high as in the previous image: this is due to the fact that the sound of the violin is more fricative than the piano notes. However, it doesn't affect the note detection.
FLUTE CHROMATIC STACCATO SCALE
VIOLIN AND FLUTE CHROMATIC STACCATO SCALE
What happens if we play the same notes with a violin and a flute at the same time? The spectrogram shows that the notes are still recognizable.
PIANO AND VIOLIN CHROMATIC STACCATO SCALE
The same happens for the piano and the violin scales played at the same time.
2 NOTES AT THE SAME TIME IN THE PIANO
Let's change a little bit the recorded clip. This time we have played two different notes at the same time in the piano, and we can also see that there are yellow lines at two different frequencies. This way we can easily see which notes they are.
VIOLIN CHROMATIC LEGATO SCALE
Until now, all the scales have been played separately from each other -> They are easy to identify
If we play them with the legato technique, this is, joining them to each other, we can see that we will have problems to identify them.
There are not visible blue and yellow columns.
12 NOTES
We have splitted the piano scale time spectrum to extract each note and be able to identify the main frequency of each of them.
It is possible to observe that the main frequency of the spectrum that determines the note does not always correspond to the highest power peak.
One of our duties in this project is to be able to detect correctly the main frequency whether it is the frequency component with the largest amplitude or not. The FIRST PEAK is the one that identifies the note, the rest are harmonics.
In this project we will identify at least all the notes in the 3rd, 4th and 5th scales.