Elec 484 Final Project Report. Marlon Smith

Transcription

1 Elec 484 Final Project Report Marlon Smith

2 Abstract This report discusses the implementation of a variety of audio effects using a phase vocoder. Effects such as time stretching, pitch shifting, and robotization are discussed, as well as other tools like audio file compression and filtering. Introduction The phase vocoder is a method of audio processing that allows for easy implementation of a variety of audio effects including time stretching, pitch shifting,robotization, whisperization, denoising, transient/ stable component seperation, and filtering. The phase vocoder works by windowing chunks of audio in the time domain, and converting each of those chunks to the frequency domain. Once in the frequency domain, processing is performed to create a variety of audio effects and filtering. Since this phase vocoder was implemented in MATLAB, the built-in FFT function was used to convert signals to the frequency domain. The amplitude and frequency components resulting from the FFT were separated, as both sets of information are required to properly reconstruct the original signal, and both are required for calculations involving some of the audio effects (such as time stretching). Phase Vocoder Basics As mentioned above, the first step for the implementation of a phase vocoder is to split the audio signal into segments. Each segment is then multiplied by a raised cosine, to reduce the amount of harmonics that would be generated by a square window. The distance between the centers of two of these windows, in samples, is know as the analysis hop size. For most of my phase vocoder, a window was 2048 samples long and the hop size was 1024 samples. The next step is to take the cyclic shift of each windowed segment. This is done by simply swapping the first and second halves of the segment, sample by sample. Next, the MATLAB FFT function is used to take the FFT of each windowed segment. This converts the signal to the time domain, with a resolution of (sampling frequency/window size). The resulting amplitude and phase components are separated. Now that we are in the frequency domain, most of the effects can be implemented. These effects will be explained in more detail in the following sections. The fourth step is to combine the amplitude and phase, now modified or filtered to achieve the desired audio effect, back together. We can then take the inverse FFT of our signal and return to the time domain. At this point the signal is still separated into windowed segments. Next, a cyclic shift is performed again, returning the signal to its original form.

3 Finally, the windowed segments are overlapped and added together through a process known as overlap-add. This reconstructs the signal in the time domain, allowing the modified audio to be played back. Figure 1: Amplitude and Phase vs. Frequency vs. Time Figure 1 above shows amplitude (top) vs. frequency vs. time and phase (bottom) vs. frequency vs. time. This is the information directly obtained by taking the fft of the windowed function of Toms_diner.wav.

4 Testing with Different Frequencies The windowing, cyclic shift, and overlap add sequence was tested at six different frequencies: An integer and non-integer number of samples per cycle at a frequency giving many cycles per segment, 1 cycle per segment, and a small fraction of a cycle per segment. Plots for amplitude vs. time vs. frequency and phase vs. time vs. frequency are shown below in figures 2-7. Figure 2: Integer samples per cycle, less than one cycle per segment Figure 3: Integer samples per cycle, one cycle per segment

5 Figure 4: Integer samples per cycle, greater than one cycle per segment Figure 5: Non-integer number of samples per cycle, less than one cycle per segment

6 Figure 6: Non-integer number of samples per cycle, one cycle per segment Figure 7: Non-integer number of samples per cycle, greater than one cycle per segment From the above graphs, it is apparent that the phase vocoder handles integer numbers of samples per cycle much better than non-integer numbers. A non-integer number of samples per cycle will create extra frequency components near the edge of the signal where it is chopped off. This is visible in the amplitude plots shown above.

7 Time Stretching The first effect I implemented with my phase vocoder was time stretching. Time stretching is done partly by modifying the phase of the signal in the frequency domain, and partly by changing the hop size for reconstruction of the signal in the time domain. Once the FFT has been performed on the original signal, we are able to modify the phase to correct for the pitch shift that would normally accompany a time stretch. By looking at the phase value of the same frequency bin in the previous window, and compensating for a new synthesis hop size, we can retain the original pitch of our audio signal while stretching it in time. Once the new phase values have been calculated, we bring the signal back to the time domain. While overlap-adding the windowed segments, we move the segments further apart (stretching) or closer together (shrinking). This changes the length of the audio sample, while the pitch is retained because of the phase compensation mentioned above. My implementation of time stretching sounds a little distorted. The reason for this is that my hop size was 1024, which was half my window size of When I stretched my signal by two times in the time domain, this meant that there was no signal for the section of time where the two windows were no longer overlapping. The solution to this problem would be to simply use an analysis hop size that was less than half the window size. This would mean that the windows would continue to overlap even when stretched by a factor of two or more. Pitch Shifting Pitch shifting was the second effect I implemented with my phase vocoder. The implementation of this effect was relatively simple compared to the time stretching effect. Once the signal was converted to the frequency domain, each window was resampled by a shifting factor. This caused all of the frequency components in the signal to be moved either up or down. Because they were all multiplied by the same factor, pitch relationships were preserved, which would not be the case if they were all simply shifted by the same amount in the frequency domain. Once the windowed segments were resampled in the frequency domain, they were converted back to the time domain using the same hop size that they were created with. The result was a change in pitch of the signal, without changing its length. Robotization and Whisperization Several interesting effects can be created by modifying the phases of the windowed segments in the frequency domain. The first effect is robotization, which is realized by setting all of the phases to zero. The result is that the audio file is reduced to one pitch. This resulting pitch is affected by the size of the window used when the signal is segmented. Smaller windows give a higher pitch, while larger windows give a lower pitch.

8 The second effect that can be created by modifying phase information is whisperization. Whisperization is created when the phases of each windowed segment are set to random numbers. Window size is also important; a smaller window generally gives a more whispery-sounding result. Denoising Denoising is a process which completely removes sounds that are below a certain threshold. It can be used to remove a quiet background hiss from a sound file. Denoising is achieved by setting the amplitude of any frequency components below a certain threshold to zero. This is done easily with a phase vocoder, since the signal is always converted to the frequency domain. Bandpass Filter Implementation A bandpass filter can be implemented using a phase vocoder. This is done by converting the transfer function of the filter in the z domain to its magnitude response in the frequency domain. This gives a filter fresponse H(n), where n is one of the steps between 0 and 2048 in the frequency domain. Once this is done, the response can be multiplied by each windowed segment in the frequency domain. This has the effect of passing signals inside the passband region, and attenuating all other signals. For my phase vocoder, I swept the center frequency of my bandpass filter up and down creating a wah-wah effect. One of the key details required to successfully implement a wah-wah filter the way I did was to understand the relationship between analog and digital frequencies. For my digital filter, I used a samping frequency or 8000 Hz, which was the same as the sampling frequency for my phase vocoder. This meant that each window in the frequency domain had a range of which corresponded to a frequency of Hz. Knowing this correlation, one can find the resolution in the frequency domain, which is simply 8000/2048 = Hz per one increment in the frequency domain. This is 2 f 2 n = also shown by the equations showing that =. fs windowsize Below is a graph of the frequency response of my filter with its center frequency set at 1000Hz. This corresponds to n = 256 using the equation above, and this is visible from the graph. Note that the filter response is mirrored around fs/2, which is the nyquist frequency. Signals containing frequencies above fs/2 will produce undesirable effects in this system.

9 Audio Compression Audio compression can be implemented using a phase vocoder by simply keeping only the largest n frequency comonents, or by eliminating any frequencies below a certain threshold. If done properly, the listener will not be able to discern a difference between the compressed file and the original one, but the compressed file will take up much less storage space. I created two matlab files to demonstrate this effect: audiocompression.m and audiocompression2.m. audiocompression.m compresses the file by removing all frequencies below a certain threshold, set by the threshold variable in the code. audiocompression2.m compresses the file by keeping the largest n frequency comonents, set by the variable keepbins. Audio output from these 2 techniques is included with this report. It is interesting to note that using audiocompression2, which only keeps the n largest frequency components, a fairly good signal can be obtained with only a few hundred frequency components. Keeping only a few hundred out of 2048 is a lot of frequencies to throw away, but the audio quality is still almost useable (depending on the situation). This is demonstrated by the output files included with this report.

10 Conclusion The phase vocoder is a very useful tool for easily implementing a variety of audio effects, as this document and accompanying MATLAB code has shown. Once signals have been brought into the time domain, it is easy to modify them to create a variety of effects. The phases can be adjusted to create effects like pitch shifting and time stretching, and the windowed segments can be moved around in the time domain. Having a signal easily accessible in the frequency domain also opens up possibilities for analysis and compression. Speech recognition can be done in the frequency domain, and audio compression can be done by removing certain frequency components based on knowledge of the sensitivity of the human ear and the amplitude of the particular frequencies in the signal. An audio equalizer could also be created to amplify certain frequencies more than others, or a system could be designed to work similarly to the 'loudness' button found on many older stereo systems, amplifying (at low volume levels) the frequencies which are less sensitive to the human to give a more pleasing sound.