HIGH-QUALITY FREQUENCY DOMAIN-BASED AUDIO WATERMARKING. Eric Humphrey. School of Music Engineering Technology University of Miami

Transcription

1 HIGH-QUALITY FREQUENCY DOMAIN-BASED AUDIO WATERMARKING Eric Humphrey School of Music Engineering Technology University of Miami ABSTRACT An investigation of current audio watermarking technology is provided, from which a system is selected and implemented. In the algorithm presented, watermark embedding and detection are based in the frequency domain, achieved through the modification and subsequent crosscorrelation of the Fourier coefficients, respectively. The system is prototyped and implemented in both MATLAB and C ++ environments, and the performances of both systems are verified for accuracy and consistency. Particular emphasis is given to the translation of the algorithm from prototype to production implementation. Brief analysis of post-embedding audio quality and detection robustness to perceptual audio coders is also given. Index Terms Digital audio watermarking, copyright protection, data hiding 1. INTRODUCTION In short, the proliferation of digital technologies has led to the ubiquity of digital media. While the benefits of this sea change are staggering as they are innumerable, it has not come about without technological growing pains, being namely distribution and reproduction control. One of the foremost reasons for the rise of the digital age (the ease with which one can transfer information) has simultaneously become one of the greatest issues facing technology and the adoption of it by reluctant parties (the ease with which one can transfer information illegally). Brought to light by the Napster court case at the turn of the century, digital piracy is easy, and as a result of other communications advances, it is also getting faster, more efficient, and even easier. Digital watermarking provides the prospect of a solution to the distribution and reproduction issue for electronic multimedia, and while watermarking methods are generally equivalent across media formats, digital audio is considered here. Conceptually, watermarking strives to embed data that is inseparable from the media itself, and actually doing so would require the degradation of the medium s perceptual quality below some acceptable limit. Beyond copyright protection, there are further applications for audio watermarking, including use-enhancement and broadcast monitoring. The benchmark of a good watermarking scheme is that it must be imperceptible to the end-user, while robust enough to survive a variety of potential removal efforts (intentional or otherwise). A great deal of effort has been spent on the development of such a watermarking scheme, and there now exist a variety of implementation methods. Regardless of the specific watermarking approach, all schemes exhibit the same properties and trade-offs, stated succinctly in [07]: Audio Quality the allowable level of audio fidelity degradation, Robustness the ability with which a watermark survives un/intentional signal modification, Security the ability with which a watermark survives watermark tampering, Complexity the computational power necessary to embed/detect data, Data Capacity the amount of data per processed frame (payload), Granularity the amount of discrete time samples necessary to recover the watermark. As often noted in watermarking literature, these properties are conflicting and in direct competition with each other, highlighting the difficulty in obtaining a high quality watermarking system. There are several approaches to achieving this collection of properties. Watermarking schemes can generally be divided into two classes, based on the domain in which the watermark is generated and embedded (time vs. frequency). Temporal watermarking schemes are typically less expensive computationally than frequency-based methods, but generally yield poorer results. These systems often employ additive pseudo-random noise modulated by the time envelope of the audio waveform. An alternative time-based scheme, referred to as echo coding, places cropped, delayed versions of the signal back into the waveform, exploiting the Haas precedence effect. In the frequency domain, either Fourier phase or magnitude components are modified directly by some operation (summation, multiplication). Processing frequency information also allows for the exploitation of masking properties in the Human Auditory System, though this may present problems with regard to robustness against compression algorithms (codecs) employing psychoacoustic models (mp3, AAC, etc). Conceptually, watermarking is based on the idea that there may is information in an audio signal that is

2 perceptually unimportant, and attempts to hide data as such imperceptible information. Simultaneously, the ideal psychoacoustic-based compression algorithm aims to remove all perceptually redundant or unnecessary information, and would thereby remove a watermark perfectly. Some published methods have proven more robust to these audio compression algorithms, and are significantly more attractive options to further develop. With that in mind, a comparable watermarking algorithm is presented, fundamentally derived from the work of van der Veen et al [07]. 2. PROPOSED SYSTEM A watermarking system is developed based on the algorithm presented in [07]. However, in the course of implementing the system, adjustments were deemed necessary and are outlined below Watermark Embedding Initially, a normally distributed pseudorandom sequence with zero mean and unity standard deviation is generated by seeding a random number generator with some key, shown by (1), with length N/2. This key must be known beforehand by the watermark embedding/detection pair to allow for the recreation of the same watermark at the decoder. Cyclically shifting the PN sequence,, produces the data-dependent sequence,, where each shift corresponds to a different payload symbol (log 2 (N/2) bits per frame). 2ln 1 _ cos 2 (1) _ The audio signal is segmented into non-overlapping frames of length N, and the Fourier transform is computed. To maintain a real-valued inverse Fourier transform, only the first N/2 coefficients of are considered, as the magnitude spectra is even-symmetric about the DC frequency bin. The magnitudes are then multiplied by the final watermark sequence as (2) where m represents the frame number, is the range of the watermark, and is the mean (or offset). Changing the parameters and change the standard deviation and mean of the sequence, respectively, where the mean should be shifted approximately unity and the sequence range should be kept relatively small. Watermarked audio is achieved by computing the inverse Fourier transform for each complex-valued sequence,, and recombined in the time domain. The strength parameter is then directly tied to the robustness, as well the level of perceptibility, of the watermarked audio. A small and =1 are expected to produce a nearly identical signal in the time domain to the original audio. In particular, close attention must be given to the output watermark-embedded audio, so that no clipping occurs to the signal. This system deviates from the algorithm proposed in [07] by performing the summation from (2) in the frequency rather than the time domain after taking the inverse transform. Offsetting the normally distributed sequence about 1 prior to weighting the spectral coefficients ensures that phase values are unaffected. A watermark with zero mean will cause phase discontinuities in adjacent bins by flipping phase (φ±180 ) due to changes in sign Watermark Detection As outlined in [07], watermark detection is also performed in the frequency domain as the cross-correlation between the recreated watermark sequence,, and the Fourier coefficient magnitudes. Assuming synchronization, which will be discussed later, the audio signal,, is segmented into frames of length N (similar to the embedding process) and the Fourier coefficients are computed, given by. The algorithm in [07] recommends employing an accumulation buffer of L frames to average the spectral information in time, thus increasing the signal-to-noise ratio (SNR). This method, however, does not prove useful in the event that each frame may contain a different payload. As averaging is not essential to the detection process, it is not implemented in this system. Cross-correlation can be efficiently computed for the two sequences by taking the product of the N/2 length discrete Fourier transform representations of each sequence after computing the natural logarithm, given below as, ln (3) ln (4) (5) Taking the natural logarithm in (3) and (4) aims to separate the product that forms the spectral envelope of the watermarked waveform, generated by (2), and is conceptually equivalent to homomorphic filtering in image processing. A simple exercise in expanding the natural logarithm of (2) shows this, where the contribution to the correlation measure is roughly a DC offset. The cross-correlation vector is computed per frame, and must be processed to produce consistent results, defined in (6) and shown in Figure 1. Data detection is defined by the coordinate [peak, index], the max of, where the value peak corresponds to the detection reliability, or confidence level, and index, the actual payload. Note that and denote the mean and the standard deviation of C, respectively. In the case that there

3 is no watermark, the peak would, and should, not breach a set threshold. the embedded data rate is defined as log, so the data capacity decreases exponentially with the increase of. It is therefore desirable to find a compromise between a somewhat large and a small, rather than the inverse, to yield worthwhile channel capacity and imperceptibility. The selection of a threshold for the confidence level in the detection process is desirable, but less important, to the operation of the system. Even as a worst case scenario, the detector could simply return the data corresponding to the maximum of the cross-correlation, or monitor the level of the peaks detected (indirect thresholding). This peak monitor could operate either as an average value function, or tally peaks that exceed a threshold, while keeping the peak values regardless. A lowvalued peak monitor implies no (or an incorrect) watermark, while a high value conveys watermark detection. 3. IMPLEMENTATION: REACTIONS & RESULTS Figure 1 - Normalized cross-correlation for a single frame of audio (top) and its derivative (bottom). (6) The peak of can be further accentuated, providing better detection results, by filtering the vector (i.e. computing a low order derivative) and finding the area of greatest change. Statistically, the presence of a watermark will produce such a large spike in the contour of that its value is a local maximum exhibiting inflection points on both sides of the peak Parameter Selection The functionality of the system outlined depends heavily on the selection of,, and. The offset of the normally distributed watermark,, should be slightly less than 1 to maintain approximately the same signal energy of the original signal. It is for this reason the system requires a normally, and not uniformly, distributed pseudorandom sequence. Audibility and strength of the watermark are directly linked by the trade-off between and. Larger values of or will increase the confidence level in the detector, constrained by (the watermark sequence should never be negative). However, these large values will also increase the audibility of the watermark. Additionally, Given the laundry list of initial difficulties faced in the development of a functioning system based on the algorithm outlined, the fact that the system works, regardless of perceptual quality, is admittedly surprising. It is even more impressive, then, that the algorithm developed sounds exceedingly good, in addition to producing high confidence values for rather high data rates. Before delving into performance data and discussion, some realizations precipitating from the development process should be noted. Most importantly, Fourier coefficient magnitudes must maintain even symmetry and the zero and Nyquist bins must remain purely real to produce a real-valued inverse transform. This constraint results in a watermark sequence that is half the length of the frame size. Separate from the watermark embedding/detection system itself, data synchronization needs to be employed in some facet within the system. This is particularly necessary in the case that a signal has been cropped/zero padded, and the first sample of the audio to the detector does not correspond to the first sample of a frame. Technically, assuming no other data alteration has occurred, this would merely result in frame sharing, as the watermark information is spread across the length of the initial frame. As a worst case, where the signal is padded/cropped by half the frame size, each detection frame would produce a confidence value corresponding to the data in the two adjacent frames at half the strength of an otherwise synchronized detection. Clearly, synchronization can be easily achieved through redundancy in the embedding stage at the cost of halving the data capacity, i.e. by embedding payload symbols in pairs of adjacent frames. At this point, the problem is a data management one, and beyond the scope of this project.

4 Figure 2 - Multi-frame Normalized Cross-correlation measure for (left to right) Watermark present, Alternate Watermark Present, and no Watermark Present Clipping, and signal normalization on a whole, is of notable interest, especially in the case the system is deployed in production. Modifying the spectral coefficients will change the signal energy, and stands to produce a timedomain signal beyond the range of [-1, 1]. Scaling is the most obvious solution, but by no means the most desirable one. Watermark detection will not be affected, but the output audio will unavoidably be quieter, a non-ideal result given the aim of a perceptually indistinguishable watermarked signal Algorithm Performance The system was originally implemented with an unnecessarily large frame size to encourage detection, but found to maintain perfect detection for a data rate of 237bps (N=2048). Watermarked audio was found to produce confidence values regularly exceeding a cross-correlation score of 6, while non- or alternate-sequence watermarked audio seldom produces cross-correlation scores over 4. A threshold for data detection can optimally be selected in this region, depending on the application, selectivity, and how the threshold will be employed. The multi-frame crosscorrelation results are shown for the three scenarios mentioned above in Figure 2. Independently, the watermarked audio is perceptually indistinguishable from the original audio. Tests and figures were initially computed with the values of.75,.95, and , which maintains a data rate of approximately 35bps. At that rate, a 128-byte message requires slightly under 30 seconds for extraction. Given the values of and, it is expected that scaling will be necessary, causing the only perceptible difference between the original and watermarked versions of the audio. Limits were found to exist for.2,.8, and 2048, allowing extremely reliable detection and perceptually equivalent audio Programming Environment Translation After the successful completion of a system prototype in MATLAB, facilitated by a built-in functions and GUI verification tools, a production implementation was developed in C ++. This process required locating and including external libraries to allow Fourier transform computations and reading/writing to.wav files. The system presented employs the KissFFT and libsndfile libraries to complete these tasks. As is good practice in programming, both libraries were instantiated separately and compared with MATLAB results to verify proper functionality. While a good majority of the prototype system mapped directly to the production environment, a few instances in translation proved troublesome. A normal distribution function was written, returning a value in the set [-1 1], since the random number function native to C ++ produces uniformly distributed integers. In order to normalize the cross-correlation result, a standard deviation formula also needed to be obtained and implemented. Complex conjugate multiplication also needed to be translated, which, admittedly, was initially overlooked for its intricacies, and not computed correctly. This type of error highlights the required level of detail for even simple operations that may be taken for granted Verification The benchmark for a confirming the transition from one environment to another is to directly compare the outputs of both systems. In this case, the dual-sided nature of the system, embedding and detecting, produces two quantifiable outputs. As shown in Figure 3, the errors between the watermark embedding systems (output audio) and detection systems (correlation matrix) are accountable to the accumulation of machine precision errors. The embedded audio differs by a mean absolute error of , while the more computationally intensive correlation matrix has a mean absolute error of Taking the base- 2 logarithm of each mean absolute error shows that the audio is, on average, equal through over 16 bits of precision (exceeding the original audio quality), and the correlation matrix is equal through over 11 bits.

5 For testing, a 135-second watermarked.wav file is encoded as an.mp3 file according to the bitrates stated in Table 1. The four files are then converted back into.wav files and parsed to the watermark detection process. Detection is measured as both the percentage of peaks corresponding to the correct shift (i.e. the peak correlation index per frame is the intended data) and the average peak value. Partial detection matrices are shown in Figure 4 for the various formats. 5. CONCLUSIONS Figure 3 - MATLAB and C++ System Performance Differences for embedded audio (top) and detection matrix (bottom) 4. TESTING Without exception, the most commercially viable application for a good audio watermarking system is copyright protection and management, given the ease with which files can be distributed electronically. So far, all work and testing has been conducted with CD-quality audio (44.1kHz, 16-bit). Unfortunately, digital audio rarely exchanges hands in this format due to enormous file sizes (upwards of 600MB for an album), but rather as files that have been compressed using a variety of psychoacousticbased lossy compression algorithms. Therefore, above most everything else, a quantifiably good watermarking system should still survive in reasonably compressed audio. The qualifier reasonably comes from the fact that, beyond a certain point of compression, the audio will not sound good (i.e. has been effectively destroyed), and a watermark has no reason to survive beyond this limit (though, if it did, that would be an impressive watermarking algorithm). Table 1: Detection Performance for Various Audio Formats Format Detection Ratio Average Peak.wav (uncompressed) mp3-320kbps mp3-192kbps mp3-128kbps mp3-96kbps wav (no watermark) A frequency-domain watermarking system is implemented in two programming environments, verified for accuracy, and tested for performance. Slight modifications were made to the initial algorithm design, and both the auditory and mathematical results far exceed expectations. It was verified that the two systems are significantly equivalent, particularly the audio output of the watermark embedding process. The rather small discrepancies between resulting detection matrices likely arise from a combination of factors, mainly error accumulation, and possibly the use of different Fourier transform libraries. The algorithm s performance after mp3 encoding conveys a great deal of information about the watermarking system itself. An initial observation is that even for a bitrate typically deemed perceptually indistinguishable from highfidelity audio (320kbps), the watermark suffers significantly. However, performance, and, of most importance, the confidence peak values, do not diminish much farther for lower bitrates. As Figure 4 shows, there are still many areas in the detection matrices where the watermark survives, even for the different compression ratings. The reason that performance deteriorates so much develops from parts of the song which are spectrally vacant (intros and interludes, for example). For this style of watermark to be detected, the audio must be very active though at least a good majority of the spectra. In the cases that it is not, a perceptual audio coder will readily disregard all inaudible frequency information that was holding parts of the watermark sequence. It is conceivable that the performance of this algorithm could be greatly enhanced by, for a given frame length, analyzing the spectral activity and embedding the watermark sequence only in the most active spectral region (for example, the watermark embedding process could find a N/8 th length window with the most spectral energy, and only apply the watermark there). Regardless, the watermark system does survive in some detectable capacity despite perceptual coding, and based on the application and its data throughput demands, this system could conceivably be satisfactory. As will be stated shortly, there is still work to be conducted with this style of watermarking system, but with the ease and quality a system like this can be developed, it seems inevitable that some universal audio watermarking algorithm will be developed.

6 5.1. Future Work Numerous instances in this paper state steps that could be taken to further the algorithm presented. Achieving imperceptibility does not seem to be an issue with any of the simple listening tests conducted over a small sampling of audio. Future work would likely focus on achieving more consistent watermark detection and data extraction. Experimentation may yield ideal parameters that, in turn, directly yield better detection results. Filtering, and in particular low order derivatives, may also prove beneficial in enhancing peaks in the detection matrix. As mentioned, some form of data synchronization needs to be implemented in the ideal watermarking system. This would allow for other forms of signal corruption to be tested, such as analog/air transmission, time-variation, cropping/padding, filtering, or any combination of that mentioned. Also, performance against other audio coders should be considered, but were not available at the time. 6. REFERENCES [01] Neubauer, Christian et al. A Compatible Family of Bitstream Watermarking Schemes for MPEG-Audio. Journal of the Audio Engineering Society, presented at the 110 th Convention (Paper 5356). May [02] Garcia, Ricardo. Digital Watermarking of Audio Signals Using a Psychoacoustic Auditory Model and Spread Spectrum Theory. Journal of the Audio Engineering Society, presented at the 107 th Convention (Paper 5073), September [03] Lemma et al. A Temporal Domain Audio Watermarking Technique. IEEE Transactions on Signal Processing. Vol. 51 No. 4, April [04] Oh et al. Imperceptible Echo for Robust Audio Watermarking. Journal of the AES, presented at the 113 th Convention (Paper 5644), October [05] Esmaili et al. A Novel Spread Spectrum Audio Watermarking Scheme Based on Time-Frequency characteristics. Canadian Conference on Electrical and Computer Engineering, May [06] Jackson, Tim et al. Watermarking and Copy Protection by Information Hiding in Soundtracks. AES 25 th International Conference, London UK, June [07] van der Veen, Michiel et al. Robust, Multi-functional and High-quality Audio Watermarking Technology. Journal of the AES. Presented at the 110 th Convention (Paper 5345), May [08] Bassia et al. Robust Audio Watermarking in the Time Domain. IEEE Transaction on Multimedia, Vol. 3, No. 2, June [09] Malvar et al. Improved Spread Spectrum: A New Modulation Technique for Robust Watermarking. IEEE Transactions on Signal Processing, Vol. 51, No. 4, April [10] Swanson et al. Robust Audio Watermarking Using Perceptual Masking. Signal Processing, V.66 No.3, p , May Available Online. [11] Gomes et al. Cyclostationarity-based Audio Watermarking with Private and Public Hidden Data. Journal of the AES, Presented at the 109 th Convention (Paper 5258), Los Angeles, CA, September Figure 4 - Detection Matrices for Perceptually Coded Audio (from top left, counterclockwise), 320kbps, 192kbps, 96kbps, and 128 kbps.