Standards compliant watermarking for access management

Standards compliant watermarking for access management Viresh Ratnakar and Onur G. Guleryuz Epson Palo Alto Laboratory 3145 Porter Drive, Suite 104 Palo Alto, CA 94304, USA ABSTRACT We present standards-compliant visible watermarking schemes for digital images and video in DCT-based compressed formats. The watermarked data is in the same compressed format as the original and can be viewed with standard tools and applications. Moreover, for most of the schemes presented, the watermarked data has exactly the same compressed size as the original. The watermark can be inserted and removed using a key for applications requiring content protection. The watermark application and removal algorithms are very efficient and exploit some features of compressed data formats (such as JPEG and MPEG) which allow most of the work to be done in the compressed domain. Keywords: Visible watermarks, scrambler, discrete cosine transform, JPEG, MPEG, content protection. 1. INTRODUCTION Visible watermarks on digital image and video data can be used to prevent or discourage unauthorized use of the data. The watermark may be something simple and unobtrusive, such as a small logo in a corner of the image or frame. But such a watermark can easily be cropped away or concealed without a drastic loss in quality. At the other extreme, the entire image area may be corrupted in a manner that can be undone using a key. This would be similar to encrypting all the data, except that straight-forward encryption would not leave the data format-compliant. In this paper, we address a specific visible watermarking problem which lies in between these extremes, and propose a range of solutions for still images and video. The problem arises in several real and practical commercial imaging applications, and can be described as follows: Let us consider a still image, I (Figure 1 (a)). We would like to devise a keyed scrambling algorithm S such that I = S(I,K) is an image with the same dimensions as I, and is similar to I except in certain regions that form a distinctive pattern. In these regions comprising the visible watermark, the scrambled image I appears random or corrupted. The overall and intended effect is exemplified by the image in Figure 1 (b). The scrambling should be secure in the sense that without knowing the key K, an attacker should not be able to recover the original image from I. Moreover, the scrambling should be fully reversible, given the key. That is, a complementary descrambling algorithm D must exist such that D(I,K)isexactly the same as I. The objective in case of video is the same the scrambled video should be viewable, but should show a distinctive pattern corrupting parts of the video frames. For comparison, examples of two traditional visible watermarking schemes are also shown in Figure 1. Figure 1 (c) shows a blended visible watermark (this example was created by simple alpha-blending, but more secure schemes are also possible). This traditional approach to visible watermarks (, 1 3 for example) has the goal of applying a watermark that does not substantially degrade the quality of the image. Figure 1 (d) shows the simplest possible visible watermark a logo in a corner. Clearly, for uncompressed raw images the desired effect shown in Figure 1 (b) can be achieved, for example, by XORing the pixel values (within the pattern regions) with a keyed pseudo-random number sequence. The interesting case is that of (lossy) compressed data. Let us restrict our attention to still images for the time being; we will return to video towards the end of the paper. Our goal is to take a compressed image I, and scramble it to get I = S(I,K) which is also a valid image in the same compressed format and shows the distinctive pattern. Note that since we will only be dealing with compressed images, we have avoided complicating the notation and have not used any additional qualifiers such as c(i) to indicate compressed images. A nice feature of this format-compliant visible watermarking approach is that I can be viewed by any image viewer, and can also be passed through any compression-aware Further author information: E-mail: {ratnakar,oguleryuz}@erd.epson.com

(a) Original (b) Desired obstructive visible watermark (c) Traditional blended visible watermark (d) Corner logo as visible watermark Figure 1. Different visible watermark possibilities. components of an imaging and/or networking system. But compression introduces a complication: if we take the approach of modifying raw pixels, then we would first have to decompress, then apply the modifications, and then compress again a lossy process. It would be hard to ensure that the original image can be recovered exactly upon subsequent descrambling. We present a scrambler S and descrambler D which solve this problem for Huffman-coded JPEG 4 images, and then discuss complications and partial solutions for extending the same approach to DCT-based video coding formats. Another desirable property of the scrambling algorithm would be that it introduces no coding overheads. That the size (in bytes) of I should be exactly the same as that of I. The main algorithm in this paper does achieve this property. We also describe a variant where a moderate size increase is incurred to make the watermark appear more strongly. 1.1. Applications We consider two sample applications, one for still images and one for video. Consider an Internet-based image shop which sells high-quality professional art that customers may purchase remotely and print. The customer runs an Internet client software to browse thumbnails of the artwork and retrieve the ones that he/she is interested in. When he/she retrieves a high-quality image, the server sends a watermarked image which he/she can view, but only in a much inferior version because of the obtrusive visible watermark. Upon providing the correct key, the client may have an integrated descrambler and printer-driver, which will print the image free of the watermark. We do not address key management issues in this paper, as that will be a part of the overall needs of the system. The key in this system, for example, can be session-based, to be retrieved securely by the client software upon the customer s payment.

The advantage of using this visible watermarking scheme in the above system, as opposed to simple encryption of the whole image, is that the client software can display the watermarked image. Moreover, standard technologies for retrieving content over the Internet can be used, with the knowledge that cached copies of the artwork (in the network as well as the client s local disk cache) will only be of marginal value because of the obstructive watermark. For video, the sample application would be delivery over satellite or cable networks. Again, the scrambled video can be viewed to get a flavour of the content without revealing all of it. Paying customers may retrieve the keys needed to descramble and view the unwatermarked video. 2. JPEG REVIEW The JPEG image compression standard is widely used for efficient storage and tranmission of images. The techniques discussed in this paper utilize certain properties of the JPEG format. In this section, we review the core concepts of JPEG, emphasizing those pertinent to our visible watermarking techniques. A JPEG image consists of 8 8 blocks which have been transformed using the Discrete Cosine Transform 5 (DCT), quantized using a table of 64 scalar quantizers, and then entropy-coded. The 8 8 blocks of the image are grouped into Minimum Coded Units or MCUs. Each MCU corresponds to an equal area of the image, with the dimensions of this area being multiples of 8. The number of 8 8 blocks from different color planes that comprise an MCU may be different, depending upon subsampling. Subsampling details can be found in any JPEG reference, such as. 6 For our purpose, it is sufficient to summarize the structure of a JPEG image I consisting of N color planes as follows: 1. I is a sequence of coded MCUs. There are h rows of w MCUs each. The MCUs are coded in raster order. 2. Each MCU consists of m n 8 8 blocks from color component number n (0 n<n). 3. An MCU is coded by coding the m 0 blocks from color component number 0 (in raster order within the MCU), then the m 1 blocks from component 1, and so on. It is useful for us to consider this JPEG image structure in a color component-wise manner. That is, we look at the 8 8 blocks for each (potentially subsampled) color component separately, and in the relative order in which they would be found within the JPEG format. This simplifies our presentation as we do not have to refer to further details about MCUs. Let F n,k denote the 8 8 block (of quantized coefficients) numbered k from color component number n. This notation has absorbed the fact that k is not necessarily the raster index of the block, because of subsampling and MCU structure. Here, 0 n<n and 0 k<whm k. We illustrate this notation in Figure 2, which shows the three color planes of a typical JPEG image in Y,Cb,Cr format where the chrominance components (Cb and Cr) are subsampled by 2 both horizontally and vertically. The blocks lying on the pattern are shown shaded. The image width is 160, MCUs are 16 16 areas of the image. Let F n,k (z) denote coefficient number z (0 z 63) in the zig-zag order used in JPEG. The first coefficient, F n,k (0) is called the DC coefficient as it is the scaled average of all the pixel values in the 8 8block. The DC values within each color plane s blocks are differentially coded in JPEG, to exploit correlation across blocks. Thus, for k>0, the value coded in the bitstream is F n,k (0) F n,k 1 (0). Let us denote this differential DC value by D n,k, keeping in mind that D n,0 = F n,0 (0). The JPEG standard allows either Huffman coding or arithmetic coding to be used for entropy coding. We restrict our attention to Huffman-coded images as Huffman coding is the chosen alternative most of the time. In a Huffmancoded JPEG image, differential DC values D n,k are coded as follows. Given a value v to code, let s(v) denote the magnitude category of v, which is the number of bits in v up to the most significant 1. For example, s(0) = 0, s(1) = s( 1) = 1, s(3) = s(2) = s( 2) = s( 3) = 2, and so on. The JPEG bitstream consists of a Huffman code for s(d n,k ), followed by s(d n,k ) least significant bits from the 1 s complement representation of D n,k. The DC Huffman coding tables may be different for different color components, or some of them may be the same. The coding of AC coefficients (1 z 63) also uses the magnitude category function s( ). Additionally, the frequently occurring situation of long runs of zeros within AC coefficient sequences is also exploited in the JPEG format. For the sequence of AC coefficients in a block F n,k, consider the representation (R 0,V 0 ), (R 1,V 1 ),...,(R l,v l ).

Figure 2. The block ordering notation used in the paper, illustrated using a Y-Cb-Cr JPEG image with width 160 and chrominance subsampled by 2 horizontally and vertically. The thicker lines separate MCUs while the thinner lines separate the 8 8 blocks within the Y-component of the MCU. Each Cb and Cr MCU consists of only one 8 8 block. The shaded blocks are the ones lying on the watermark pattern. Here, R 0 is the length of the run of zeros leading to the first non-zero coefficient (which has the value V 0 ). This is followed by a run of R 1 zeros and then another non-zero value (V 1 ), and so on. The JPEG bitstream consists of Huffman codes for pairs of (R i,s(v i )). Each such code is followed by s(v i ) additional bits taken from the least significant positions in the 1 s complement representation of V i. The AC Huffman coding tables may also be different for different color components. In this review, we have omitted some concepts such as end-of-block codes and restart markers. These concepts are easily incorporated in the techniques we describe, but a detailed exposition would only complicate the presentation. 3. THE SCRAMBLING ALGORITHM In the scrambling process, our goal is to corrupt those 8 8 blocks which lie in the pattern area, while making sure that the decoded image will not be corrupted outside the pattern areas. We also have to ensure that this corruption can be undone by the descrambler. Let ρ(n, k) be a binary function that is 1 if and only if F n,k (block number k from component number n) lies in the pattern region. The function ρ (and hence the pattern) is not a secret, and is available to any attacker. Our scrambling algorithm efficiently produces a JPEG image I = S(I,K) such that I and I are exactly the same for MCUs where ρ(i, j) is 0, and are distinctively different where ρ(i, j) is 1. In other words, the blocks where ρ is 1 are scrambled by S. The descrambler D reverses the scrambling completely, with the right key. The scrambling algorithm consists of a set of operations. In a practical implementation, these operations are applied in a single pass through the image. Here, we present the operations separately, for clarity. The scrambling and descrambling algorithms use a common pseudo random number generator (PRNG) whose seed is the secret key. We will use the PRNG to generate masking values as well as random permutations and selections. To keep the notation simple, we will simply use the subscript PRNG to indicate that a value, permutation, or selection has been obtained by invoking the PRNG.

3.1. AC masks For each block F n,k for which ρ(n, k) = 1, we can mask the magnitude bits of the AC coefficients by XORing with a value obtained from the PRNG. Recall that the AC coefficients are coded by grouping them into run-value pairs, (R i,v i ) and that the coded bitstream consists of a Huffman code for (R i,s(v i )) followed by s(v i ) bits from the least significant positions of the 1 s complement representation of V i. Let s refer to these s(v i ) bits as B. Then, the AC masking operation consists of replacing B with B X PRNG, where X PRNG represents random values. This maintains format compliance and does not change the size of the coded image. It is evident that the descrambler can reverse this by XORing again with X PRNG. 3.2. AC swaps Since the AC coefficients are coded independently from block to block, we can swap the entirety of the AC coefficients across any two blocks for which ρ is 1. That is, if ρ(n, k 1 )=ρ(n, k 2 ) = 1, then we can exchange F n,k1 (1,...,63) with F n,k2 (1,...,63). This would not change the bit rate at all. In fact, if the AC Huffman tables for two color components are the same (as is usually the case for the chrominance components Cb and Cr in JPEG images), then the swaps can be done across these components too. In practice, we choose to do the swaps within a small number of rows of MCUs, to minimize the buffering requirements of the scrambler and descrambler. Thus, the AC-swap operation consists of selecting (using the PRNG) a number of pairs of blocks F n1,k 1,F n2,k 2 constrained to lie within a buffered amount of MCU rows and such that both the selected blocks lie in the pattern and use the same AC Huffman table. For each such pair, all the AC coefficients are exchanged. The previous two operations modify the AC coefficients. In practice, the impact upon visiblity of modifications to DC coefficients is far more than AC coefficients. The next two operations modify the DC coefficients. 3.3. DC shuffles Consider (within color component number n), a maximal consecutive sequence of l blocks F n,k,f n,k+1,..., F n,k+l 1, each lying on the pattern. This is maximal in the sense that ρ(n, k 1) = ρ(n, k + l) =0andρ(n, i) =1for k i<k+ l. The DC-shuffle operation corrupts the DC value of each block in the sequence, ensuring that the DC value for the first block after the sequence remains unchanged, and that the bits required remain the same, by shuffling the differential DC values. Recall that we use the notation D n,k to denote the differential DC value for F n,k. It is apparent that the DC value for the block after the sequence is given by Equation 1: k+l F n,k+l (0) = F n,k 1 (0) + D n,i. (1) Thus, as long as we do not change the value of the sum k+l i=k D n,i, we are free to change the individual D n,i values without affecting the DC value of the first block after the sequence. Moreover, if our changes are made by simply permuting the D n,i, then the bit rate will also be exactly the same. The DC-shuffle operation generates a random permutation σ PRNG and applies it to the sequence of D n,i values, k i k + l. Note that the differential DC value of the first block after the sequence also participates in this permutation, but the net result is that its DC value (actual, not differential) is unchanged. 3.4. DC bit-shuffles Figure 3 shows the sequence of DC values identified in the previous subsection. Here, the first l blocks are the ones to be corrupted, while the block F n,k+l has ρ(n, k + l) = 0 and needs to be left unmodified upon decompression. In the figure, we have shown 1 and 0 bit-values starting from the rightmost column (numbered 0) upto the column number corresponding to the magnitude category of the differential DC value. Thus, in the row for D n,i all columns numbered s(d n,i ) and greater are marked by X to indicate that these bits are not present in the bit-stream. i=k

Figure 3. The magnitude bits for a sequence of differential DC values. Let s look at the bit-wise composition of the D n,i values in this sequence of blocks, k i k +l. For the moment, we drop the subscript n for simplicity, and let s i denote s(d n,i ). Further, let b i (j), (0 j<s i ) denote the s i bits of the 1 s complement representation of D n,i. Then, s i 1 D n,i =[b i (s i 1) 1][2 si 1] + b i (j)2 j. (2) Here, the first term just makes sure that if the left-most bit (i.e., bit number s i 1) is 0, then the appropriate negative number is obtained (by subtracting 2 si 1). We can rewrite Equation 2 as: s i 2 D n,i =1 2 si + b i (s i 1)[3 2 si 1 1] + b i (j)2 j. (3) Observe that because of the 1 s complement representation, the left-most bit b i (s i 1) of each D n,i has a disproportionate weight. By substituting the value of D n,i from Equation 3 into Equation 1, it can be seen that the DC value of the first block after the sequence, i.e., F n,k+l (0) is a constant plus a weighted sum of all the b i (j) values, with the weight being 2 j for bits other than the left-most ones. The weight of each left-most bit is 3 2 j 1 1. Thus, we can apply bit-plane permutations to these bits, while keeping F n,k+l (0) unchanged, as long as we are careful about not mixing left-most bits with non-left-most bits. Let q denote the maximum value of s i,for k i k + l. Then, for each bit-plane j, (0 j<q), we apply two pseudo-random permutations within the b i (j) values: σ PRNG (j) andπ PRNG (j). In the first permutation, σ PRNG (j), only those bits from bit-plane j particpate which are the left-most bits. That is, only the b i (j) for which s i 1=j participate in σ PRNG (j). The non-left-most bits participate in the second permutation. That is only the b i (j) for which s i 1 >jparticipate in π PRNG (j). It can be seen that applying these 2q permutations (two disjoint permutations σ PRNG (j) andπ PRNG (j) oneach bit-plane) ensures that the DC values within the pattern areas are corrupted securely, while the DC value of the first following block is untouched. Moreover, the coded size remains exactly the same as it was prior to this operation We have now described all the four operations that are applied within our scrambling algorithm S. The descrambler D is straight-forward, as it merely has to apply the inverse operations. We have carefully designed these j=0 j=0

Figure 4. The result of applying the scrambling algorithm. operations such that they are reversible the determinations made during scrambling (such as the s(d n,i ) values) are not affected by the scrambling and hence can be repeated by the descrambler with identical results. Figure 4 shows the result of applying these four operations on the test image that we used in Figure 1. 3.5. A variant with modest size increase We have been very careful in the previous four operations to ensure that the compressed size of the scrambled image I is exactly the same as that of the original image I. Sometimes, this may not be a strict requirement, and it may be acceptable to incur a modest increase in size. In such situations, we can increase the visibility of the watermark considerably, by one additional operation. Here, we just add a pseudo random number to each DC value within the pattern region. Thus, if ρ(n, k) = 1, then we set F n,k (0) = F n,k (0) + X PRNG, where X PRNG can be positive or negative. Note that this is a reversible operation. Since we are modifying the actual DC value, the differential DC values will also change. Moreover, the changes are not guaranteed to maintain the compressed size. Indeed, as we are increasing the randomness of the DC coefficient differentials, the compressed size will increase. By placing limits on the additive value X PRNG, we can keep this size increase small. 4. VIDEO DATA In case of DCT-coded video data, such as in the various MPEG formats (for example, MPEG-1 7,8 ), motion prediction causes a problem. If we apply the still-image scrambler described here to MPEG video, then blocks outside the pattern (for which ρ is 0) may still show corruption because their motion vectors may require prediction from areas of other frames which have been corrupted. Thus, the distinctive pattern would show in the video, but there would be additional corruption outside the pattern. This additional corruption may be small or extensive, depending upon the video sequence and block-matching algorithms used during coding. One way to alleviate this problem is to only apply the scrambling to bidirectional-coded frames (B-frames) as these themselves are not used for predicting any blocks on other frames. Typically, B-frames form a significant fraction of all frames, and in such cases, the desired effect is obtained (the pattern shows up distinctively, albeit in a flashing manner). But sometimes, video sequences are coded without any B-frames at all. In such cases the only frames which can be properly corrupted (without the corruption propogating to arbitrary areas on other frames) are those unidirectionally predicted frames (P-frames) which lie just before a non-predicted frame (I-frame). 5. CONCLUSION We have presented obstructive visible watermarking schemese that leave the data format compliant. The scrambling and descrambling algorithms are very efficient and work mostly in the compressed domain they operate on DCT coefficients obtained by decoding the Huffman codes, and do not require these coefficients to be de-zigzagged or

dequantized. In fact, not only is de-zigzagging not needed, the algorithms only examine the compact sequence of non-zero coefficients within each block. A nice feature of the each of the four main scrambling operations presented is that they do not change the compressed size even by one bit. It would be interesting to see if there are any more possible operations that have this property one possibility is to make the operations intelligently use the details of the Huffman tables used in the image to make changes such that the rate does not increase. The general problem for coded video appears hard to solve; we have only sketched some restricted solutions for video. REFERENCES 1. Chen, Pei-Min, A visible watermarking mechanism using a statistic approach, 5th International Conference on Signal Processing 2, pp. 910 913, 2000. 2. Mohanty, S.P., Ramakrishnan, K.R., and Kankanhalli, M.S., A DCT domain visible watermarking technique for images, IEEE International Conference on Multimedia and Expo 2, pp. 1029 10323, 2000. 3. Meng, J. and Chang, Shih-F, Embedding visible video watermarks in the compressed domain, International Conference on Image Processing 1, pp. 474 477, 1998. 4. ISO 10918-1 JPEG Draft International Standard and CCITT Recommendation T.81. 5. Ahmed, N., Natarajan, T., and Rao, K. R., Discrete Cosine Transform, IEEE Trans. Computers C-2390-3, Jan. 1974. 6. Pennebaker, W. B. and Mitchell, J. L., JPEG Still Image Data Compression Standard, Van Nostrand Reinhold, New York, 1993. 7. MPEG I draft: Coding of Moving Pictures and associated audio for digital storage, 1991. Document ISO/IEC- CD-11172. 8. Le Gall, D., MPEG: A Video Compression Standard for Multimedia Applications, Commun. ACM 34, April 1991.