SoftCel - An Application of Raster Scan Graphics to Conventional Cel Animation

Transcription

1 SoftCel - An Application of Raster Scan Graphics to Conventional Cel Animation Garland Stern Computer Graphics Lab New York Institute of Technology Old Westbury, NY Abstract The system described uses frame buffers to provide digital analogues to the traditional processes of copying pencil drawings onto cels, painting them, and photographing them. The copying process is replaced by scanning the pencil drawings with a television camera and digitizing the video signal. Several image processing operations are performed on the digitized drawings to condition them for the next step, interactive painting of the drawings mediated by a computer program. The painted drawings are then combined with each other and painted backgrounds for frame by frame recording on film or video tape. Keywords: cel animation, paint, raster scan graphics, image scanning CR categories: 8.2, 3.41 Introduction Of the many steps in the process of conventional animation, the animation proper is performed by two levels of artists - animators, who prepare rough sketches showing the extremes of motion of a character or object; and assistants, who clean up the rough sketches and provide inbetween sketches to fill in the motion. The results of the assistants' work is a collection of pencil and paper drawings. Before these drawings can be combined with others and photographed, they must be copied onto clear acetate cels and painted. Unpainted areas of acetate allow the background or characters from other levels of cels to show through giving a fully colored image when the cel/background sandwich is photographed. SoftCel, the system described in this paper, does not attempt to provide assistance to the creation of animation proper, but rather a replacement for the phase of conventional animation known as 'ink and paint' wherein animation is copied onto cels and painted. It also provides means to combine painted drawings and backgrounds for recording on either film or video tape. A common approach to providing computer assistance to animators had been interpolation based inbetweening systems (Refs. i, 2, 3, 5). These systems are useful when large n~bers of inbetweens are to be generated, but if only one or two, or worse, no inbetweens are to be produced they become impractical due to the length of time necessary to diaitize the lines comprising key frame drawings. SoftCel derives partly from the authors prior attempt to automate the digitization process by extracting contours from digitized video images (Ref. 4). It also derives partly from a form of filling algorithm, described below, which was used in an early version of Catmull's Tween system (Ref. i) to provide coloration of machine generated inbetweens. Scanning Drawings The conventional step of copying an animator's drawing onto a cel is replaced in SoftCel by scanning it with a television camera and digitizing the resulting video signal. Other methods of image digitization could provide more resolution and/or precision but at the cost of either longer digitization time or fixed drawing size. Ideally, image input to the system should be as fast as the conventional step of copying pencil drawings onto acetate cels via xerography. Frame buffers are used throughout SoftCel for temporary storage and viewing of images. The form of frame buffer used at NYIT is an array of 8-bit pixels 512 wide by 480 high which produces red, green, and blue video signals for a color television monitor. The values of the three color components are determined from the 8-bit pixel values by lookup in three 256-element tables, known collectively as the color map. When a pencil drawing is initially digitized it may contain the full range of pixel values and the color map used is a linear ramp from zero to maximtmn value in all three components, thus producing a black and white image. In order to provide for interactive painting of a drawing it is necessary to reduce the nt~ber of pixel values used for representation of lines. In the extreme, a single pixel value could be used to indicate presence of a line leaving the other 255 values available to represent distinct colors. Practically, however, this would result in a line that was ragged or 'jaggy'. To prevent jagginess it is necessary to represent lines by several shades of gray or color. It simplifies implementation if the n~ber of gray shades retained is a power of two in which case the 8-bit pixel may be partitioned into some n~ber of bits, say n, representing 2n-i colors available for painting and.^l.transparent 'non-color' each of which has 2 to-n) different shades of gray. A tra ACM O / $00.75 See Copyright Pg. 284

2 fl~off is involved here - using more bits for gray shades reduces the n~nber of colors available for painting, and using less bits for gray shades increases the ragged appearance of lines. The compromise taken in SoftCel is to use 4 bits for each - the 4 least significant bits of each pixel represent gray shade and the 4 ndst significant color. Ordinarily, the 16 color map values for each paint color are constructed by componentwise interpolation from the red, green, and blue values for the desired color to zero. If colored lines are desired, the interpolation may go to the component values for the desired color of the line rather than zero. Color maps constructed in this fashion are called tint fill type color maps. All lines in any given drawing must be of the same color, but as noted already, several drawings with different colored lines may be combined into a single frame at record time. Enhancing Image Quality Drawings scanned with a vidicon can be processed to improve the quality of the lines and to make them more suitable for painting. Fig. la shows a digitized pencil drawing with a gray scale color map. The strip at the top of Fig. la is a linear ramp of pixel values from 0 at left to 255 at right. Fig. ib shows the same image displayed with a tint fill type color map. The ramp at the top of the picture clearly shows the 16 separate ranges of pixel values corresponding to 15 colors and a transparent range. It can be seen that the background area of the image exihibits shading from purple to black to orange producing irregular contours. The shading and contours indicate that the background white of the paper is represented by a wide range of pixel values. This undesirable effect can be produced by uneven lighting of the drawing or by nonuniformity of response over the surface of the camera's vidicon tube. Video cameras provide adjustments to reduce the amount of this error, called shading error, but it is not necessary to eliminate it in the video signal. Shading error can be compensated for by scanning a blank sheet of paper of the same kind used for the drawings and at each pixel calculating a correction factor equal to 255 divided by the digitized value. Each drawing scanned subsequently then has each pixel value multiplied by the corresponding correction factor in the blank reference scan. Figs. 2a and 2b show the image of Fig. 1 after this shading correction is applied. The effect of the correction is more noticable in Fig. 2b which uses the tint fill type color map. Inherent in the use of video as image input is a certain amount of blurring of the image. To compensate for this effect the next operation performed on each drawing is an edge enhancement operation, in particular, subtraction of the image's Laplacian from itself (Ref. 8). The effect of this operation is shown in Figs. 3a (before) and 3b (after). 2The ~p~acia~ is a derivative operator defined as d f/dx +d f/dy. In discrete form, if the neighborhood of a pixel, a, appears as bcd eaf ghi then the value of the Laplacian at a is c+e+f+h-4a. The value of the enhancement operation performed in Fig. 3b at pixel a is 5a-c-e-f-h. It can be seen that in addition to enhancing lines, the Laplacian operation also enhances salt and pepper background noise. Before the drawing is ready to be painted, this noise must be removed and the range of pixel values must be reduced from the initial 256 to 16. It is also possible and desirable at this point to increase contrast of the lines and to make sure that the regions to be filled are as flat as possible. Our intent is to find two pixel values, pl and p2, pl < p2, to define a function, f(p), which will map the pixel values onto the 0-15, transparent, range as (0, if p.< pl f(p) =~ 15(p-pl)/(p2-pl), if pl < p ~< p2 ( 15, p > p2. Fig. 4 shows histograms of the image of Fig. 1 taken at each stage of processing. Fig. 4a is the histogram of the original scanned image. The clear, single peak corresponds to the pixel values of blank white paper. Fig. 4b is the histogram of the image after the shading correction is performed. The peak has shifted to the right and has become narrower reflecting the fact that the pixel values of the blank white paper have become more uniform over the image. In Fig. 4c, the histogram after the Laplacian operation, the peak is seen to widen, reflecting the enhancement of the background noise. If the areas to be filled are to be flat then p2 must fall to the left of the peak. The almost monotonic nature of the histogram from 0 to the maximt~nof the white peak indicates that a condition on the slope of the histogram might provide a criterion for choice of p2. Experimentation has shown that a reasonable way to choose p2 is to start at the top of the White peak and search left until the slope of the histogram is less than 8 for 16 consecutive pixel values. Since the value of the histogram at p2 gives an indication of how dense the paper is with lines it seems reasonable to relate pl to it. Accordingly, pl is taken to be the pixel value at Which the ntmnber of pixels with value less than or equal to pl is equal to the value of the histogram at p2. Painting the Drawings The interactive coloring of a drawing is mediated by a specialized painting program which Offers the user 3 basic tools - tint fill, tint paint, and eraser or value paint. Tint fill is a filling algorithm developed by Alvy Ray Smith at NYIT (Ref. 6). The operation of the algorithm may be described briefly as follows. Given one of the 16 colors for filling and a 'seed' or starting pixel I) change the color bits of the pixel to the fill color, 2) for each neighboring pixel left, right, above, or below, if the color of the neighbor is the same as the original color of the seed and the value of the gray shade bits is less than or equal to the value for the seed, then recursively apply the algorithm with the neighbor as the new seed. In effect the tint fill algorithm fills 'downhill' to the darkest pixel in the center of a line but will not proceed 'uphill' to the region beyond the iine. 285

3 Occasionally, gaps are present in lines which would cause the tint fill algorithm to appear to leak into a neighboring region. Tint paint is provided for the user to apply a different color on one side of the gap thus creating a boundary that tint fill will not cross. Tint paint is implemented as a digital paintbrush that changes only the color bits of a pixel. Eraser or value paint is provided for the erasure of stray marks, animators notes, or noise that may have escaped the processing performed. It is complementary to tint paint in that it changes only the gray shade bits of a pixel. The limitation of 15 colors is unacceptable for assembly of frames ready for recording. For this reason finished frames are assembled into a set of three frame buffers, each frame buffer supplying only one color component. As a result, the final assembled image can contain an arbitrary n~ber of colors, while each individual cel is limited to 15. When the painted drawing is restored into three frame buffers, the transparent range of pixel values is treated specially to avoid a jagged edge effect. Pixel values of 15 in the painted image correspond to completely transparent areas and thus do not result in any modification of the result image. Pixel values less than 15 correspond to parts of lines at various densities and the corresponding pixels of the result image are changed by interpolation of the existing value towards black, the interpolation paraneter depending on the density. Pixel values greater than 15 simply replace the corresponding pixel in the result image. In conventional animation, there is a practical limit to the ntmlber of cel levels used due to the fact that an unpainted cel is not completely transparent. With digital cel assembly, this limitation does not exist. Fig. 5 shows the image of the previous figures in final, painted form, and Fig. 6 shows this image combined with a background. thank Earl James and Lance Williams. I also wish to express thanks to Dr. Alexander Schure for supporting this research in particular, and computer animation in general. References [i] Catmull, Edwin, Personal commlmication, Spring [2] Diment, A. R., "Computer Animation", System s, December/January [3] Kitching, Alan, "Computer Animation - Some New ANTICS", British Kinematography Sound and Television, December [4] Stern, Garland, GAS - A System for Computer- Aided Keyframe Animation, PhD Dissertation, University of Utah, Spring [5] Wein, M., and L. Bertnyk, "Computer Generated Key-Frame Animation", Journal SMPTE, March [6] Smith, Alvy Ray, "Fill and Tint Fill", NYIT Graphics Lab Technical Memo No. 6, July [7] Smith, Alvy Ray, "Paint", NYIT Graphics Lab Technical Memo No. 7, July [8] Rosenfeld, Azriel, and Avinash C. Kak, Digital Picture Processing, Academic Press, New York, Summary SoftCel has been in use for well over a year in several productions. Its design was intended to have no unnecessary impact on the production of animation up to the point at which it is scanned into the system. This is largely the case, however assistants and inbetweeners are encouraged to use a uniform line, and not to leave breaks in lines, because of potential leaks using tint fill. The major problem with the system is the 15 color per cel limitation. This problem has led animators to work with more levels and to restrict the number of colors used in the design of characters. Several approaches are available for solution of this problem, the most readily available being reduction of the n~ber of bits per pixel allocated for gray scale. It may be that 8 gray shades are adequate for line representation, although this may also necessitate other enhancement operations. Acknowledgements I wish to express my appreciation to my colleagues at NYIT for their help and encouragement in this research and preparation of this paper, in particular Alvy Ray Smith and Ed Catmull. For the illustrations used in this paper I would like to 286

4 Figure la Figure Ib IIIIIII Figure 2a Figure 2b Figure 3a Figure 3b 287

5 L28 ~8 255 Figure 4a Figure 4b c 6s 28 2s5 [28 p xol V Iu Figure 4c Figure 4d Figure 5 Figure 6 288