Visual Storytelling through Lighting Sudeep Rangaswamy Pixar Animation Studios 1001 W. Cutting Blvd., Richmond, CA

Transcription

1 Visual Storytelling through Lighting Sudeep Rangaswamy Pixar Animation Studios 1001 W. Cutting Blvd., Richmond, CA Overview Lighting plays a pivotal role in defining the look of a film. It adds visual depth and complexity to a scene, thereby heightening its dramatic impact. While the issues in lighting for a live-action film have often been examined, this paper will investigate the lighting process for computer animation. Specifically, we will discuss: where lighting fits into the production pipeline of a computer animated film, what comprises a computer graphics (CG) illumination model, and how lighting artists can use this model to enhance a story. Using examples from Pixar s A Bug s Life and Toy Story 2, we will demonstrate how lighting can be used as a powerful and effective tool in visual storytelling. The Production Pipeline In order to understand the role of lighting in computer animation, we must first determine where lighting fits into the production process. Each scene in a computer animated film goes through several stages, evolving into a finished product. At Pixar, these stages are: story/art, modeling, shading, layout/set dressing, animation, effects, lighting, and rendering. In the story/art phase, the concepts for the film are fleshed out in the form of storyboards. This phase is also where the film is broken down into shots. A shot is a contiguous set of frames, where the camera is either stationary or in motion. Whenever there is a cut to another camera in a film, it signifies a new shot. Once the shots are created, they move through the subsequent stages in the pipeline. Figure 1 shows a color storyboard from A Bug s Life, where the hero, Flik, is about to confront the evil Hopper. Figure 1: A storyboard from the Story/Art phase. Disney/Pixar.

2 In this case, the artist has already considered some of the color and composition issues that will come into play in later stages. Note the use of orange and red, denoting a nearby flame as the dominant light source. Also note that the scene is generally dark, giving it a spooky, ominous tone. We will revisit these issues when shots go through the lighting process. Once the story is in place, we need to actually create the characters and sets. This work is done in the modeling stage, where 3D models are designed and built for the production. When a character is modeled, the points of articulation are also defined, so that they can be manipulated by animators. Figure 2a shows a 3D model of Flik. After a model is created, the shading department defines its surface characteristics. A shader is a piece of code that describes the surface of a model, and how it responds to light. Because all of the subtle details of an object are programmed into the shader, the object maintains a degree of consistency from one scene to the next. Figure 2b shows a rendering of the Flik model with its shader. Figure 2: a) (left) A three-dimensional model of Flik. b) (right) The same model rendered with Flik s shader. Disney/Pixar. Now that we have a set of models and shaders, the shots themselves can go into production. First, each shot goes through a layout/set dressing phase, where the shot is populated with a camera, characters, and props, which are then positioned in the appropriate setting. The motion of the camera and the characters is blocked out in this phase as well. The layout of a shot defines its visual composition. It provides a general notion of what the viewer will be focusing on and which objects will be grouped together in the scene. Figure 3 shows the confrontation between Flik and Hopper after layout has been completed.

3 Figure 3: Flik vs. Hopper (after the layout stage). Disney/Pixar. Next, the shot is animated, and the characters are brought to life. The animator focuses on conveying the action and dialogue in the shot. The motion that was blocked out in layout is also made fluid and expressive in this stage. Figure 4 shows the animated version of our scene. The shot has taken shape here, but its visual details have not yet been finalized. Figure 4: Flik vs. Hopper (after the animation stage). Disney/Pixar. If a shot requires any special tricks, it proceeds to the effects phase. In our example shot, the flames in the background are a special effect. Figure 5 shows the results when the flame element is added into our scene. Note that the shaders have also been applied, so we can now see the surface detail on Flik and Hopper. Figure 5: Flik vs. Hopper (after the effects stage). Disney/Pixar.

4 Finally, we enter the lighting phase. Here, a lighting artist sets up and positions digital lights in the environment. Lighting is the last stage before the shot is actually rendered. As a result, it synthesizes many of the concepts introduced in the earlier stages. Figure 6 shows our shot in its final form, after lighting is complete. Note how the composition of the shot is accentuated by the lighting: Flik and Hopper now stand out from their background. The lighting also reinforces the tone set by the initial storyboards. With its reddish hue and low illumination, this scene now has a heightened sense of drama, making the climax of the film more exciting for the viewer. Figure 6: Flik vs. Hopper (after the lighting stage). Disney/Pixar. Once a shot is lit, all of its frames can be rendered and converted to film. This entire process is repeated for all of the shots in the movie. Finally, the shots are assembled in order, and the movie is complete. With an understanding of lighting s role in production, we can now examine the building blocks used by the lighting artist. The Illumination Model In live-action, the director of photography has a variety of lights that he or she can manipulate in any given scene. In the world of computer graphics, there is an analogous set of lights used by the lighting artist. We will present a common set of these lights, and discuss how their controls give the artist a great deal of flexibility. Often, a scene contains an overall base light that is used for flat illumination across the set. This is an ambient light, which generally has low-intensity. Ambient lights are used primarily for background lighting, to prevent objects from appearing completely dark. The main source of light in a shot is the key light, which is used to define the subject. Because the key light is the dominant light source, it often emanates from the direction of the camera. The key light models the surface it hits,

5 accentuating some parts and concealing others. It is often used to emphasize the prominent characters of a shot. Typically, a key light illuminates two-thirds of a character, as shown in Figure 6a. A fill light is a low-intensity light that fills an area that is in shadow or otherwise too dark. Fill lights are diffuse, which means that they give off a light that does not make objects look shinier. In computer graphics terms, these lights have no specular component. A back light is used to separate a character from its background. The most common form of back lighting is the rim light, which is usually positioned behind a character and slightly above the horizon. The rim light generates an outline around the character that makes it pop out of its surroundings, increasing its on-screen readability. Figure 7 shows how key, fill, and rim lights contribute to the overall look of a character. Figure 7a shows Woody illuminated only by a key light. In Figure 7b, we see Woody lit with just a fill light, while Figure 7c isolates a rim light. When all of the lights are used together, Woody has a sense of depth and realism. Figure 7: (left to right) a) Key light. b) Fill light. c) Rim light. d) All lights. Disney/Pixar. A few other lights are often used to illuminate characters. A kicker light defines edges on the opposite side of the key light. Kicker lights are generally used to make a character look more three-dimensional. Bounce lights can simulate reflections of an area light on nearby surfaces. In a radiosity render, the reflections between surfaces are automatically computed, but it is often desirable to tweak the amount of light being reflected onto a character. A bounce light is positioned near a character to essentially fake a light reflection. In Figure 8, we see all of the above lights used to illuminate Flik. Notice how each light contributes to the final image. As shown here, lighting in a computer graphics environment consists of setting up several lights, each with a subtle effect on the models. The amalgam of these lights produces the overall look of the scene.

6 Figure 8: (left to right) a) Key light. b) Fill light. c) Left rim light. d) Right rim light. e) Kicker light. f) Bounce light. g) All lights. Disney/Pixar. While we have described a fundamental set of lights above, any light possesses a set of attributes that an artist can control. The most obvious controls are the properties of the light itself: its color, intensity, specularity, etc. We can also control its direction. A light can be defined as radial, which means its rays travel in all directions. These lights are also called point sources. Conversely, a solar, or parallel, light source casts rays only in a specified direction. The shape of a light can also be manipulated. In the real world, barn doors are attached to the sides of a light to create a shaft. Synthetic lights can also cast rays in shaft, as shown in Figure 9a. We can also cast light in a cookie-cutter outline to create specific shapes, as shown in Figure 9b. Figure 9: a) (left) A barn door light. b) (right) A light projected through a cookie-cutter outline, producing a star-shape. Disney/Pixar.

7 Some attributes of digital lights can defy the realities of the physical world. A CG light has no actual geometry, so it can be positioned anywhere and remain invisible. Every physical light has a dropoff rate, which describes how the light attenuates. But a digital light can have a dropoff rate of zero, which means it will never fade off, no matter how far away it is positioned. In fact, a light is just a function that modifies an object s surface characteristics. Thus, it is really a specific kind of shader, and can be manipulated as such. Digital lights can also be turned on and off on a per-object basis. In other words, a lighting artist can create a set of lights that shine exclusively on a character and not on his surroundings. There is no analogue for this behavior in the real world. Furthermore, a computer artist can change the direction from which shadows are cast. He or she can even prevent lights from generating shadows. These controls give the CG lighting artist an astonishing amount of flexibility. Since lighting attempts to mimic the real world, however, it generally adheres to physical laws. Visual Storytelling We can now investigate how lighting develops and enhances a story. The role of lighting can be broadly classified into the following categories: Directing the viewer s attention Establishing a mood and atmosphere Creating a sense of depth Maintaining visual continuity The primary goal of lighting is to direct the viewer s eye. Lighting organizes the objects and characters in the larger scene, identifying the areas of interest. Effective lighting can transform a busy scene with many characters into one that is easy to read and understand. Often, emphasis is placed on a character by contrasting him with his surroundings. In Figure 10, notice how Hopper is set apart from the background by the shaft of light shining on him. While the background might contain some interesting details, the lighting clearly identifies the subject of this shot. Also notice the effective use of back lighting, which makes Hopper stand out even more. By synthesizing many of the concepts presented earlier, the lighting in this scene attracts attention. Figure 10: The contrast in lighting emphasizes Hopper. Disney/Pixar.

8 Scale and balance can also direct the viewer s eye. In Figure 11, Dot is shown standing on a leaf. Her shadow indicates to the viewer the tiny scale of the insect world. Also notice how light is beaming down from the sky above. The overall balance of sunshine makes the leaves seem more translucent. These details provide the viewer with information that is not communicated directly in the film. Figure 11: Scale and balance help convey a bug s view of the world. Disney/Pixar. The previous examples of rim lighting showed how a shape can grab the viewer. This notion is further demonstrated in Figure 12. Here, Emperor Zurg is primarily back lit, making him pop out from his background. Because the other lights in the scene are minimal, we see a near-silhouette of Zurg, making him even more menacing. Figure 12: Light shaping can make a character stand out. Disney/Pixar. Mood and atmosphere are also established by effective lighting. The use of color can be essential to setting the overall tone for a scene. Often, red lights are used to excite the viewer, while green denotes a more calm setting. Tranquil scenes are often lit in high-key style. This means that soft lights bathe the set, and there are few shadows and little contrast. The scene is well-lit, giving a sense of

9 peacefulness to the viewer. Figure 13 is an example of high-key lighting. In contrast, low-key lighting is dark, with only a few areas of activity. This style of lighting is often used to indicate danger, like in the confrontation scene shown in Figure 6. Figure 13: High-key lighting reinforces the tranquil setting. Disney/Pixar. The use of shadows also enhances the mood of a scene. Hard, distant lights can create crisp shadows, which often indicate a cold, sterile environment. At the other end of the spectrum, soft lights are used in warm settings because they create faint, barely noticeable shadows. The shadows in Figure 14 help evoke a dark, lonely atmosphere. Here, the television s glow is the primary light source, and the sleeping Al casts an imposing shadow on his surroundings. Figure 14: The shadow s in Al s apartment evoke a sense of loneliness. Disney/Pixar. Lighting can also convey depth to the viewer. In most scenes, objects can be grouped into planes of interest, such as the foreground and the background. We automatically perceive smaller objects in the background as further away. Lighting can be used to further isolate these planes, as shown in Figure 15. Notice how the background is lit evenly, allowing the viewer to easily group the ant colony together. In addition, the foreground plane is blurred, which focuses

10 the viewer on the action in the background. Blurring different planes in a scene simulates what our eyes do naturally when we focus. The interesting objects remain clear and grab our attention, while the periphery is blurred. Figure 15: Lighting can add depth to a scene. Disney/Pixar. Finally, lighting can be used to create visual consistency in a film. It is important that a logical sequence in a movie maintain a consistent look, where all the scenes at a given time and location are similar. A set of shots that have similar lighting are automatically grouped together by the viewer. If viewers have difficulty making these groups, they may be confused by the flow of the movie. At Pixar, logical groups of shots are organized into sequences. These sequences are lit collectively, in a process called master lighting. Master lights are a base set of lights for each sequence, taking into account the time, weather, and location. These lights are generally used to illuminate the set, and not the characters. A lighting artist then develops area lights, which are a modified copy of the master lights. Area lights are used over several shots in the sequence, to light both characters and props. Finally, in the shot lighting stage, most of the character lights are added, and tweaks are made to the master and area lights so that they work within a specific shot. This hierarchy of lights preserves the overall look of a sequence from one shot to the next. Conclusions In the production of A Bug s Life and Toy Story 2, we have learned a great deal about effective lighting techniques. While these techniques may be different than the ones used in live-action, the role of lighting remains the same: to engage the viewer and enhance the story. In the world of computer animation, an effective lighting artist keeps in mind the goals of the story when utilizing his or her vast set of lighting tools. With the vocabulary of techniques we have described, the artist can add impact to the story, making it more resonant with the viewer. And in doing so, the lighting artist truly becomes a visual storyteller.

11 Acknowledgements Much of the material presented in this paper is covered in Advanced Renderman: Creating CGI for Motion Pictures [1]. Specifically, Sharon Calahan s chapter, Storytelling through Lighting, a Computer Graphics Perspective, and Ronen Barzel s chapter, Lighting Controls for Computer Cinematography, provide a more in-depth study of this subject. Special thanks to the production teams of A Bug s Life and Toy Story 2, who created all of the beautiful images included in this paper. [1] Apodaca, Anthony A. and Gritz, Larry. Advanced Renderman: Creating CGI for Motion Pictures. Academic Press, 2000.