Toon Boom Toon Boom Studio: 2D/3D Integration

WHITE PAPER Toon Boom Toon Boom Studio: 2D/3D Integration An overview of the different production pipelines for integrating 2D and 3D content with Toon Boom Toon Boom Studio. Considerations when exporting and importing images, importing 3D assets to rotoscope, and the full integrated pipeline for importing a 3D asset, and rendering through Autodesk Maya using Harmony. December 2012

Legal Notices Toon Boom Animation Inc. 5530 St. Patrick, Suite 2210 Montreal, Quebec Canada H4E 1A8 Tel: +1 514 278 8666 Fax: +1 514 278 2666 toonboom.com Disclaimer The content of this document is covered by a specific limited warranty and exclusions and limit of liability under the applicable License Agreement as supplemented by the special terms and conditions for Adobe Flash File Format (SWF). For details, refer to the License Agreement and to those special terms and conditions. The content of this document is the property of Toon Boom Animation Inc. and is copyrighted. Any reproduction in whole or in part is strictly prohibited. Trademarks Toon Boom and Toon Boom Toon Boom Studio are trademarks owned by Toon Boom Animation Inc. All other trademarks are the property of their respective owners. Publication Date 2013-01-15 Copyright 2012 Toon Boom Animation Inc. All rights reserved. OWHP101HAR10EN 2

Introduction This white paper is an overview of the different production pipelines for integrating 2D and 3D content with Harmony. We ll take a look at exporting and importing images, importing 3D assets to rotoscope, and the full integrated pipeline for importing a 3D asset, and rendering through Autodesk Maya using Harmony. The type of pipeline that will be useful for you will depend on the production itself. Production There are several different 2D/3D pipelines that are possible to achieve through Harmony. 1. Exporting an image sequence from Harmony to 3D. 2. Importing an image sequence from 3D to Harmony. 3. Importing 3D objects into Harmony, then rotoscoping over them. 4. Full 2D/3D Integration with Render; all in Harmony. Both 1 and 2 can be achieved with the Animate line, as well as Harmony; 3 and 4 are particular to Harmony. 3

Exporting an Image Sequence to 3D The simplest 2D/3D pipeline is simply to export an image sequence from your 2D animation software to your 3D animation software. When doing this, it s important to export individual elements to give you the flexibility of placement in your 3D software. For example, each character should have its own image sequence. Exporting image sequences of different layers is simple to do in Harmony, as you can configure multiple Write nodes for each layer you want to export separately. Why would you want to do this? The first thing to look at is what percentage of your production is 2D vs. 3D. If the majority, say 65% or more, of your production is 3D, then this is a good option. In other words, if the majority of the shot is 3D, keep it in the 3D software. Some productions will have different pipelines for different shots too. In other words, if one shot is 100% 3D, they ll do that in the 3D software itself, then if the next shot is only 20% 3D, they ll do the integration in Harmony instead. Another consideration is the availability of hardware and software. If you will render an image sequence out of Harmony, it s very light and easy to render. You also don t need to have any software but Harmony on your machine. How does it work? If you export an image sequence, you can choose to import it directly into your 3D software. Or, you can export image sequences from both Harmony and your 3D software, and integrate everything in a compositing software. 4

Disadvantages The limitation of this pipeline is that there is no way for your 2D animator to interact with the 3D scene. They have no reference of the 3D scene, which can make it difficult for the animator to work effectively. Advantages The advantage of this pipeline is that you can work with any 3D software or compositing software to achieve this task. 5

Importing an Image Sequence from 3D It is equally possible to export an image sequence from your 3D software, or to export a movie file, and to import this image sequence or movie file into Harmony. Why would you want to do this? This is a simple pipeline to follow, because the 3D work can all be done by one team, and rendered out. The 2D team simply has to work on top of the rendered images, so there s no additional skill necessary. How does it work? Simply export an image sequence, or a movie file, from the 3D software. Make sure the camera movement is final and has been applied in the 3D file, because camera changes cannot be made on the 3D object once the image sequence has been imported into Harmony. Then, the 2D animator will simply create another layer on top and work with this layer. Disadvantages The limitations of working with this pipeline are that it is very difficult to make changes to the 3D scene, because if you need to make a change, you have to go back into your 3D software, make the change, render the sequence, and reimport it. Advantages The advantage of this pipeline is that you can see your 3D sequence as you re animating your 2D sequence, which really helps the animator achieve a natural look and feel. Also, you can use any 3D software. 6

Importing 3D, then Rotoscoping This is a pipeline that is sometimes forgotten in the world of 2D/3D integration, but can still prove extremely fruitful and interesting. One of the major challenges with the 2D/3D pipeline in general is the aspect of rendering, and the rotoscoping pipeline avoids that entirely. Instead of rendering 3D models, you trace over them for a 2D look. Why would you want to do this? Often people will choose to do Rotoscoping so they have a seamless 2D look in the end. Although there s more labour involved in tracing over the images, the end result is cohesive and accurate. How Does it Work? With rotoscoping, you need only import a proxy model from your 3D software to help the animator to achieve the correct angles, and to help them to redraw things like complicated props. However, the nice thing about this pipeline is that you don t need to spend the extra effort on the 3D side to achieve a 3D render. Disadvantages Disadvantages are that there is more drawing involved. However, with some of the improvements in the Harmony paperless workflow, such as the new True Pencil, Deform, and improvements to the Xsheet and Shift and Trace, drawing itself has become faster and easier to achieve digitally. Advantages Rendering your 3D sequence can be time-consuming to set up, because you need to texture your 3D model, then you need to set up the shaders, add some lighting into your scene, and finally you then need to perform the final render. Depending on the complexity of the scene, the lighting, and the effects, 3D render times can be quite long, and this means that you need to spend more to set up a more advanced render farm. You also have to budget to finish your production earlier in order to give yourself enough time to render your 3D scene. By rotoscoping, you re not actually rendering any 3D. You re just importing your object, then drawing on top of it. This may actually be the cheapest 2D/3D integration solution, depending on the project. Advantages are that you can use any 3D software. The animator can have the freedom to push the animation beyond the limits of 3D. Render times are extremely low. 7

Full 2D/3D Integration with Render This is the most complex pipeline to set up for 2D/3D integration, but the benefits of being able to set this pipeline up far outweigh the others. With the full pipeline, you can take advantage of creating props, vehicles, and backgrounds in 3D, and can animate your characters in 2D. You can even do the compositing and effects directly in Harmony. If you take advantage of Storyboard Pro 3D to do your pre-production, then you can also export directly to Harmony. This will save you time in setting up your scenes, and it will even automatically import any OSB files that you were working with in Storyboard Pro 3D. Why would you want to do this? If less than 50% of your production is 3D, you might want to consider this one. If you re working on a hybrid project, where characters are animated in 2D, and backgrounds, props, and/or vehicles are animated in 3D, then this could be a possible solution. You would not choose this option if you have characters, or other soft-surface deformation, in your 3D objects. This is a good option when you want the animators to be able to animate characters within a 3D environment. Because the character is in the environment, you don t have to worry about clipping issues that you would have if you were simply drawing the character on top. Say, for example, you have a character sitting at a table. If part of the character is in front of the 3D object and part of it is behind it, you can simply draw it this way in Harmony. However, if you are compositing final image sequences instead of working with the original model, you would have to go through and painstakingly erase out the parts of the character that are behind the image sequence. How does it work? There are a couple of different aspects of this pipeline to be aware of. There s Phase 1, which takes place before any rendering. Then there s Phase 2, which is where the rendering occurs. The real complication in this pipeline takes place during Phase 2. In order to preserve the quality of the render that was achieved through the 3D software, we decided to simply use the 3D software. The way this works under the hood of the software is that there is a scripting interface that allows Harmony to communicate with Maya through MEL script. Being able to communicate with Maya means that we can open a session of Maya Batch and render, which allows you to preserve the lighting, textures, and shading that were created in the original 3D file. 8

Each 3D software handles rendering differently, shaders differently, and scripting differently. For this reason, we have focused on fine-tuning the Maya pipeline. If you are using other software, however, and would like us to consider your pipeline, email support@toonboom.com to let us know. We do currently have some support using scripts for Pixie and 3Delight. Although pipelines for software other than Autodesk Maya are not officially supported, there is a possibility to work with other technology. So what do you need to know about this pipeline to make it work for you? Preparing Your Models In Maya, you should create a separate scene file for each object that you would like to animate in Harmony. When you import the object into Harmony, you will have the ability to move, rotate, and scale the object, and even animate these transformations over time. Each object must be its own Maya file, however you can take advantage of Harmony s hierarchy and peg tools to create rigs of Maya files to move them in unison. Before you import into Harmony, you should also complete the lighting, texturing, and shading of your models. If you want to create generic lighting for a Toon-shader look, then you can create some ambient lighting, and then parent the lights to your model so they move along with your model. 9

You can also make use of specific Toon shaders, like the Ramp Shader, to get a Toon-shaded look in the final render. However, these advanced shaders are not currently supported in OpenGL of Harmony, so it s recommended to save two versions of these models; one with the Ramp Shaders intact for rendering, and one with basic shaders like Lamberts for importing into Harmony. They should both be exactly the same, apart from the shaders. The size and positioning of the models should be the same. If you are using Harmony 9, then you need to obtain the appropriate plug-in for your version of Maya, so you can export an OSB file format from Maya to import into Harmony. Refer to the User Guide for more information on how to export your model. If you re using Harmony 10, then you can simply export an FBX file for import into Harmony. You need to also save your original.mb file, since this is the file that Maya will use to render the object during the render phase. Animating in Harmony When you create a new Harmony file and import an FBX or OSB model, you can resize, reposition, and scale the object with the Transform tool. Just make sure to turn on Enable 3D, following the instructions in the User Guide on how to manipulate the 3D model. During the Animation stage of your project, it s not necessary for the animator to use Maya at all. They can import the object, move it around, and create 2D layers to draw on, without needing Maya. With Harmony 10, the animator can even export an OpenGL render of their animation on top of the OpenGL version of the model, without needing to use Maya. This means that the only computers that need Maya on them are the ones that will do the final render. 10

Setting up the Render Pipeline The render side of things requires some setup in order to get it fine-tuned. First, on the machine on which you want to perform the render, you should have both Harmony and Maya batch installed. Note that you only need a full version of Maya installed if you want to open your original Maya file and adjust the model, lighting, or shaders. In addition to having both softwares available on the machine, you also need to make sure that you have obtained a 3D licence from Toon Boom. Contact your sales rep, sales@toonboom.com, or support@toonboom.com, to obtain this license. The 3D part of the license is included automatically with Harmony 10. Next, you need to set up an environment variable in order to open up a communication port between Harmony and Maya. For instructions on how to set this up, refer to the User Guide. This process is a little more technical, so if you need assistance, don t hesitate to contact support@toonboom.com For any objects that you ve imported into your Harmony file, you need to also copy the original.mb file into the same location, so Harmony can find the file it needs to complete the render. The file should be renamed to match the name of the element in your Harmony folder. You should also copy the texture files into this location as well. Follow the instructions in the User Guide to copy these files correctly. Now that you have both softwares installed on the machine, the right licenses, and you`ve opened up communication between them, you re ready to open Harmony. Open a scene file in which you ve imported some 3D models. The next step is to import into Harmony the script module that will tell Maya to render each 3D object. For more information on how to import the script module, refer to the User Guide. With the setup complete, you can test the render by clicking the Render View button to render the current frame. If all is successful, you will see the 2D and 3D objects rendered and composited together automatically. If your 3D object does not render, it could be one of a couple things. First, if your 3D object does not show up at all, you may not have your 3D licence installed. Second, if the 3D object shows up in OpenGL, but does not render, then there is probably a problem with the handshake between Harmony and Maya. Verify that you have both softwares on the machine, and that the environment variable is set up, and that you have imported the RenderMayaBatch script into your Network View according to the instructions in the User Guide. Finally, check to make sure that you have copied the original.mb file and the textures into the elements directory of the object. If all else fails, contact support@toonboom.com for further assistance. 11

System Requirements Be aware that when you re working with a pipeline that combines 2D and 3D rendering, your system requirements will be higher than they would be if you were just doing 2D alone. Make sure that you fulfill the system requirements for both Harmony, as well as Maya system. It s also recommended to test the pipeline on one system before deploying on a render network. Setting up a Render Network Just like when you re working with Harmony in 2D, you can also set up your project to render over the Render Network. There s no additional setup necessary in terms of what you need to do in Control Centre, but be aware that you need to perform the setup to confirm the handshake between Harmony and Maya on each individual machine first to verify that it works. When the pipeline works in Standalone mode, or at least in the Render View of each machine, then you can go ahead and render over the render network. Working with Other Pipelines We also have the ability to render through Pixie or 3Delight. So if you can export from your 3D using these scripts, then we can investigate that pipeline with you. However it is highly recommended to work with the Maya pipeline, since this pipeline has been thoroughly investigated and tested. 12