A.R.I.S.E. Architectural Real-time Interactive Showing Environment. Supervisor: Scott Chase

Transcription

1 A.R.I.S.E Architectural Real-time Interactive Showing Environment Supervisor: Scott Chase Kasper Grande, Nina T. Hansen Kasper J. Knutzen, Anders Kokholm Long T. Truong

2

3 Contents A Unity 2 A.1 Unity in General A.1.1 The Unity Editor A.1.2 Scripting in Unity A.2 Lightmapping A.3 First Person Controller A.4 Sound Design A.4.1 Theory A.4.2 Sound Scripts A.5 Opening and Closing Doors A.6 Glass Shader A.6.1 Theory A.6.2 GLSL Shader A.6.3 GLSL Code Explained A.6.4 CG Code Explained A.7 Line of Sight A.8 Day and Night Cycle B 3DS MAX 37 B.1 The House B.2 Texturing Roof and Walls B.3 Furniture C Test 1 43 C.1 Process and Decisions C.2 Test 1 Material List C.3 Questionnaire and Meaning C.4 Test 1 Questionnaire C.5 Test 1 Data C.5.1 Demography C.5.2 Test Subject C.5.3 Test Subject C.5.4 Test Subject C.5.5 Test Subject C.5.6 Test Subject

4 Contents Contents C.5.7 Test Subject C.5.8 Test Subject C.5.9 Test Subject D Test 2 58 D.1 Questionnaire and Meaning D.2 WebPlayer Frontpage D.3 Test 2 Survey Questionnaire D.4 Test 2 Survey Data D.5 Test 2 Architect Questionnaire D.6 Test 2 Architect Data Group mea of 86 Aalborg University

5 Portfolio A Unity A.1 Unity in General Unity is a software for creating 3D video games or other interactive content such as architectural visualizations or real-time 3D animations. Unity consists of both an editor, for developing and designing, and a game engine for compiling and executing the final product. The software is free to download with an option to upgrade the version to Pro licenses. The Pro license benefits additional features that the free version would not have, such as render-to-texture, occlusion culling, global lighting, and post-processing effects. (Technologies [2011b]) A.1.1 The Unity Editor The Unity Editor has an integrated graphical environment as the primary method of development, which means the Editor is composed of several of useful windows for developing and designing. There are five main important windows; the Project folder, the Hierarchy folder, the Inspector window, the Scene window, and the Game window. The Project folder handles all the physical asset files of the project, sort of speak. Meaning, all asset data that are relevant for the project is stored on the computer s hard drive, and is easily accessible through the Project folder. Assets are files that can be imported from other applications and used in the Unity Editor, such as images or 3D models. In the Project folder, assets are template objects which means they are the objects which can then be copied and used in the Hierarchy for creating the specific objects needed. Figure A.1 shows the Project folder in the Unity Editor. It shows a simple project named HelloUnity2 with a scene file, material assets used for texturing objects, and three folders with assets inside. Figure A.2 shows the asset files as located on the hard drive. All the same files seen in the Project folder in the Unity Editor will be stored here, as this is the path for the asset folder on the hard drive.

6 A.1. Unity in General Portfolio A. Unity Figure A.1: The Project folder in the Unity Editor. Figure A.2: The Project folder as files located on the hard drive. The Hierarchy folder handles all the objects in the world scene. An object can be of many forms, such as normal geometry, light sources, texture, or audio. However, it is also possible for an ob- Group mea of 86 Aalborg University

7 A.1. Unity in General Portfolio A. Unity ject to be empty, which means it simply has not been specified the type of object. This is useful for assigning the type later when needed, or for non-depending world positions specific scripts. Figure A.3 shows the Hierarchy folder, with the same example project as seen in Figure A.1. In this case, the Hierarchy folder contains two cube objects, a main camera, and a couple of wall objects, along with a mesh which acts as a light source in this case. Figure A.3: The Hierarchy folder which gives an overview of all objects in the world scene. The Inspector window is an essential part of the development process in Unity, as it is the one that handles all the settings for each individual object in the world scene. Figure A.4 shows the Inspector window, in this case, one of the Wall side object has been selected. Here it is possible to transform, scale, or rotate the object. Along, there is a mesh renderer and box collider attached, and additionally a material to shade the wall side. The Scene window is the primary viewport for development, since the window makes it easy for navigating and orient the objects in the world scene. Figure A.5 shows the Scene window, with the objects placed in the world scene. In this case, the two cubes are inside the surrounding walls and a camera is placed outside in front of the opening. The example also shows that the red wall side is selected, since there is a green bounding box around the wall and a corresponding gizmo is visible as well. It is also possible to align the view port whenver needed, meaning the gizmo, in the top right part of the scene, is clickable and will align the view for which ever axis was clicked on. The Game window is a preview of what the cameras will see in the world scene. In this case, Group mea of 86 Aalborg University

8 A.1. Unity in General Portfolio A. Unity Figure A.4: The Inspector folder which gives an overview of all the settings and attached scripts for the specific object. the camera that is deployed, sees the two cubes and the walls from the opening. It is not possible to see what objects are selected or transform them in the Game window, as it is only a preview. A.1.2 Scripting in Unity The Unity development environment provides options to modify and implement scripts and shaders. The scripts can be written in Javascript, C#, or Boo. The scripts in this project is written in Javascript, as Java is the programming language the group is familiar with. Scripting inside Unity consists of attached custom script objects called behaviours to game objects. Different functions inside the script objects are called on certain events. The most used method is called Update(), which will be called before rendering a frame. This is where most game behaviour code goes. Code outside any functions is run when the object is loaded. This can be used to initialise the state of the script. It should also be noted that the Unity API script documentation Group mea of 86 Aalborg University

9 A.1. Unity in General Portfolio A. Unity Figure A.5: The Scene window is the viewport for navigating and orient objects in the world scene. Figure A.6: The Game window is the viewport for the final product. was heavily utilized. Technologies [2011d] There are also specific scripts that are used often in cooperation with other scripts. These scripts handle the common but vital operations such as position of object and player, or whether the character is inside a collider box. The following sections describe the source code of those common operations. Object and Character Position Two scripts were written to handle the position of objects and the character. Listing A.1 shows the script for Object Position, which gets the attached object s position and return the coordinate values whenever called. In the first line, a simple variable declaration is made of the type Vector3, this is to hold the position in the 3D world space. Inside the Update() method in Group mea of 86 Aalborg University

10 A.1. Unity in General Portfolio A. Unity line 5, the created variable is now being continously updated by getting the attached object s transform position in the 3D world space. Line 8 gives the option to return the values of object position, for use in other scripts depending on the object s position. Listing A.1: Object Position 1 var _objectposition : Vector3 ; 2 3 function Update ( ) { 4 5 _objectposition = transform. position ; 6 } 7 8 function getobjectposition ( ) : Vector3{ 9 10 r e t u r n _objectposition ; 11 } Listing A.2 shows the Player Position, which is very similiar to the Object Position. The main differences are this is for the character s position and the script is only attached to the First Person Controller, thus the variable is continously updated with the character s position and can be returned whenever called. Listing A.2: Player Position 1 var _playerposition : Vector3 ; 2 3 function Update ( ) { 4 5 _playerposition = transform. position ; 6 } 7 8 function getplayerposition ( ) : Vector3{ 9 10 r e t u r n _playerposition ; 11 } Collider Triggers Another script that is heavily used, is the Collider Trigger script. This script checks whether the character is within a collider or not. This is useful for triggering specific events when the player is inside a collider. Listing A.3 shows the collider script. Starting with the first line, a variable declaration of the type boolean is made, to check whether the character is inside or not. Note that in line 3, the Update() method is empty, that is because no operations besides the trigger checking is needed. And for that, Unity provides with two default methods called OnTriggerEnter() and OnTriggerExit(). They are used to for when the player enters the collider and when exits Group mea of 86 Aalborg University

11 A.2. Lightmapping Portfolio A. Unity it, respectively. Lines 7-13 shows the OnTriggerEnter() method which takes a Collider object as paramter, in this case it will be the character collider. This is done by making an if-sentence in lines 9-12, with checking if the other parameter has the tag of Player and then set the enter variable to true. Similiar operation will be done in lines 16-22, for when the character exits the collider, but in this case the enter variable will be set to false. Lastly, lines can be used in other scripts to get the return value of the enter variable. Listing A.3: Collider Trigger 1 var enter : boolean ; 2 3 function Update ( ) { 4 } 5 6 //Activate the Main function when player i s near the door 7 function OnTriggerEnter ( other : Collider ) { 8 9 i f ( other. gameobject. tag == Player ) { enter = true ; 12 } 13 } //Deactivate the Main function when player go away from door 16 function OnTriggerExit ( other : Collider ) { i f ( other. gameobject. tag == Player ) { enter = f a l s e ; 21 } 22 } function getenter ( ) { r e t u r n enter ; 27 } A.2 Lightmapping Lightmaps are data structures which contains the brightness of surfaces in a 3D scene. The lightmaps are rendered offline, meaning they are precomputed and used on static objects. This technique can ease the computation time for real time 3D applications. Lightmaps are scaled with what is called lumels or lumination elements, and in Unity it is called resolution. The smaller the lumels, and therefore the more of them present in the image, the higher the resolution needs Group mea of 86 Aalborg University

12 A.2. Lightmapping Portfolio A. Unity to be set at the cost of speed and performance. Therefore, it should be noted that there must be a balance between the resolution and performance, as a too high resolution would end up using too much system resources and affect the performance negatively. Although Unity only gives the option to adjust the resolution, lightmap resolution and scale are two different things. The resolution is the area, in pixels, available for storing one or more surface s lightmaps. The number of individual surfaces that can fit on a lightmap is determined by the scale. Lower scale values, means higher quality and more space taken on a lightmap. Higher scale values, means lower quality and less space taken. The lightmaps generated are usually colored texture maps and they are usually 2D images, without information about the light s direction. While it is possible in Unity to use multiple lightmaps to provide approximate directional information to combine with normal maps, however, that feature is not used in this project, as no normal maps were used. Any lighting model may be used, to create light maps, because the lighting is entirely precomputed and real time is thus not always a necessity. Unity provides a variety of techniques including ambient occlusion, direct lighting with sampled shadow edges, and full radiosity bounced light. (Beam [2003]) The specific feature for rendering lightmaps is called Beast Lighting, and comes with three main functionalities. The first functionality is for adjusting the shadows cast from the light source. The second functionality is for adjusting the light settings and resolution. The last functionality is for adjusting the generated lightmaps. (Technologies [2011c]) Figure A.7 shows the Object tab, which handles the shadow functionality. Here it is possible to adjust the samples and the shadow radius, along with which type of shadow should be baked. Figure A.8 shows the Bake tab, which handles the setting for the light from all light sources. Here it is possible to adjust the resolution, how many bounces the light rays should take into account, how intense the ambient occlusion should be. Unity Pro also has the option to take the sky light into account and how large the final gather rays should be (bounces from surrounding). Figure A.9 shows the Maps tab, which handles all generated lightmaps. Here it is possible to adjust and modify the lightmaps. Modifying the lightmaps would require an external program which deals with graphics editing, such as Adobe Photoshop. (Incorporated [2011]) Group mea of 86 Aalborg University

13 A.2. Lightmapping Portfolio A. Unity Figure A.7: The Object tab in the Lightmapping. This tab handles the shadow samples of a light source. Figure A.8: The Bake tab in the Lightmapping. This tab handles the settings for the light of all light sources. Group mea of 86 Aalborg University

14 A.3. First Person Controller Portfolio A. Unity Figure A.9: The Maps tab in the Lightmapping. This tab handles all the generated lightmaps. A.3 First Person Controller The first person controller is included in the Unity Standard Asset, which means the controller is already available when downloading the Unity development software. The first person controller is composed of three game objects. The first game object, which is also a parent object for the two other ones, handles all the movement with keyboard and mouse. The second game object, handles the graphics rendering of the character, in this case, a simple capsule ridig body. The last game object, handles the camera functionalities, which determines what the user is going to see through that viewport. (Technologies [2011a]) Figure A.10 shows all the relevant Java scripts that handles the movement of the character. Here it is possible to adjust the movement speed, acceleration, turnaround, and jumping if enabled. It is also possible to adjust the character collider, in which how big the width, height, and slope should be. Figure A.11 shows the Graphics object, which handles the rendering of the character. There is a mesh renderer object attached, along with a mesh filter. The mesh filter simply determines what kind of polygon object it is, in this case a capsule. The mesh renderer functionality is then to draw the capsule, if enabled. It is also possible to enable cast and receive shadows for the mesh body. Figure A.12 shows the Main Camera object for the character controller. The camera object handles what will be drawn on the screen and thus determine what the user can see. Several of options Group mea of 86 Aalborg University

15 A.3. First Person Controller Portfolio A. Unity Figure A.10: The First Person Controller object. This object handles the movement settings and the character controls. Figure A.11: The Graphics object. This object handles the rendering of the first person character. are available for adjustments, including the field of view, which is the parameter to simulate depth in the scene. Other important parameters are projection and clipping planes. The projection can be set to either perspective or orthographic. This setting is set to perspective, since the orthographic setting would perceive the 3D world as 2D, which is not desired in this case. The Group mea of 86 Aalborg University

16 A.4. Sound Design Portfolio A. Unity clipping planes determine how near and far the objects will have to be, in Unity units, before it is drawn to the screen. Figure A.12: The Main Camera object. This object handles the camera settings for the character. A.4 Sound Design This section will describe some theory about the use of sound, and it will describe how the sound part of the product is created. A.4.1 Theory Sound is a difficult technique to study, because humans today are accustomed to ignoring many sounds that occur in the environment. Most are accustomed to the information about the surroundings coming from sight. Whether it is noticed or not, sound is an important technique to use. Even the old silent films where not heard without the sound of music. Connecting image and sound is something that appeals to something in the human consciousness, even babies will spontaneously connect what they hear to what they see.(david Bordwell [2010]) Group mea of 86 Aalborg University

17 A.4. Sound Design Portfolio A. Unity Fidelity In this context, fidelity refers to which extend the sound is faithful to the source as it is conceived. If the sound does not match the picture this will be a lack of fidelity an example could be a film of a dog barking, but the sound being of a cat meowing. If the film of the dog are to have a high fidelity, the sound accompanying the image will be the barking of a dog. Fidelity is thus purely a matter of expectation. (David Bordwell [2010]) In this project it will be important to achieve a high fidelity to maintain realism in the product. If an animation of a car is implemented it will be important to have the sound of a car driving by to accompany the animation. Equally important will it be to have images matching the sound in the product. Space Sound has a spacial dimension because it comes from a source. an individuals beliefs about the source have a powerful effect on how the individual understand the sound. A diegetic sound is a sound that has a source in the story world(david Bordwell [2010]). Nondiegetic sound is represented as coming from a sourece outside of the story world(david Bordwell [2010]). In this project the sounds will be diegetic, this because the sounds used in the product shall have a source in the product. The source of the sound may not be visual at all times, an example could be the sound of children playing. Even though the user is not looking at the playground, the sound from the children playing will still be there. A.4.2 Sound Scripts Sound scripts were written to handle the sound sources and are depending on collider triggers. The sound scripts consist of three different scripts, one for handling the sound effects of when the character is walking in the house, another one for handling the ambient sound effects, and the last one for when the doors open and closing. Walk Sound Listing A.4 shows the script for the sound effect of character walking. The first two lines are for declaration of two audio clips, meaning the sound effects of the walk cycle is splitted into two segments, one for each footstep. Lines 4-6 are variables for getting the player position from the Player Position script. Line 7 is to check whether the character has moved or not, this is to ensure the sound effects only play when the character is moving. Line 9 is necessary for remember which footstep is current footstep, an int is used for this purpose. Lines are to check if the character is within the colliders that will trigger the sound effects. In this case, two colliders are used to cover the house. In lines 16-23, a Start() method is used to initialize certain variables. Within the Update() method in lines 26-28, the current player position is returned, along with collider values. The Group mea of 86 Aalborg University

18 A.4. Sound Design Portfolio A. Unity first if-sentence in line 31, checks if the player is within any of the two colliders. The second if-sentence in line 32, checks whether the player has moved or not. If the player has not moved, the if-sentence will exit and the alternative effect will be active, in lines 57-60, which will simply stop the audio. If the player has moved, whichfoot will be incremented in line 34 for the next footstep. However, if the whichfoot variable is greater than one, the variable will be set back to zero. The next operations will then be to play the correct sound effects corresponding which footstep. This is done in lines Line 54 will update the last player position as the current player position, meaning the sound effect will only be triggered one time for each new footstep. Listing A.4: Play Walk 1 var channel1 : AudioClip ; 2 var channel2 : AudioClip ; 3 4 var playerposition : PlayerPosition ; 5 6 var _playerposition : Vector3 ; 7 var _lastposition : Vector3 ; 8 9 var whichfoot : i n t ; var playwalkcollider1 : PlayWalkCollider ; 12 var playwalkcollider2 : PlayWalkCollider ; 13 var _enter1 : boolean ; 14 var _enter2 : boolean ; function Start ( ) { whichfoot = 0 ; 19 audio. clip = channel1 ; 20 _lastposition = playerposition. getplayerposition ( ) ; 21 } function Update ( ) { _playerposition = playerposition. getplayerposition ( ) ; 27 _enter1 = playwalkcollider1. getenter ( ) ; 28 _enter2 = playwalkcollider2. getenter ( ) ; i f ( _enter1 == true _enter2 == true ) { 32 i f ( _playerposition! = _lastposition ) { whichfoot++; i f ( whichfoot > 1) { Group mea of 86 Aalborg University

19 A.4. Sound Design Portfolio A. Unity whichfoot = 0 ; 39 } i f ( audio. isplaying == f a l s e ) { i f ( whichfoot == 0) { audio. clip = channel1 ; 46 audio. Play ( ) ; 47 } 48 e l s e { audio. clip = channel2 ; 51 audio. Play ( ) ; 52 } _lastposition = _playerposition ; 55 } 56 } 57 e l s e { audio. Stop ( ) ; 60 } 61 } 62 } Ambient Sound The Ambient Sound scripts handle the audio sources for ambient sound effects. Each ambient script has three different audio channels that will be played one at a time, and depending on the player s position, the volume will be scaled accordingly. The following scripts are the Ambient Sound script splitted into three segments, Listing A.5 shows part one. Lines 1-3 are the declaration of the audio clips. Line 5 is for storing which audio clip should be played. Lines will be useful for the volume scaling. The Start() method in lines 23-33, initializes the necessary variables. The maximum distance that the player can at before the volume is completely turned off, that is initialized in line 30. Listing A.5: Play Ambient part 1 1 var channel1 : AudioClip ; 2 var channel2 : AudioClip ; 3 var channel3 : AudioClip ; 4 5 var whichchannel : i n t ; 6 Group mea of 86 Aalborg University

20 A.4. Sound Design Portfolio A. Unity 7 var playambientcollider : PlayAmbientCollider ; 8 var enter : boolean ; 9 10 var playerposition : PlayerPosition ; 11 var objectposition : ObjectPosition ; var distplayer : Vector3 ; 14 var distobject : Vector3 ; var dist : f l o a t ; var mindist : f l o a t ; 19 var maxdist : f l o a t ; var volume : f l o a t ; function Start ( ) { audio. clip = channel1 ; enter = true ; //very weird! mindist = 0 ; 30 maxdist = 3 5 ; whichchannel = 0 ; 33 } Listing A.6 shows part two of the Ambient Sound script, where this concerns the Update() method. In line 5, the volumedistance() method is run, which can be seen in Listing A.7. Lines 7-15 is for audio channel switching whenever the character is outside of the relevant colliders. The channel number, whichchannel, can not exceed three, else it will be set back to zero. Lines will play the audio file corresponding to which audio channel it currently is set to, when the character enters the relevant colliders. Listing A.6: Play Ambient part 2 1 function Update ( ) { 2 3 enter = playambientcollider. getenter ( ) ; 4 5 volumedistance ( ) ; 6 7 i f ( enter == f a l s e ) { 8 9 whichchannel++; i f ( whichchannel > 3) { Group mea of 86 Aalborg University

21 A.4. Sound Design Portfolio A. Unity whichchannel = 0 ; 14 } 15 } i f ( enter == true ) { //very weird! i f ( whichchannel == 0) { audio. clip = channel1 ; 22 } 23 e l s e i f ( whichchannel == 1) { audio. clip = channel2 ; 26 } 27 e l s e i f ( whichchannel == 2) { audio. clip = channel3 ; 30 } 31 e l s e i f ( whichchannel == 3) { audio. clip = channel2 ; 34 } i f ( audio. isplaying == f a l s e ) { audio. Play ( ) ; 39 } 40 } 41 } Listing A.7 concerns with the scaling of volume. Lines 3-4 get both the player and object position to later be used in line 6, to determine the distance between the two objects. Lines 8-19 handle volume strength, as line 8 first checks if the distance is between minimum and maximum distance. If so, normalize the distance to percentage and the result is stored to the audio.volume. If the distance is smaller or greater than the allowed distances, set the volume to zero. Listing A.7: Play Ambient part 3 1 function volumedistance ( ) : void{ 2 3 distplayer = playerposition. getplayerposition ( ) ; 4 distobject = objectposition. getobjectposition ( ) ; 5 6 dist = Vector3. Distance ( distplayer, distobject ) ; 7 8 i f ( ( dist > mindist ) && ( dist < maxdist ) ) { Group mea of 86 Aalborg University

22 A.4. Sound Design Portfolio A. Unity 9 10 var tempvolume = Mathf. Abs ( dist maxdist ) ; volume = tempvolume / maxdist ; audio. volume = volume ; 15 } 16 e l s e { audio. volume = 0. 0 ; 19 } 20 } Door Sound The Door Sound script handles the playback for the door sound effects when the door opens or closing. Listing A.8 shows the source code for the Door Sound. Starting with the usual declarations of variables in lines 1-7, and initialize the relevant ones in the Start() method in lines In lines 15-30, the Update() method handles which sound effect should be played, the open or closing sound effect. Lines will play the audio file for opening, while lines will play the audio file for closing, when the arguments are true for the if-sentences, respectively. Listing A.8: Play Door 1 var channel1 : AudioClip ; 2 var channel2 : AudioClip ; 3 4 var openabledoor : OpenableDoor ; 5 6 var enter : boolean ; 7 var open : boolean ; 8 9 function Start ( ) { audio. clip = channel1 ; 12 open = f a l s e ; 13 } function Update ( ) { open = openabledoor. getopen ( ) ; i f ( ( open == true ) && ( audio. isplaying == f a l s e ) && ( Input. GetKeyDown ( e ) ) && enter == true ) { audio. clip = channel1 ; 22 audio. PlayOneShot ( channel1 ) ; Group mea of 86 Aalborg University

23 A.5. Opening and Closing Doors Portfolio A. Unity 23 } i f ( ( open == f a l s e ) && ( audio. isplaying == f a l s e ) && ( Input. GetKeyDown ( e ) ) && enter == true ) { audio. clip = channel2 ; 28 audio. PlayOneShot ( channel2 ) ; 29 } 30 } A.5 Opening and Closing Doors The concept of open and closing doors contribute to the dynamic interaction, which is accomplished by implementing transformable doors. In order for this to work properly, colliders were used both as door hinges and colliders. Figure A.13 shows the front door with two bounding boxes. The smaller one acting as the hinges, while the larger one is the door body which has physics enabled, meaning there is collision on while the door is closed. Due to usability issues, collision will be deactivated when opening a door and when the door is opened. Figure A.13: A top view of the front door. Two colliders are attached to the front door model. One for the hinges, and the other for the door collision. It is also important to determine how far away the character should stand before he is able to open and close the door. Therefore, an additional collider is needed to mark out where the character may open and close a door. Figure A.14 shows the collider used for the front door, to mark out where it is possible to open and close the door. Listing A.9 shows the script relevant for the door mechanics. Starting in lines 2-8, the relevant variables are declared. Line 2 is for how fast the opening and closing will be. Line 3 will be Group mea of 86 Aalborg University

24 A.5. Opening and Closing Doors Portfolio A. Unity Figure A.14: A top view of the front door. One collider for checking whether the character is inside the collider, thus give permission to open and close the door. the transformation when opening, in this case it is an angle of 90. Line 4 is then the angle when the door is closed. The Update() method in lines 11-37, consists of three smaller parts. Lines handles the transformation of opening the door, while lines handles the transformation of closing the door. The last part in lines 29-36, will check if there is a door body collision and whether the player has pressed the key for open and close a door, and then set the variable respectively. Listing A.9: Openable Door 1 2 var smooth : f l o a t = 2. 0 ; 3 var DoorOpenAngle : f l o a t = ; 4 var DoorCloseAngle : f l o a t = 0. 0 ; 5 var open : boolean ; 6 7 var _OpenableDoorCollider : OpenableDoorCollider ; 8 var iscollision : boolean ; 9 10 //Main function 11 function Update ( ) { iscollision = _OpenableDoorCollider. getcollision ( ) ; i f ( open == true ) { 16 var target = Quaternion. Euler ( 0, DoorOpenAngle, 0) ; 17 // Dampen towards the t a r g e t r o t a t i o n 18 transform. localrotation = Quaternion. Slerp ( transform. localrotation, Group mea of 86 Aalborg University

25 A.5. Opening and Closing Doors Portfolio A. Unity target, 19 Time. deltatime * smooth ) ; 20 } i f ( open == f a l s e ) { 23 var target1 = Quaternion. Euler ( 0, DoorCloseAngle, 0) ; 24 // Dampen towards the t a r g e t r o t a t i o n 25 transform. localrotation = Quaternion. Slerp ( transform. localrotation, target1, 26 Time. deltatime * smooth ) ; 27 } i f ( iscollision == true ) { 30 i f ( Input. GetKeyDown ( e ) && open == f a l s e ) { 31 ( open ) = true ; 32 } 33 e l s e i f ( Input. GetKeyDown ( e ) && open == true ) { 34 ( open ) = f a l s e ; 35 } 36 } 37 } function getopen ( ) : boolean { r e t u r n ( open ) ; 42 } Listing A.10 shows the script for handling the door body collision. The implementation is quite trivial and simply checks whether the door is open or not. If the door is closed, in lines 13-16, then enable collision, otherwise if it is open, deactivate collision. Listing A.10: No Door Collision 1 var _openabledoor : OpenableDoor ; 2 var nocollider : boolean ; 3 4 function Start ( ) { 5 6 nocollider = f a l s e ; 7 } 8 9 function Update ( ) { nocollider = _openabledoor. getopen ( ) ; i f ( nocollider == true ) { collider. enabled = f a l s e ; Group mea of 86 Aalborg University

26 A.6. Glass Shader Portfolio A. Unity 16 } 17 e l s e { collider. enabled = true ; 20 } 21 } A.6 Glass Shader A.6.1 Theory Specular Highlights The term specular describes how the light is reflected perfectly from its source to the eye of the viewer. Smooth surfaces reflect light at an angle equal to the angle at which they arrive, called the angle of incidence. These surfaces are known as specular surfaces and the reflection as specular reflection. Specular highlights are bright spots of light that appears on specular surfaces where the surface normal is oriented precisely between the view direction and the direction of the incoming light. (Brooker [2008]; John Daintith [2008]) Figure A.15: View direction (V ) Surface normal (N) Light direction (L) geometrically reflected direction (R). (Kraus [2011a] The size of the highlights depends on how shiny the object is. The shinier the object the smaller the highlight. An object which is perfectly shiny will only reflect light from the light source (L) in the geometrically reflected direction (R). For a non-perfectly shiny object the light will be reflected in the directions around the perfectly shiny reflection direction. See figure A.15. Based on this, the intensity of the specular highlights should be large if V and R are close together. This can be represented by the cosine of the angle between R and V to the n shininess th power based on the Phong Reflection Model: (R V ) n shininess The dot product has to be clamped so that all negatives are set to 0 as negative values would mean that the light source is on the wrong side. The specular highlight can then be computed Group mea of 86 Aalborg University

27 A.6. Glass Shader Portfolio A. Unity through the following equation: I specular = I incoming k specular max(0, R V ) n shininess Where I incoming is the intensity of the incoming light, k specular is the material color for the specular reflection, V is the direction of the viewer and R is the normalized reflected direction.kraus [2011b] Cubemap Reflection The Glass shader, which the chapter will describe the implementation of, is, as described earlier, is based on the idea of using specular highlight on one side of the window and the other side would show a reflection of the surrounding area. One way to implement the second part is to use a cubemap which is then reflected in the window. A cubemap is basically a collection of six separate images which each is mapped to a face of an imaginary cube, hence the name cubemap, see figure A.16. They share some functionality with skyboxes as they often are used to show objects or scenery far away in the background. Figure A.17 shows the images of a cubemap mapped into a sphere. Technologies [2010] The idea Figure A.16: A cubemap in Unity showing all six different images. behind this part of the shader is to depict parts of the cubemap on the shader surface as if it was a reflection. The general principle is that a view ray travels from the viewer to a point on the surface where it is reflected. The reflected ray will then intersect with a point on the cubemap and this point will have a color value. This color value is then transferred to the point on the surface of the object where the ray is reflected. The mathematics for calculating this reflection is exactly the same as the mathematics for calculating the reflection of a light ray at a surface normal vector as mentioned in the specular highlight chapter.however this calculation will make a perfect reflection of the cubemap on the Group mea of 86 Aalborg University

28 A.6. Glass Shader Portfolio A. Unity Figure A.17: A cubemap in Unity shown with the images mapped to a sphere. Figure A.18: This illustration shows the principle behind cubemapping. A view ray is sent from the viewer to a point of on the surface of the object. From here the ray is reflected awy from the object and will intersect with a point on the cubemap. This point will have a pixel value which is applied to the point on the object from where the ray originated. (TopherTG [2010] surface of the object which will not look realistic in most cases. A glass window will have some reflection of the outside world but it is still possible to look through the glass. Furthermore the level of reflectance is not linear from all angles. According to a theory made by the French en- Group mea of 86 Aalborg University

29 A.6. Glass Shader Portfolio A. Unity gineer Augustin-Jean Fresnel known as the Fresnel factor, the reflectance is depending on the incidence angle of the viewer and the object: Fresnel factor A factor that describes how light is reflected from each smooth microfacet within a rough surface, as a function of incidence angle and wavelength. John Daintith [2008] This means that it would be possible, using the Fresnel factor, to implement a shader in which the angle of incidence will determine the intensity of the reflected cubemap. Using Schlick s Approximation of the Fresnel factor it is possible to use this theory in the shader. The Schlic s approximation is: F λ = f λ +(1 f λ ) (1 H V ) 5 Where V is the normalized direction of the viewer and H is the normalized halfway vector. The normalized halfway vector is described as H = (V + L)/ V + L where L is the normalized direction towards the light source. However in this application of the approximation there is no light. In this instance we use the reflected direction R instead of the light direction. This makes it a little bit simpler as the halfway vector between V and R is the normal vector due the way the reflection vector is computed: H = (V + R)/ V + R = N. F λ = f λ +(1 f λ ) (1 N V ) 5 f λ describes the intensity when the direction of the viewer, Reflection direction and normal direction all are identical, which is N V = 1. Because this shader is meant to mimic a window an alpha-value can be used here to determine the level of transparency when looking straight at the window, which should not be 1. A.6.2 GLSL Shader This following section will describe how the shader which has just been described was implemented in GLSL. GLSL stand for OpenGL Shading Languange and is a high level shading language based on C programming language and gives more direct control of the graphics pipeline without having to use lower level languages. OpenGL is based on parallel processing at different stages of a pipeline. These stages are: Vertex data, Vertex shader, Primitive Assembly, Rasterization, Fragment Shader, Per-Fragments Operations and Framebuffer. Figure A.19 shows the pipeline. In the Vertex Data stage the raw data of the scene is transferred to the pipeline. This could e.g. be triangle meshes provided by 3d models. This stage of the pipeline is not programmable but purely data. Based on the vertex data is different attributes (position, color, normal vector and texture coordinates) which are communicated to the Vertex shader. The Vertex Shader is a small program in GLSL that is applied to each vertice. This stage is programmable which means that small programs or shaders written in GLSL are applied in these stages. Vertex shaders can manipulate properties such as position, color, and texture coordinate, Group mea of 86 Aalborg University

30 A.6. Glass Shader Portfolio A. Unity Figure A.19: The OpenGL Pipeline. Blue boxes represent data, Gray boxes fixed-function stages and green programmable stages.krauss [2011b] but cannot create new vertices. The purpose of the vertex shader is to transform each vertex s 3 dimensional position provided by the Vertex Data stage to tge 2 dimensional coordinate at which it appears on the screen as well as the depth buffer (z-buffer).there are five different transformations that have to be done for the original coordinates to get the screen coordinates: Modeling, viewing, projection, perspective and viewport transformation. The first three of these are applied in the vertex shader and the projection and viewport transformation is applied automatically in the of the fixed-function stages after the Vertex shader. Uniforms, which are variables that have the same value for all shaders (vertex and fragment) that are called when rendering a particular primitive, but may defer for other primitives, are being initialized here. Examples of uniforms are vertex transformation, details of light sources etc. The output of vertex shader is varyings, which are variables that are constantly being defined by the shaders. Examples of varyings are colors, normal vectors etc. After the Vertex shader is the Primitive Assembly stage which setup the primitives according to the data received from the Vertex shader. This is a fixed-function stage which cannot be programmed. Group mea of 86 Aalborg University

31 A.6. Glass Shader Portfolio A. Unity In the Rasterization stage is the data interpolated. It determines the pixels covered by a primitive and interpolates the output varyings and depth of the vertex shader for each covered pixel. The interpolated varyings and depth is outputted to the fragments shader and is now interpolated as pixels. This is also a fixed-function stage. The Fragment Shader, also sometimes refered to as a pixel shader, is the only other programmable stage. The fragment shader computes color and other attributes for each pixel. The output is fragment color and depth which send to the Per-Fragment Operations stage. The penultimate stage is the Per-Fragments Operations. This stage consists of different tests and operations which if called can alter the fragment color which was outputted by the Fragment Shader. These alterations are then sent to the Framebuffer. The Framebuffer is the last stage where the array of pixels is computed from the colors processed by the earlier stages. Lipchak [2008]; Krauss [2011a] A.6.3 GLSL Code Explained The next section will describe the most essential parts of the GLSL code for implementing the shader. The first thing to notice is that the shader is written to be used on a plane. This is important because a plane has a backface and a frontface and this code is written specifically to show different effects on the two sides. This would not work on a cube or a sphere as the backface of these objects is the inside. Now that this is clear the first essential element is the declaration of the properties. See Listing A.11. Listing A.11: Properties method 1 Properties { 2 _Color ( Diffuse Material Color, Color ) = ( 1, 1, 1, 1 ) 3 _SpecColor ( Specular Material Color, Color ) = ( 1, 1, 1, 1 ) 4 _Shininess ( Shininess, Float ) = 10 5 _Cubemap_Intensity ( Cubemap I n t e n s i t y, Float ) = _FresnelFactor_Pwr ( Fresnel Factor Power, Float ) = _Alpha ( Alpha, Range ( 0, 1 ) ) = _Cube ( R e f l e c t i o n Map, Cube ) = {} 9 } The properties are variables that can be specified after the creation of the shader. The first property Color changes the value of the diffuse color on the backface, which is the inside of the window. The next SpecColor changes the color of the specular highlight on the same side and the Shininess how big the specular reflection is. Cubemap intensity is used to increase or decrease the intensity of the cubemap reflection on the frontface and FresnelFactor Pwr can modify the Fresnel factor on the same cubemap reflection. Alpha is the alpha value of the backface and Cube is the cubemap. All of these properties except the cubemap is used to finetune the appearance of the shader without having to change to code. See Listing A.12. Group mea of 86 Aalborg University

32 A.6. Glass Shader Portfolio A. Unity Listing A.12: SubShader 1 SubShader { 2 Tags { Queue = Transparent } // draw a f t e r a l l opaque geometry has been drawn Next is the start of the code. It start with the declaration Subshader which lets unity choose the subshader that fits the hardware and then Tag which is set to Queue = Transparent that tells the compiler to draw after all opaque geometry has been drawn. Passes The program is split into two passes. See Listing A.13 and A.14. Listing A.13: Pass1 1 Pass { 2 Tags { LightMode = ForwardBase } // pass f o r ambient l i g h t and f i r s t l i g h t source 3 ZWrite Off // don ' t write to depth b u f f e r in order not to occlude other o b j e c t s 4 Blend One OneMinusSrcAlpha // use pre m u l t i p l i e d alpha blending 5 Cull Off The first pass computes the scene with the first lightsource and ambient light. The tag Lightmode = FowardBase is set to make sure the uniforms are correctly set. The ZWrite off make sure that nothing is written to the depth buffer (z-buffer) in order not to occlude other objects. Cull off makes sure that both back and frontfaces are shown and not discarded. Lastly the line Blend One OneMinusSrcAlpha makes sure that this pass uses pre-multiplied alpha blending. Listing A.14: Pass2 1 Pass { 2 Pass { 3 Tags { LightMode = ForwardAdd } // pass f o r a d d i t i o n a l l i g h t sources 4 Blend One One // additive blending 5 ZWrite Off // don ' t write to depth b u f f e r in order not to occlude other o b j e c t s 6 Cull Off The second pass is for any additional light sources. LightMode = ForwardAdd makes sure that the uniform are correctly set for the second pass with the additional light sources. Blend One One set the alpha blending to additive blending and both ZWrite and Cull are off as it was in the first pass. Each pass is divided into a Vertex and a Fragment shader, but before any of these some different variables have to be defines. The Properties declared earlier has to be defined in each pass as well and besides this some built-in and scripted uniforms that has to be used later too (see code for details). Group mea of 86 Aalborg University

33 A.6. Glass Shader Portfolio A. Unity Vertex Shader The next code is written in the Vertex shader for both pass 1 and 2. See Listings A.15, A.16, A.17, A.18 and A.19. Listing A.15: VertexShaderpart1 1 i f ( 0. 0 == _WorldSpaceLightPos0. w ) // d i r e c t i o n a l l i g h t? 2 { 3 attenuation = 1. 0 ; // no a t t e n u a t i o n 4 lightdirection = normalize ( vec3 ( _WorldSpaceLightPos0 ) ) ; 5 } 6 e l s e // point or spot l i g h t 7 { 8 vec3 vertextolightsource = vec3 ( _WorldSpaceLightPos0 modelmatrix * gl_vertex ) ; 9 f l o a t distance = length ( vertextolightsource ) ; 10 attenuation = 1. 0 / distance ; // l i n e a r a t t e n u a t i o n 11 lightdirection = normalize ( vertextolightsource ) ; 12 } The next essential part of the code is the distinction between different light sources. It is important to distinguish between point/spot light and directional light as both the attenuation and light direction is essential for later calculations but computes differently. The distinguisment is rather simple and can be done with a if-else construct. This is based on the fact that the unity specific uniform WorldSpaceLightPos0 specifies the position of a light for a point/spot light and the 4th coordinate of a point is 1 while for directional light it specifies a direction which fourth coordinate is 0. So if 0.0 == WorldSpaceLightPos0.w it is a directional light. Else it is a point or a spot light. Now that the distinguishment has been made the different properties can be computed. The main difference is that directional light has no attenuation while the other types have. Listing A.16: VertexShaderpart2 1 vec3 diffusereflection = attenuation * vec3 ( _LightColor0 ) * vec3 ( _Color ) max ( 0. 0, dot ( normaldirection, lightdirection ) ) ; 2 * 3 } The diffuse color is computed with the attenuation calculated above as well as the LightColor0 and the userspecified Color. Listing A.17: VertexShaderpart3 1 vec3 specularreflection ; 2 i f ( dot ( normaldirection, lightdirection ) < 0. 0 ) // l i g h t source on the wrong side? 3 { Group mea of 86 Aalborg University

34 A.6. Glass Shader Portfolio A. Unity 4 specularreflection = vec3 ( 0. 0, 0. 0, 0. 0 ) ; // no specular r e f l e c t i o n 5 } 6 e l s e // l i g h t source on the r i g h t side 7 { 8 specularreflection = attenuation * vec3 ( _LightColor0 ) * vec3 ( _SpecColor ) 9 * 10 } pow ( max ( 0. 0, dot ( reflect( lightdirection, normaldirection ), viewdirection ) ), _Shininess ) ; The specular reflection is dependent on the lightsource being on the right side of the object. An if-else construct tests wether the dotproduct of the lightdirection and normaldirection is smaller than 0. If that is the case light is on the wrong side and the specular reflection is 0. If the light is on the right side the specular reflection is calculated. Listing A.18: VertexShaderpart4 1 color = vec4 ( ambientlighting + diffusereflection + specularreflection, 1. 0 ) ; The different reflection models in combined and added to color which is a varyings. There is a slight difference here in the two passes. In the first pass the ambient light, the diffuserelflection and specularreflection is added together and an alpha value of one is assigned while the ambient light is neglected in the second pass. Listing A.19: VertexShaderpart5 1 gl_position = gl_modelviewprojectionmatrix * gl_vertex ; This last line that will be explained in the Vertex shader is the Vertex transformations described earlier in the chapter. gl Vertex which is a predefined attribute that contains the vertex data is multiplied with gl ModelViewProjectionMatrix which is a combination of the Model, View and Projection Transformations. Fragment Shader The fragment shader is much more compact than the Vertex shader is this implementation. See listin A.20. Listing A.20: FragmentShader 1 #ifdef FRAGMENT 2 3 void main ( ) 4 { 5 i f (! gl_frontfacing ) 6 { 7 gl_fragcolor = vec4 ( vec3 ( color ), _Alpha ) ; Group mea of 86 Aalborg University

35 A.6. Glass Shader Portfolio A. Unity 8 } 9 e l s e 10 { 11 vec3 reflecteddirection = reflect ( viewdir, normalize( normaldir ) ) ; 12 f l o a t reflectivity = _Alpha + ( 1. 0 _Alpha ) * pow ( 1. 0 abs ( dot ( normalize ( viewdir ), normalize( normaldir ) ) ), _FresnelFactor_Pwr ) ; 13 gl_fragcolor = _Cubemap_Intensity * texturecube ( _Cube, reflecteddirection ) * reflectivity ; gl_fragcolor. a = reflectivity ; 16 // gl FragColor =vec4 ( viewdir, 1. 0 ) ; 17 } 18 } #endif The fragement shader in the first pass is based on a if-else construct to define what will happen to the frontface and the backface. It is implemented by using the if the GLSL command gl Frontfacing is not true (!gl Frontfacing) then it is the backface and else it is the frontface. If it is the frontface then the Fresnel computation is done to assign the cubemap to the surface. If it is the backface then the varyings Color will be assigned to gl Fragcolor. A.6.4 CG Code Explained This section will describe the Surface shader written for the windows in unity. Surface shaders are Unity s high-level shader code approach that simplifies the shader through autogeneration of repetitive code which would be necessary in normal vertex and pixel shaders. The shader is written in CG programming language. Technologies [2011e]; Wittayabundit [2011] The shader is simplified version of the one described earlier as there will not be any specular highlight on the backface of the plane but only the Fresnel modified cubemap and transparency. As with the GLSL implementation the CG implementation uses some properties that can be altered without changing the code. See Listings A.21, A.22, A.23, A.24 and A.25. Listing A.21: CGShaderPart1 1 Shader CG Glass shader { 2 Properties { 3 _Cubemap_Intensity ( Cubemap I n t e n s i t y, Float ) = _FresnelFactor_Pwr ( Fresnel Factor Power, Float ) = _Alpha_Frontface ( Alpha f o r the f r o n t f a c e, Range ( 0, 1 ) ) = _Alpha_Backface ( Alpha f o r the backface, Range ( 0, 1 ) ) = _Cube ( R e f l e c t i o n Map, Cube ) = {} Group mea of 86 Aalborg University

36 A.6. Glass Shader Portfolio A. Unity These properties are. Cubemap intensity which is used to increase or decrease the intensity of the cubemap reflection on the frontface, FresnelFactor Pwr that can modify the Fresnel factor on the same cubemap reflection, Alpha Frontface and Alpha Backface that can alter the opacity of the shader on the front and backface and last Cube which is the cubemap. Listing A.22: CGShaderPart2 1 SubShader { 2 Tags { Queue = Transparent } // draw a f t e r a l l opaque geometry has been drawn 3 Cull Off 4 5 CGPROGRAM 6 #pragma surface surf Lambert alpha After the properties has been defines the subshader command is called which lets unity choose the subshader that fits the hardware and then Tag which is set to Queue = Transparent that tells the compiler to draw after all opaque geometry has been drawn. Cull off makes sure that both back and frontfaces are shown and not discarded. The line CGPROGRAM just tells the compiler that the following code is a CG program. The #pragma surface is the directive to indicate it is a surface shader. Surf is the name of the function called as the main function, Lambert is the lightmodel used and alpha is a optionalparameter that allows alpha blending mode. Listing A.23: CGShaderPart3 1 struct Input 2 { 3 float3 viewdir ; 4 float3 worldnormal ; 5 float3 worldrefl ; 6 } ; The next part of the code is a struct that defines all the input values that has to be used in the main function. Here it contains viewdir which contains view direction, worldnormal which contains world normal vector and worldrefl which contains world reflection vector. Listing A.24: CGShaderPart4 1 samplercube _Cube ; 2 f l o a t _Alpha_Frontface ; 3 f l o a t _Alpha_Backface ; 4 f l o a t _Cubemap_Intensity ; 5 f l o a t _FresnelFactor_Pwr ; Group mea of 86 Aalborg University

37 A.7. Line of Sight Portfolio A. Unity Just after the struct any additional variables defined will have to be declared. Here all the properties are declared. Listing A.25: CGShaderPart5 1 void surf ( Input IN, inout SurfaceOutput o ) 2 { 3 i f ( dot ( IN. worldnormal, IN. viewdir ) < 0. 0 ) 4 { 5 o. Alpha = _Alpha_Backface ; 6 } 7 e l s e 8 { 9 10 f l o a t reflectivity = _Alpha_Frontface + ( 1. 0 _Alpha_Frontface ) * pow ( 1. 0 abs ( dot ( normalize ( IN. viewdir ), normalize ( IN. worldnormal ) ) ), _FresnelFactor_Pwr ) ; 11 o. Emission = _Cubemap_Intensity * texcube ( _Cube, IN. worldrefl ). rgb ; 12 o. Alpha = reflectivity ; 13 } 14 } The main function is called as void surf (Input IN, inout SurfaceOutput o). The purpose of this function is to determine first what is backface and frontface. This is done simply with an if-else construct which test whether the dotproduct of normalvector and the viewdirection is below zero. If it is below zero it is the backface and the alpha value is set to the value specified in the properties. If it is the frontface the reflectivity is first calculated using Schlick s approximation of Fresnel Factor. Then the output is set to use emission and output the Cubemap intensity multiplied with the cubemap and in the end the alpha level is set equal to the Fresnel reflectivity. A.7 Line of Sight Line of sight is a method to optimize performance and rendering time. The objects will only be rendered if the camera is facing towards the object and is visible, meaning no other object blocks the object. Normally, line of sight implementation is based on ray casting and back tracing, which is also a technique Unity supports, as it has runtime classes for raycasting and linecasting. However, the group chose a different approach for line of sight implementation for the sake of simplicity and time restrictions. The technique is then instead of raycasting, it is based on box colliders, meaning the objects will only be rendered when the character is within the relevant colliders. Figure A.20 shows three colliders for the bedroom objects. When the character is inside one of the colliders, the bedroom objects will then be rendered, otherwise not. Listing A.26 shows the script related to the line of sight. In this case, the script is related to the bedroom objects. Starting in lines 1-7, all the relevant variables for the colliders are declared, Group mea of 86 Aalborg University

38 A.7. Line of Sight Portfolio A. Unity Figure A.20: A top view of the house ground and environment. Three colliders are used to check when the bedroom should be rendered. in this case, it is three colliders. Line 9 is a variable for storing objects with the same tag in an array of type GameObject. This will be useful for fast search of objects with the same tag, since all objects in the house is tagged with a tag. Lines is the Start() method, which only stores the tag with Bedroom in the array, objectstag. Lines is the Update() method which is here the checking is handled, to check whether the character is inside of the colliders. If that is true, then all the objects in the objectstag array will be rendered. If none of the colliders are true, then the renderer will be deactivated. Listing A.26: Line of Sight for Bedroom 1 var trigger1 : LOSCollider ; 2 var trigger2 : LOSCollider ; 3 var trigger3 : LOSCollider ; 4 5 var _trigger1 : boolean ; 6 var _trigger2 : boolean ; 7 var _trigger3 : boolean ; 8 9 var objectstag : GameObject [ ] ; function Start ( ) { objectstag = GameObject. FindGameObjectsWithTag ( Bedroom ) ; 14 } function Update ( ) { _trigger1 = trigger1. getenter ( ) ; Group mea of 86 Aalborg University