Room Simuation for Mutichanne Fim and Music 4993 (B-2) Knud Bank Christensen and Thomas Lund TC Eectronic A/S DK-8240 Risskov, Denmark Presented at the 107th Convention 1999 September 24-27 New York This preprint has been reproduced from the author s advance manuscript, without editing, corrections or consideration by the Review Board. The AES takes no responsibiity for the contents. Additiona preprints may be obtained by sending request and remittance to the Audio Engineering Society, 60 East 42nd St., New York, New York 10165-2520, USA. A rights reserved. Reproduction of this preprint, or any portion thereof, is not permitted without direct permission from the Journa of the Audio Engineering Society. AN AUDO ENGNEERNG SOCETY PREPRNT
Room Simuation for Mutichanne Fim and Music KNUD BANK CHRSTENSEN AND THOMAS LUND TC Eectronic A/S Sindasvej 34, DK-8240 Risskov, DENMARK To fuy expoit new Reverb and Spatia agorithms for 5.1 and 7.1 environments in music and fim, effect mixing procedures may have to be changed sighty. The paper wi describe production techniques for positioning of sources and istener in virtua rooms, and an agorithm structure to achieve this. 0. NTRODUCTON Typica production techniques invove ony one or two inputs to the room simuation system, thereby imiting the precision of source positioning to be ony a matter of send eve differences and power panning. This one source - one istener mode is not very satisfying when producing for mono or stereo, but even worse when the reproduction system is mutichanne. Mutichanne recording and reproduction is an opportunity for the production engineer to discriminate deiberatey between scenes or instruments heard from a distance, and sources directy engaging the istener. For fim work, engaging audio has a very pronounced effect for stimuating the viewer emotionay, and may therefore significanty add to the iusion presented by the picture. n the search of more authenticity in artificia room generation, ong term studies of natura eary refection patterns have ed us to propose new production and agorithm techniques. Using ray tracing in conjunction with carefu adjustments by ear, we have achieved simuation modes with higher naturaness and fexibiity, which is the basis of true source positioning. The paper wi discuss two aspects of precise room simuation for muti source, mutichanne environments to cover distant and engaged istening: Present different production techniques Describe an agorithm structure to achieve the objectives
. SNGLE SOURCE REVERB By having ony one or two inputs in a room simuator, the rendering is based upon mutipe sources sharing the same eary refection pattern, and therefore it is not reay convincing. n the rea word, a actors or instruments are not pied up on top of each other.. Music Production n many studios, one good reverb is used to render the basic environment of a particuar mix. One aux send, set at different eves on the different channes, is used to obtain depth and some compexity in the sound image. To obtain a sound image of a higher compexity and depth, severa auxes and reverbs have normay been used. Tuning of the eves, pans and reverb parameters in such a setup may be very time-consuming. For effect purposes, anything goes, but if the goa is a representation of a natura room or a consistent rendering of a virtua room, it may be hard to achieve using conventiona reverbs..2 Fim and Post Production For appications where picture is added to the sound, severa psychoogica studies have proven audio to be better at generating entertainment peasure and emotions than visua inputs. When it comes to counting neuroogica synapses to the brain, vision has ong been known to be our dominant input source. However, a study by Kar Ktipfmtier [4] has suggested, that stimuation of even our conscious mind is amost equay we achieved from visua compared to auditive inputs. Sense No of Synapses Conscious input, bps Eye O. 000.000 40 Ear 100.000 30 Skin. 000.000 5 Sme 100.000 1 Taste 1.000 Stimuation of conscious mind [4] Reaism in audio is just as important when it is accompanied by picture. 2
n mutichanne work for fim, severa reverbs configured as mono in - mono out are often used on discrete sources. By doing so, the direct sound and the diffused fied are easy to position in the surround environment. The technique is therefore especiay effective for point source distance simuation. As an aternative, severa stereo reverbs are used on the same sources to achieve a number of de-correated outputs routed to different reproduction channes. With both approaches, adjustments can be very time-consuming, and a truy engaging istening experience is difficut to achieve. 2. MULTPLE SOURCE ROOM SMULATON To obtain the most natura sounding and precise room simuation, an artificia reverb system shoud be based upon positioning of mutipe sources in a virtua room. Each source shoud have individua eary refection properties with regards to timing, direction, fitering and eve. We have found this to be true for both stereo and mutichanne presentations. f the target format is 5.1, at east two directiona configurations shoud exist in the room simuator, namey for home (110 degree surround speakers) and theatre (side array surround speakers) reproduction. The room simuator shoud aso be fexibe enough to easiy adopt to new mutichanne formats, e.g. the Doby EX scheme. By changing the production technique sighty, mutipe sends from e.g. the Auxes, Group busses or Direct outs of the mixing consoe can be used to define severa discreet positions as inputs to the room simuation system. From a production point of view, mutipe source room simuation can be configured two ways, as described beow. Any arge scae consoe buid for stereo production can adapt to both routing schemes. 3
2. The Additive Approach The conventiona approach to reverb is additive. Dry signas are fed to the reverb system, and wet-ony signas are returned and added at the mixer. With a mutipe input room simuator, this configuration works much better than with an singe source reverb, because at east each source can be approximated to fit the nearest position rendered. However, norma power panning sti needs to be appied in the mixer. An even more precise rendering can be achieved using the integrated approach described beow. 2.2 The ntegrated Positioning Approach The sources in a mix needing the most precise positioning and room simuation, shoud be treated this way: The source is competey positioned and rendered into a precise position by passing the dry signa through the simuation system, from which a composite output from a number of source positions are avaiabe. XY positioning to any target format, stereo or mutichanne, wi be rendered as a best fit. The positioning parameters (repacing conventiona power panning) can be controed from a screen, a joystick or discrete X and Y contros. With a positioning done in the room simuator, consoes made for stereo production may thereby overcome some of their imitations. 4
3. ALGORTHM STRUCTURE This part of our paper describes a generic agorithm currenty in use for Mutichanne Room Simuator deveopment. t is not a description of any particuar present or future product, but rather a presentation of the framework and way-of-thinking that has produced our atest Room Simuation products and is expected to produce more in the future. 3. Design conditions The overa system requirements can be stated as foows: The system must be abe to produce a natura-sounding simuation of a number of sources in acoustic environments ranging from phone-booth to canyon The system shoud not be imited to simuating natura acoustics: Often quite unnatura reverb effects are desired, e.g. for pop music or science fiction fim effects. The system shoud be abe to render the simuation via a number of different reproduction setups, e.g. 5.1,7.1, stereo etc. The system shoud be moduar so that new rooms, new source positions in existing rooms, new source types or new target reproduction setups can be added with minima change to existing eements. The system shoud be easiy tuneabe: n our experience, no semi-automatic physica modeing scheme, however eaborate, is ikey to produce subjective resuts as good as those obtained by skied peope tuning a user-friendy, interactive deveopment prototype by ear. Fortunatey there are a few factors that make the job easier for us: There are no strict requirements for simuation accuracy: Certainy not physica accuracy (the sound fied around the istener s head), and not even perceptua accuracy (the istener s menta image of the simuated event and environment). The istener has no way of A/E3 switching between the simuation and the rea thing, so ony credibiity and predictabiity counts: The simuation must not in any way sound artificia, uness intended to, and the perceived room geometries and source positions shoud be reativey, but not absoutey, accurate. Moore s Law is with us. The continua exponentia growth in memory and cacuation capacity avaiabe within a given budget frame has two effects: t constanty expands the practica imits for agorithm compexity, and it makes it increasingy feasibe to trade in a bit of code overhead for improved moduarity, tuneabiity, etc. There are physica modeing systems readiy avaiabe, which may provide a starting point for the simuation. 5
3.2 Bock diagram The overa bock diagram of the Room Simuator is shown in fig. 1. As often seen, the system is divided into two main paths: An eary refections synthesis system consisting of a so-caed Eary Pattern Generator (EPG) for each source and a common Direction Rendering Unit (DRU) that renders the eary refections through the chosen reproduction setup. And a Reverb system producing the ate, diffuse part of the sound fied. Note that - contrary to what is normay the case - there is no direct signa path. The dry source signas are merey Oth order refections produced by their respective EPGs. n the foowing, a more detaied description of the individua bocks is given. 3.3 Eary Pattern Generators Each EPG takes one dry source input and produces a arge set of eary refections, incuding the direct signa, sorted and processed in the foowing dimensions Leve Deay Diffusion Coor Direction The Leve and Deay dimensions are easiy impemented with high precision, the other 3 dimensions are each quantized into a number of predefined steps, for instance 12 different directions. Normay, the direct signa wi not be subjected to Diffusion or Coor. The quantization and step definition of the Direction dimension must be the same for a sources, because it is impemented in the common Direction Rendering Unit. Physica modeing programs such as Odeon [] may provide an initia setting of the EPG. 3.4 Direction Rendering Unit The purpose of this unit is to render a number of inputs to an equa number of different, predefined subjective directions-of-arriva at the istening position via the chosen reproduction setup, typicay a 5-channe speaker system. Thus, the DRU may be a simpe, genera panning matrix, a VBAP [2] system or an HRTF- or Ambisonics-based [3] system. 3.5 Reverb Feed Matrix The reverb feed matrix determines each source s contribution to each Reverberator input channe. Besides gain and deay contros, some fitering may aso be beneficia here. 6
3.6 Reverberator To ensure maximum de-correation between output channes, each has its own independent reverb tai generator. Controabe parameters incude: Reverberation time as a function of frequency T,(f) Diffusion Moduation Smoothness We take particuar pride in the fact that our tai can achieve such smoothness in both time and frequency, and that moduation may be omitted entirey. This eiminates the risk of pitch distortion and even the sightest Dopper effect, which tends to destroy focus of the individua sources in a mutichanne room simuator. Again, an initia setting of T,(f) may be obtained from Odeon. 3.7 Speaker Contro This bock is by defaut just a direct connection from input to output. But it may aso be used to check the stereo- and mono compatibiity of the fina simuation resut by appying a down-mixing to these formats. Aso it provides deay- and gain compensation for non-uniform oudspeaker setups, which may aso - as a rough approximation - be used the other way around to emuate non-uniform or mispaced setups and thus check the simuation s robustness to such imperfections. 7
4. CONCLUSON The system described above is evidenty a very open system under continua deveopment. At the time of writing these words, our test system is running in rea time on a mutiprocessor SG server with an 18-window graphica user interface providing interactive access to approximatey 2000 ow- and higher-eve parameters. However, this is not the time or pace to go into more detais. When this paper is presented at the 107* AES Convention in about 4 months, we wi have more rea ife experience with the system. f integrated positioning is used with muti-source room simuation, our experiments have aready shown how much there is to gain in terms of reaism and working speed. But even with the ess radica additive approach, virtua rooms may be rendered more convincingy with muti-source simuators. For appications where picture is added to the sound, the most stimuating source wi be one, where audio and video are treated with equa attention to quaity and detai. The new possibiities avaiabe from muti-source room processors may be expoited to generate a rea quaity improvement at the end istener, especiay when his reproduction system is mutichanne. More convincing sound generates more convincing picture. 8
REFERENCES [] http://ww.dat.dtu.dk/-odeon/ [2] Vie Pukki: Virtua Sound Source Positioning Using Vector Base Ampitude Panning, JAES Vo. 45, No. 6, pp. 456, 1997. [3] Jerome Danie, Jean-Bernard Raut & Jean-Dominique Poack: Ambisonics Encoding of Other Audio Formats for Mutipe Listening Conditions, AES Preprint no. 4795, 1998. [4] Kiipfmtier, Kar: Nachrichtenverarbeitung im Menschen, University of Darmstadt, 1975. 9
Speaker Contro and Down-Mixer + N channes output N drecbons, Eary Pattern Generator 1 Directon Rendering Unit N channes L - Reverb Feed Matrrx N channes 1 Reverberator N channes : 4