2006 IEEE Inernaional Conference on Fuzzy Sysems Sheraon Vancouver Wall Cenre Hoel, Vancouver, BC, Canada July 16-21, 2006 DigialBeing: an Ambien Inelligen Dance Space Magy Seif El-Nasr, Thanos Vasilakos Absrac DigialBeing is an ambien inelligen sysem ha aims o use sage lighing and lighing in projeced imagery wihin a dance performance o porray dancer s arousal sae. The dance space will be augmened wih pressure sensors o rack dancers movemens; dancers will also wear physiological sensors. Sensor daa will be passed o a hree layered archiecure. Layer 1 is composed of a sysem ha analyzes sensor daa. Layer 2 is composed of wo inelligen lighing sysems ha use he analyzed sensor informaion o adap onsage and virual lighing o show dancer s arousal level. Layer 3 ranslaes lighing changes o appropriae lighing board commands as well as rendering commands o render he projeced imagery. I. INTRODUCTION hrough he years, dancers have been rained o use heir T bodies as a media o express heir inner feelings while improvising wih music. Theare dance expanded he basic se of expressive gesures adding lighing and scenery as par of he arisic expression. In he lae 1990s, many ariss and researchers explored he use of echnology in dance spaces. One of he mos popular ideas is he use of compuer generaed images and animaions projeced on he backdrop of a dance performance [1-4]. Alernaively, we seek o expand he se of expressive gesures in dance hrough he use of sensors and inelligen sysems ha conrol physical sage lighing as well as virual lighing wihin projeced imagery. We call he resuling arifac DigialBeing. We see he DigialBeing as an ambien inelligen environmen. Ambien Inelligence (AmI) is a vision ha inegraes conceps ranging from ubiquious compuing o auonomous and inelligen sysems o compose an environmen where echnology is invisible and embedded in everyhing around us acing auonomously on our behalf responding o our needs. In such environmens [5] elecronics will be inegraed ino clohing, furniure, cars, houses, offices, and public places. We seek o exend his vision o a dance space. In paricular, dancers will wear wireless physiological sensors ha measure arousal hrough skin conducance and body emperaure. We will also place pressure sensors over he enire physical dance floor o collec dancers locaions and movemens. We will develop an inelligen sysem ha analyzes he physiological sensor signals compuing a coninuous signal represening dancer s arousal sae. This sysem will also analyze pressure signals o idenify on-sage lighs affecing he dancer. This informaion will hen be disribued o wo inelligen lighing sysems ha adjus onsage lighing and lighing of projeced 3D imagery, accordingly. The idea is o express he arousal sae of he dancers hrough ligh color wihin he sage as well as in virual imagery projeced on he backdrop. We believe ha using his ambien inelligen sysem will allow dancers o use heir environmen as an expressive medium. This akes dance o a new direcion ha has no been explored before. The conribuion of his paper is in he concepual framework discussed hrough he archiecure of he ambien inelligen sysem. The sysem iself is sill under developmen. The paper will be divided ino he following secions. In he previous work secion, we will review relaed work. The following secion will discuss he space and equipmens used, including sensors and lighing equipmen. We will follow his discussion wih a descripion of he archiecure. We will conclude by discussing conribuions of he work presened as well as oulining some fuure work. II. PREVIOUS WORK Numerous composers, choreographers, dancers, and heoriss have explored he use of echnology in heare and dance. We do no inend o describe all he work ha has been done in he realms of academic research, insallaions, or ineracive producions here. However, we would like o discuss few examples ha have influenced our work. Discussing hese examples will help siuae our work, uncover is uniqueness, and is purpose in exending curren work and echniques. One of he mos influenial and significan work ha used animaed figures for choreography is he work of Merce Cunningham. In his dance performance Trackers, he used a compuer sysem called Life Forms devised by Tom Calver [6] o choreograph his dance movemens. Life Forms is a piece of sofware designed o provide several sylized animaed characers ha allow users o choreograph a scene or explore cerain dance seps. M. Seif El-Nasr, Assisan Professor, Penn Sae Universiy, 316E IST Building, Universiy Park, PA 16802 (corresponding auhor o provide phone:814-865-6166; e-mail: magy@is.psu.edu). T. Vasilkos, Professor, Universiy of Thessaly, Greece. (e-mail: vasilako@ah.forhne.gr). 0-7803-9489-5/06/$20.00/ 2006 IEEE 907
In addiion o using animaion for choreography, Cunningham also developed a virual dance insallaion in collaboraion wih Paul Kaiser and Shelley Eshkar. This work was presened a Siggraph 1998. I was composed of a menal landscape in which moion-capured hand-drawn figures performed inricae choreography in 3D [1]. Besides he use of animaed characers in a virual performance, several performers have explored he use of animaion wihin a real-life dance performance. For example, projeced graphics has been used on backdrops in he San Francisco balle Pixellage [2]. In one of he scenes hey used a virual animaed ball (projeced on he screen behind he dancers) which dancers hrew o each oher. Anoher balle performance called The Caherine Wheel [3] used an animaed characer o represen he spiriual figure of Sain Caherine. By using an animaed characer, ariss can easily represen he spiriual naure of he characer as opposed o using real life effecs or make-up. Anoher example of he mix beween virual and real characers is in he work of Meador e al. [4]. They developed a collaboraive producion ha mixes he use of virual and real dancers wihin a dance sage. They used hree differen projecors wihin a dance performance; one of hese projecors was used o projec a virual characer ha ineraced wih he dancers on sage [4]. This work was influenced by he work of Dan Salz who direced The Tempes 2000 produced by he Ineracive Performance Lab Group a Universiy of Georgia [hp://dpa.nu.ac.uk/dpa_ search/resul.php3?projec=136]. In his producion of The Tempes, hey projeced he characer Ariel as a virual characer. Moion capure equipmen was used o animae he characer in real-ime. The use of a synheic characer for Ariel added o his magical qualiy, hus enhancing he overall performance. Troika Ranch is a Dance company ha explored he use of echnology in dance. They developed a moion capure sysem called MidiDancer, which uses several cameras o capure performers moions. They used his echnology in several producions using projecion video and moion capure o enhance he medium of expression for dance [hp://www.roikaranch.org/]. Ulyae and Bianciardi showed heir work on he Ineracive Dance Club in Siggraph 1998 [7, 8]. The ineracive dance club is composed of several pieces where hey experimened wih several seups and sensors, including infra-red, pressure, and cameras. They divided he dance floor ino differen zones which induced differen ineraciviy paradigms. For example, in one zone hey had a se of parallel ligh beams ha deec when beams are broken. By breaking beams of ligh, paricipans can rigger 4-16 noes of musical phrases. Similarly, Todd Winkler explored he use of gesures as a media for music composiion [9-11]. He focused on he use of dance and space o compose elecronic music. He used he Very Nervous Sysem (VNS) [12-14] which is a sysem composed of one or wo cameras ha deec speed and locaion of dancers in a 3D sage. Using his as a daa poin, he hen explored he use of algorihms o generae music [10]. He explored several mehods of mapping he oupu daa from VNS o musical parameers, such as frequency, pich, and imber. He presened wo producions in he lae 1998 showing his sysem a work [11]. Several ariss have explored he reversal problem of how o visualize music in a 2D or 3D projecion used in a dance performance. Currenly WinAmp and Windows Media Player boh include buil in algorihms ha map music o 2D space using a frequency specrum exraced from he music file [15, 16]. DiPaola and Arya explored he abiliy o exrac emoional parameers from music and mapping hem ino a synheic face [17]. Wagner and Carroll developed a 3D music visualizaion sysem called DeepWave [18]. DeepWave analyzes music files exracing frequency, pich, vocals, ec. and maps hem o shape, color, exure, and animaions in 3D space. Through experimenaion hey found ha vocals are bes mapped o color and ransparency, percussion o size and shape, and guiars and keyboards o animaion. DeepWave also allows users o auhor skins and inpu 3D scenes and exures. Beyond projecion as a way o influence he dance space, Louis-Philippe Demers have explored he use of adjusing physical sage lighing wihin an ar insallaion [19, 20]. He describes a sysem ha uses several sensors including, pas sensors, video sensors, opical and infrared sensors, sonar sensors, and 3D ulrasound devices o predic blocking and gaher gesure informaion. Using hese as inpus, he developed a sysem ha changes brighness or inensiy, color paern, and angles of on-sage lighs. He showed his sysem in several projecs, including The Shadow Projec [21] and Los Referenial [22]. Our work follows he same idea. However, we will change ligh direcion, color, and inensiy based on a formal model of lighing design developed based on heare and film design heories [23, 24]. We are also adaping he lighing o he arousal sae of he dancer which was no explored in Demers work. I. THE DANCE SPACE We envision a dance space similar o a proscenium heare sage. Sage lighs will be rigged on poss above he dancers. We will use a backdrop o projec a 3D world sharing he heme of he dance performance. We will ask dancers o wear physiological sensors o rack heir inernal sae. We will also implan pressure sensors in he space o rack dancers posiions and movemens. The space will include a 3D surround sound sysem o play music and ambien sounds relaed o he projeced virual world. Dancers will be free o move around in he space. Sensor informaion will be ransmied wirelessly hrough a local nework o a compuer ha analyzes he informaion and alers on-sage lighing as well as he projeced 3D world in erms of is lighing o express dancers arousal sae. A. Physical Equipmen 908
For on-sage lighing, we will use physical lighs ha accep commands o roae in 3D space and change color. There are several inelligen dynamic ligh models in he marke oday. These lighs can be programmed o roae. They also include a color wheel ha allows ariss o load up o 20 color gels. Through a ligh board ariss can give commands o specific lighs o direc hem o change color a any given momen. The ransiion from one color o anoher will no be as smooh as in a virual world, however. This is due o he fac ha in virual environmens color is described using hree digis represening RBG values. In a heare producion he color is represened by color gels wih predeermined RBG values. Therefore, color mixing will be inflexible and difficul compared o a virual world. Preparaion and planning is needed a he pre-producion sage. Dancers will wear wireless physiological sensors; we op o use SenseWear. SenseWear PRO 2 Armband is a wearable body monior ha enables coninuous collecion of low-level physiological daa: hea flux, skin emperaure, near body emperaure, and galvanic skin response. This device also includes a HealhManager web-enabled service ha combines wearable compuing, wireless conneciviy, and online services, which is perfec for our purposes o allow dancers o freely move abou. Using his device we will collec and ransmi physiological daa coninuously. In addiion, we will use pressure sensors on he floor. For his purpose, we will adop he design discussed in Srinivasan e al. s work [25]. They designed a pressure sensor floor ma specifically for dancing. The floor will be consruced of several ligh-weigh high resoluion pressure sensor mas covering he enire sage. The acual number of sensors will range from 20,800 o approximaely 100,000 sensors depending on he size of he sage. The sensors will be clusered ogeher. They will send signals o a hos compuer ha assembles and fuses his informaion o deermine lighs affecing he dancer. B. Arisic Conen As wih any performance, arisic conen is imporan. Arisic conen for his paricular piece include: he 3D virual environmen o be projeced wihin he performance, he acual music pieces composed for he performance, and he sound effecs used in he performance. We also believe ha ariss should have conrol on he arisic change projeced by he lighing. For his purpose, ariss will direc he inelligen sysems o compose changes ha mach he arisic vision. All inelligen sysems developed in his paper will include ools ha allow ariss o auhor or dicae arisic goals ha compose his/her arisic vision. II. ARCHITECTURE The archiecure is composed of several subsysems (shown in figure 1). The Sensor Analysis Sysem analyzes wo sensor signals: physiological sensor signals o idenify he dancer s arousal sae, and pressure sensor signals o idenify lighs relevan o dancer s posiions. The arousal sae will be sored in a srucure called Dancer Arousal Sae represened in XML. The lighs relevan o dancer s posiions will be sored as a lis of ligh IDs ha are coninuously changing as he dancer moves. Sensor signals ransmied wirelessly Ligh IDs Inelligen on-sage Lighing Sysem Lighs{ID, color gel, angle} On-sage Lighing Trans Sysem Signals o Ligh Board Sensor Analysis Sysem Dancer Arousal Sae Fig. 1. Archiecure of he Sysem Inelligen Virual Lighing Sysem Ligh Seup, Lighs{colors & angles} Game Engine Signals o Projecor The Dancer Arousal Sae will be passed o wo sysems: Inelligen Virual Lighing Sysem and Inelligen on-sage Lighing Sysem. Boh sysems are based on our earlier work on Expressive Lighing Engine (ELE) [23, 24]. The Inelligen Virual Lighing Sysem uses virual space info (provided by he aris a a preproducion sage), arisic consrains, as well as he Dancer Arousal Sae o compue a ligh seup if none exiss. A ligh seup consiss of he number of lighs o use in he virual environmen and heir placemens. Once a ligh seup is calculaed, he inelligen virual lighing sysem deermines color and angle changes for each ligh currenly specified in he ligh seup. These changes will be deermined o mach he curren arousal sae of he dancer based on heare and film lighing design heory [26-29], as will be discussed below. The ligh seup, colors, and angles will be given o a game engine o render he frame, which is hen projeced on he cyclorama. Similarly, he inelligen on-sage lighing sysem deermines colors and angles for sage lighs given dancers locaions, Dancer Arousal Sae, and arisic consrains. I, however, deermines color based on color gels specified in is daabase (which conains all gels uploaded by aris in preproducion and heir color wheel locaion). I also does no generae a ligh seup. Insead, i caegories lighs on sage as: focus lighs which are lighs affecing dancers given by he lis of ligh IDs (oupu of Sensor Analysis Sysem), and non-focus lighs, all lighs no in he lis of 909
ligh IDs. Based on his difference, i compues, for each physical on-sage ligh, a color from he color wheel and an angle roaion. This informaion is hen ranslaed o ligh board hex code by he On-sage Lighing Trans Sysem for he physical lighs. For virual lighs, we will ranslae inelligen virual lighing sysem oupus o rendering commands which are inerfaced wih a rendering engine. Separaing he lighing sysems from heir implemenaion (e.g. rendering engine and he ranslaion o lighing board insrucion) is imporan because i enables us o plug in oher inelligen lighing sysems wihou disurbing he implemenaions of he oher layers, hus enabling modulariy and reuse. A. Sensor Analysis Sysem We will collec GSR and Body Temperaure. These signals are coninuous numerical values ha capure arousal. We will pass hese signals hrough a filer and will synchronize heir readings and sampling raes. The oupu of his sysem is a signal idenifying arousal in ime sored in XML (Dance Arousal Sae). Lighs are rigged above he sage. Idenifying dancer s posiion and movemen is imporan o allow dynamic lighing conrol as discussed above. Insead of compuing he lighs affecing dancers auomaically, we will manually map lighs o specific ma numbers. Therefore, given a signal from a specific ma, he sysem can easily deermine he lighs affecing he ma. Using his mehod, we can deermine which lighs affec he dancers a any paricular momen in ime. This may be a crude echnique; however, i will suffice as a firs aemp o prooype his sysem. The oupu of his sysem is a lis of ligh IDs of lighs affecing dancers a a specific momen in ime. We anicipae his paricular oupu o be coninuously changing. Therefore, oupu from his sysem will be buffered and fed ino he nex layer for processing as a process wihin he nex layer becomes available. B. Inelligen Virual Lighing Sysem This sysem is based on our earlier work on ELE [23, 24]. The inelligen virual lighing sysem exends ELE o include a emporal modulaion of lighing based on an arousal signal. Before I discuss he inelligen Virual lighing sysem, I will briefly discuss ELE. ELE, Expressive Lighing Engine, is an auomaic inelligen lighing conrol sysem developed based on cinemaic and hearical lighing design heories; i is designed o auomaically selec he number of lighs, heir posiions, colors, and angles. To accomplish his ask, ELE uses lighing design rules formulaed based on a sudy of film and heare lighing. These rules are represened mahemaically in an opimizaion funcion. The use of opimizaion is imporan o balance conflicing lighingdesign goals. While adaping he lighing o he ineracion, ELE also mainains visual coninuiy and syle. Fig. 2. ELE s Archiecure ELE as a black box is illusraed in figure 2. As shown, ELE akes in several parameers, represened as an XML srucure called WAMP. These parameers are as follows: Sage layou or scene graph Locaions of characers Local props ha emi ligh, e.g. windows, orches, lamps Sylisic parameers including: low-key/high-key, overall conras level, overall palee, specific ideal sauraion, warmh, inensiy or hue values for paricular areas in he level or scene Dramaic inensiy of he scene ELE hen emis an XML-based srucure called LAMP, which includes he following: Number of lighs o be used. For each of hese lighs: o ype of insrumen (e.g., spo ligh or poin ligh) o color in RGB color space o aenuaion o posiion as a 3D poin o orienaion including he facing and up vecors o range o o ELE Allocaion Subsysem Angle Subsysem Color Subsysem WAMP (World Acion Message Proocol) Game/Rendering Engine LAMP (Lighing Acion Message Proocol) masking parameers Depending on he ligh insrumen used, he Penumbra and Umbra angles. These parameers are given o a rendering engine o render he frame. As shown in he figure, o configure he lighing in he scene ELE is divided ino hree subsysems: allocaion subsysem used o selec he number of lighs and heir relaive locaion based on he areas in he scene, angle subsysem which selecs angles for each ligh, and color subsysem which selecs colors for each ligh. I will discuss hese subsysems briefly below. Using hearical and cinemaic lighing design heories, ELE uses sage layou or scene graph informaion as well as arisic sylisic consrains o device a ligh layou. I divides he scene ino n differen cylindrical areas. I hen caegorizes hese areas as: focus, describes he focus of he scene, non-focus, areas surrounding he focus area, and 910
background areas. This is imporan because lighing designers ofen use ligh o bring ou he focus, increase deph by varying brighness or color of lighs in differen areas, or increase conras (deermined by colors of lighs lighing focus and non-focus areas). ELE deermines where o direc viewers aenion (or he focus) given he characers in he frame. By aking arisic syle direcions considering wha he aris cares abou, e.g. deph, moivaion, conras, ec., ELE opimizes a muli-objecive funcion o deermine he number of lighs o use for each area. The funcion is as follows: p = argmax λv( p) + λ D( p) + λ M( p) + λ VC( p), ( ) op v d m vc p where p is ligh configuraion, λ v is he imporance of visibiliy, λ d is he imporance of deph, λ m is he imporance of modeling, and λ vc is he imporance of visual coninuiy, and where V(p) is visibiliy given p, D(p) is deph given p, M(p) is modeling given p, and VC(p) is visual coninuiy given p. We formulaed a greedy algorihm ha allocaes lighs o each visible area in he scene, as follows: 1. each area is assigned he maximum number of lighs i can have; 2. remove one ligh ha will incur he smalles loss; and 3. repea sep 2 unil he number of lighs assigned is less han or equal o he maximum. Ofenimes, ariss wan heir lighing design o reflec realisic direcions. This desire can be encoded as an arisic direcion ha ELE hen uses o deermine angle of ligh. In deermining he angle of ligh, ELE also akes ino accoun he qualiy of ligh and heir influence in projecing deph, modeling, and mood. ELE uses a non-linear opimizaion sysem based on hill climbing o selec an angle for each key ligh ha minimizes he following funcion: λ (1 V( k, s)) + λ k k + λ k m + λ min k l, v m l i i where k and s are defined as he key ligh azimuh angle relaive o he camera and he subjec angle relaive o he key ligh, respecively, as shown in figure 3, k - is he key ligh azimuh angle from he previous frame, λ - is he cos of changing he key ligh angle over ime (o enforce visual coninuiy), λ m is he cos of deviaion from he mood azimuh angle, m is he mood azimuh angle suggesed by he aris, λ l is he cos of azimuh angle deviaion from a pracical source direcion, l i is he azimuh angle of ligh emied by he pracical source i, and λ v is he cos of deviaion from an orienaion of ligh ha esablishes bes visibiliy. Based on Millerson s [30] documened rules we formulaed he following equaion o evaluae he visibiliy and modeling of a given key ligh azimuh angle: V( k, s) = sin( k)cos( s). ELE uses rules based on Millerson s [30] guidelines o selec fill and backligh azimuh angles depending on he value of he key ligh angle. According o Millerson s guidelines [30], fill ligh azimuh and elevaion angles are calculaed o be he mirror image of he key ligh angle. We define backligh azimuh angle as: b = ( k+ π)mod2 π. The ineracion beween colors assigned for each area in a scene composes he conras and feeling of he enire image. Using he ideal values and heir associaed coss, ELE uses non-linear opimizaion o search hrough a ninedimensional space of RGB values. I differeniaes among focus colors, non-focus colors, and background areas o selec a color for each individual ligh in he scene. I evaluaes his color by using a muli-objecive cos funcion, where each objecive evaluaes he color agains he lighingdesign goals, including esablishing deph, conforming o color syle and consrains, paralleling dramaic ension, adhering o desired hue, sauraion, and lighness, and mainaining visual coninuiy. The cos funcion is defined as follows: 2 2 1 λd ( Dc ( ) d) + λc( conras φ ( c) δ) + vx ( ) + Pc ( i, ci ), where pc c = λ 1 ( i, i ) λ i { f, n, b} ( Sc ( ) s) λ ( Hc ( ) h) i ( Lc ( ) l) λ ( Wc ( ) w) 2 2 si i i h i i + + 2 2 li i i wi i i 1 ch Ec ( i, ci ), + + λ where c is a vecor of ligh colors for focus f, non-focus n, and background b, and areas a frame. Color c i is represened in RGB color space; S(c) denoes he sauraion of color c; H(c) denoes he hue of color c; L(c) denoes lighness of color c (in RGB color space). ELE uses CIEDE2000, a well-known formula for measuring color difference [31, 32] as follows: 2 2 2 L C H E = + + + R, ks L L ks C C khsh where R = RT f ( C H ) and L, C, and H are CIELAB meric lighness, chroma, and hue differences respecively; S L, S C, S H are weighing funcions for he lighness, chroma, and hue componens; and k L, k C, k H are parameers o be adjused depending on model maerial informaion. The deph, D(c), of a color vecor c is defined as he color difference beween colors lighing he background areas and hose lighing oher areas, formulaed as follows: D() c E( c, c ), = b B n NB b n 911
where B are he indices for background lighs; NB are he indices for non-background lighs; and E is he color difference defined above. Based on he resuls colleced by Kara and Wooen described in [33], we used a muliple, linear regression mehod o formulae color warmh in RGB color space, as follows: R 0.008 R warmh G = 0.0006 G 0.422. B 0.0105 B The opimizaion problem discussed above is a consrainbased opimizaion problem, where he color, c, is consrained o a specific space of values defined by syle (e.g., realisic syle resrics he values of sauraion or hue). ELE uses a boundary mehod o bind he feasible soluions using a barrier funcion v(x), such ha v(x) as x approaches he boundary defined by he feasibiliy region. ELE uses he following formula for v(x): ε ρ j T log( g ( x)). Alhough gradien descen has major drawbacks, including occurrence of oscillaions and being easily suck in a local minimum, ELE uses gradien descen for several reasons. Firs, i provides a fas and simple soluion. Second, a local minimum in his case is preferable because i provides a soluion closer o he older one, hus ensuring visual coninuiy. Third, alernaive mehods rely on he exisence of a second derivaive, which is no necessarily rue in his case. The inelligen virual lighing sysem exends his work o allow manipulaion of lighing in ime as a funcion of arousal. The general idea is o derive a dynamic echnique ha can se arisic sylisic consrains and vary dramaic inensiy as subjec o he arousal inpu from he sensors. Based on a sudy of film and heaer echniques [26, 27, 34-36], we formulaed four major paerns for maching lighing changes o arousal. The decision of which paern o use is lef up o he aris. These paerns are as follows: 1. Arousal mapped linearly o brighness conras beween focus and non-focus areas, i.e. difference in brighness beween colors of lighs lighing focus areas and ohers lighing non-focus areas. 2. Arousal mapped linearly o warm/cool color conras beween focus and non-focus areas, i.e. difference in warm and cool colors of lighs lighing focus areas and ohers lighing non-focus areas. 3. Arousal mapped linearly o sauraion of colors. 4. Arousal mapped linearly o warmh of colors. The oupu of his sysem is a ligh seup, if one did no exis. The ligh seup consiss of number of lighs and heir layou. In addiion, he sysem also deermines angles and colors for each ligh in he ligh seup. This informaion is coninuously changing in ime as dancers move or heir j arousal sae changes. Therefore, his coninuous informaion will be passed o he rendering engine a a rae slower han 30 per second. C. Rendering Engine We have implemened a sysem ha ranslaes he inelligen virual lighing sysem commands o hree engines: Unreal Tournamen, Ogre 3D, and WildTangen. The sysem acceps lighing commands from he inelligen virual lighing sysem and invokes differen mehods in he engines ha se he lighs, heir posiions, heir angles, and colors. Figure 3 and 4 show screenshos where we fabricaed an arousal signal; we used paern 1 and 3 from he lis above, resuls of which are displayed in figures 3 and 4, respecively. Figure 3 shows a simple 3D room rendered using WildTangen, where we linearly varied brighness conras beween he focal poin (cener of he room) and surrounding areas. The figure shows hree screenshos of he room aken a differen poins during he ransiion. Figure 4 shows a firs person shooer rendered using Unreal Tournamen, where we mapped sauraion level o he number of enemies in he scene (i.e. if number of enemies are high he sauraion is high and vise versa). The figure shows four screenshos aken a differen poins wihin he game. Fig. 3. Linearly increasing brighness conras (where cener of room is he focus) Fig. 4. Linearly increasing Sauraion D. Inelligen on-sage Lighing Sysem Using acual sage lighing resrics changes in several ways. Coninuous changes can only be made o brighness hrough he use of dimmers and o angle hrough roaion of he lighing devices. Since we are using a color wheel, coninuous conrol of color is no feasible. Therefore, color changes will be predefined and made on cerain cues or a specific hresholds. Acual color changes can no longer be made in erms of fluid changes in RGB color space as is he oupu of ELE. The color wheel conains weny slos where ariss can insall differen color gels. Therefore, he colors will be chosen apriori and will also be documened in he sysem. The Inelligen on-sage lighing sysem reuses several componens from he inelligen virual lighing sysem, such as he color and angle sysems. In addiion o hese wo 912
sysems, he inelligen on-sage lighing sysem sores values of colors insalled on he color wheels of each ligh wih heir Sauraion, Hue, Brighness, and Warmh color values. In addiion, i also sores he ligh seup given he rigged lighs and heir IDs. Given he ligh IDs of lighs affecing he dancers, he inelligen on-sage lighing sysem compues angles of each ligh affecing he dancer using he same angle sub-sysem as he one used in he inelligen virual lighing sysem. Thus, using film rules o model he dancers wih ligh. The oher lighs on he sage are se o a defaul angle ha creaes a wash on he sage. Reusing he color subsysem, he inelligen on-sage lighing sysem compues RGB color values of each ligh caegory. The inelligen on-sage lighing sysem caegorizes lighs affecing he dancer as focus lighs and oher lighs as non-focus lighs. Given his caegorizaion and he RGB color values and he lis of lighs and heir color gels, we will develop anoher sub-sysem o deermine he bes color gel ha maches he RGB color compued. This algorihm will loop hrough all color gels (20) calculaing he color difference beween desired (r d, g d, b d ) or (H d, S d, L d ) color and RGB color in he color wheel a index i, denoed by (r i, g i. b i ) or (H i, S i, L i ). The i wih he leas difference will be seleced. In his case, he search space is consan, 20 color gels. To calculae he difference beween he colors, we will use he color difference formula oulined [31, 32] as follows: 2 2 2 L C H E = + + + R, ks L L ks C C khsh where R = RT f ( C H ) and L, C, and H are CIELAB meric lighness, chroma, and hue differences respecively; S L, S C, S H are weighing funcions for he lighness, chroma, and hue componens; and k L, k C, k H are parameers o be adjused depending on model maerial informaion. The oupu of his sysem is a lis of ligh IDs; for each ligh ID, an angle and a color gel locaion. This oupu is hen passes hrough he On-sage Lighing Trans sysem which ranslaes hese commands o ligh board Hex code. III. CONCLUSIONS AND FUTURE WORK In his paper, we have discussed a new ambien inelligen environmen we are developing, argeing an adapive ineracive dance space ha expresses dancers inner feelings hrough manipulaion of sage lighing as well as lighing of a virual world projeced around he dancers. The conribuion of he paper is in he archiecure presened which describes an ambien inelligen environmen composed of several layers. Layer 1 is composed of a sensor analysis sysem ha analyzes and synhesizes sensor informaion. Layer 2 is composed of wo inelligen adapive lighing sysems ha use he analyzed sensor informaion o 913 adap on-sage and virual lighing o porray a emporal progression showing dancer s ension level. Layer 3 ranslaes high-level lighing changes o appropriae lighing board commands as well as rendering commands for he sofware engine used o render he projeced 3D imagery. As i can be seen, he described sysem is work in progress. Fuure work includes inegraing he sensor analysis sysem as well as implemening and esing he physical lighing componen and inegraing i o he sysem. Once his is complee, we plan o evaluae he use of his ambien inelligen environmen wihin a dance floor wih acual dancers, aiming a evaluaing is aesheic abiliy and is uiliy in exploring differen forms of expression for ineracive dance. REFERENCES [1] M. Cunningham, P. Kaiser, and S. Eshkar, "Handdrawn Spaces," presened a Siggraph 1998, Special Sessions, 1998. [2] F. Crow and C. Csuri, "Music and Dance Join a Fine Aris and a Pain Machine," IEEE Compuer Graphics and Applicaion, pp. 11-13, 1985. [3] J. Gruen, "Dancevision," in Dance Magazine, vol. 57, 1983, pp. 78-79. [4] S. Meador, T. J. Rogers, K. O'Neal, E. Kur, and C. Cunningham, "Mixing Dance Realiies: Collaboraive Developmen of Live-Moion Capure in a Performing Ars Environmen," ACM Compuers in Enerainmen, vol. 2, 2004. [5] A. Vasilakos and W. Pedrycz, Ambien Inelligence, WIreless, Neworking, Ubiquious Compuing. MA, USA: Arech House Press, 2006. [6] T. Calver and S. Mah, "Life Forms: an Applicaion of Compuer Graphics o Suppor Dance Choreography," presened a Siggraph 96 Visual Proceedings: The ar and inerdisciplinary programs of Siggraph, New Orleans, LO, 1996. [7] R. Ulyae and D. Bianciardi, "Ineracive Dance Club," presened a Siggraph 98, Orlando, Florida, 1998. [8] R. Ulyae and D. Bianciardi, "The Ineracive Dance Club: Avoiding Chaos in a Muli Paricipan Environmen," presened a CHI'01 Workshop New Inerfaces for Musical Expression (NIME'01), 2001. [9] T. Winkler, "Making Moion Musical: Gesure Mapping Sraegies for Ineracive Compuer Music," presened a Proceedings of Inernaional Compuer Music Conference, 1995. [10] T. Winkler, "Creaing Ineracive Dance wih he Very Nervous Sysem," presened a Proceedings of Connecicu College Symposium on Ars and Technology, 1997. [11] T. Winkler, "Moion-Sensing Music: Arisic and Technical Challenges in Two Works for Dance," presened a Proceedings of he Inernaional Compuer Music Conference, 1998. [12] D. Rokeby, "Very Nervous Sysem," 1986. [13] D. Rokeby, "Very Nervous Sysem," Rokeby, David, 1986-1990.
[14] D. Cooper, "Very Nervous Sysem," in Wired, 1995. [15] "Windows Media Player." [16] "WinAmp." [17] S. DiPaola and A. Arya, "Affecive Communicaion Remapping in MusicFace Sysem," presened a Proceedings of European Conference on Elecronic Imaging and he Visual Ars, London, England, 2004. [18] M. G. Wagner and S. Carroll, "DeepWave: Visualizing Music wih VRML," presened a Proceedings of he Sevenh Inernaional Conference on Virual Sysems and Mulimedia (VSMM '01), 2001. [19] L. P. Demers, "Ineracive and Live Accompanimen Ligh for Dance," presened a Dance and Technology Conference, Simon Fraser Universiy, Vancouver, 1993. [20] L. P. Demers and P. Jean, "New Conrol Approaches on Lighing," presened a Shadow Ligh '97, Flemish Opera House, 1997. [21] J. Crawford, T. Schiphors, M. Gofri, and L. P. Demers, "The Shadow Projec," presened a Symposium on Ars and Technology, 1993. [22] L. P. Demers and B. Vorn, "Los Referenial." New York, USA: The Grea Hall of New York Hall of Science, 1998. [23] M. Seif El-Nasr, "Inelligen Lighing for Game Environmens," Journal of Game Developmen, vol. 1, 2005. [24] M. Seif El-Nasr and I. Horswill, "Auomaing Lighing Design for Ineracive Enerainmen," ACM Compuers in Enerainmen, vol. 2, 2004. [25] P. Srinivasan, D. Birchfield, G. Qian, and A. Kidane, "A Pressure Sensing Floor for Ineracive Media Applicaions," presened a ACM SIGCHI Inernaional Conference on Advances in Compuer Enerainmen Technology (ACE), Valencia, Spain, 2005. [26] J. M. Gillee, Designing wih Ligh, 3rd. ed. Mounain View, CA: Mayfield, 1998. [27] J. Alon, Paining wih Ligh. Berkeley: Universiy of California Press, 1995. [28] J. Birn, "Digial Lighing & Rendering," G. Maesri, Ed. Indianapolis: New Riders, 2000. [29] B. Block, The Visual Sory: Seeing he Srucure of Film, TV, and New Media. New York: Focal Press, 2001. [30] G. Millerson, The Technique of Lighing for Telivision and Film, 3rd ed. Oxford: Focus Press, 1991. [31] B. H. Hill, Roger, T., and Vorhagen, F. W., "Comparaive Analysis of he Quanizaion of Color Spaces on he Basis of he CIELAB Color-Difference Formula," ACM Transacions on Graphics, vol. 16, pp. 109-154, 1997. [32] M. R. Luo, Cul, G., and Rigg, B., "The Developmen of CIE 2000 Colour Difference Formula: CIEDE2000," vol. 2000: CIELAB, 2000. [33] E. a. W. Kara, B. R., "Preceived lighness/darkness and warmh/coolness in chromaic experience," 1995. [34] D. Campbell, Technical Theare for Non-echnical People: Allworh Press, 1999. [35] S. Calahan, "Soryelling hrough lighing: a compuer graphics perspecive," presened a Siggraph Course Noes, 1996. [36] B. Brown, Moion Picure and Video Lighing. Boson: Focal Press, 1996. 914