AUTOMATIC AND CONTINUOUS PROJECTOR DISPLAY SURFACE CALIBRATION USING EVERY-DAY IMAGERY

Transcription

1 AUOMAIC AND CONINUOUS PROJECOR DISPLAY SURFACE CALIBRAION USING EVERY-DAY IMAGERY Ruigang Yang and Greg Welh he Offie of the Future Projet, Henry Fuhs, PI Deartment of Comuter Siene, CB# 3175 University of North Carolina at Chael Hill Chael Hill, NC, 27599, USA {ryang, htt:// ~welh} ABSRAC Projetor-based dislay systems have been used in omuter grahis for about as long as the field has existed. While rojetor-based systems have many advantages, a signifiant disadvantage is the need to obtain and then adhere to an aurate analytial model of the mehanial setu, inluding the external arameters of the rojetors, and an estimate of the dislay surfae geometry. We introdue a new method for the latter for ontinuous dislay surfae autoalibration. Using a amera that observes the dislay surfae, we math image features in whatever imagery is being rojeted, with the orresonding features that aear on the dislay surfae, to ontinually refine an estimate for the dislay surfae geometry. In effet we enjoy the high signal-to-noise ratio of "strutured" light (without getting to hoose the struture) and the unobtrusive nature of assive orrelation-based methods. he aroah is robust and aurate, and an be realized with ommerial off-the-shelf omonents. he method an be used with a variety of rojetor-based dislays, for sientifi visualization, trade shows, entertainment, tele-immersion, or the Offie of the Future. And although we do not demonstrate it in this aer, we have also been woring on extending the method to inlude ontinual estimation of other system arameters that vary over time. CR Categories and Subjet Desritors: [Image Proessing and Comuter Vision]:Digitization and Image Cature Imaginggeometry; Sanning; [Image Proessingand Comuter Vision]: SeneAnalysis Range data; Shae Additional Keywords and Phrases: Comuter Vision, Image Proessing, Shae Reognition 1. INRODUCION ehnologial and eonomi imrovements are heling to mae rojetor-based dislay systems inreasingly a viable otion for aliations suh as large-sale sientifi visualization, simulation, or entertainment. Examle systems inlude the CAVE [Curz93], the ReAtor Room (and similar systems) by rimensions, the Offie of the Future [Rasar98], the Prineton Dislay Wall [Li00, Saman99], and the Stanford Information Mural [Humh99]. Beyond ermanent fixtures, suh dislay systems are often used for ortable visualization, for examle at onferenes or trade shows. On a muh larger sale, newer and more owerful light rojetors are inreasing oortunities to turn large hysial strutures into temorary rojetor dislay surfaes. For examle, during the millennium elebration in Egyt, the Pyramids were used as dislay surfaes for dynami imagery. While rojetor-based systems offer many advantages over other dislay otions for many aliations, a signifiant disadvantage is the need to obtain an aurate analytial model of the mehanial setu, inluding the external arameters of the rojetors, and a desrition of the dislay surfae. he roblem is that the dislay surfae is not an integral art of a single devie, and therefore it must be initially haraterized, and eriodially monitored. We resent an iterative aroah to automatially determine the dislay surfae geometry, without human intervention, unobtrusively and ontinuously while the system is being used for real wor. We use

2 a Figure 1.(a) An image is diretly rojeted on a urved surfae. (b) he image is orreted (re-wared) based on the dislay surfae estimation. his eliminates the urved distortion. () A simulation shows a desto window distorted due to a shar disontinuity on the dislay surfae. (d) he same window after orretion. ameras in a losed-loo fashion to automate the roess. Given the hysial relationshi between rojetor and a amera, and an initial (rough) estimate of the dislay surfae geometry, we iteratively refine the estimate based on image-based orrelation between the nown rojetor image, and the observed amera image. Seifially we use a Kalman filter to estimate the length of a (arametri) ray from eah rojetor ixel. he result is a omlete 3D desrition of the surfae, allowing one to modify the rojeted imagery so that it aears orret from any given viewoint [Rasar99]. Some exeriments results are shown in Figure 1. b d! Self-alibrating. One started, no human intervention is needed.! Continuous and unobtrusive. Close-loo ontinuous alibration that does not affet the rojeted image quality. When there are visible roblems it orrets them, when there are not, it does nothing.! Robust. We use a Kalman Filter (minimum variane stohasti estimator) to otimally weight the measured orrelation, with a relatively onservative tuning to redue the lielihood of a negative imat from a false orrelation.! Minimal equiment. No need for high-seed ameras or rojetors, or seialized image roessing hardware! Flexible setu. he ameras must be rigid but an be loated relatively asually with reset to the rojetors. he only restrition is that what they annot see they annot be used to alibrate.! Stohasti framewor. Beause the framewor is in lae, other arameters an be added to the list of elements to be estimated. For examle, internal rojetor arameters ould be estimated using tehniques similar to [Azarb95]. Our goal is to imrove the setu and maintenane of onventional rojetor-based dislay systems, and to further enable the rendering of ersetively orreted imagery on more unusual surfaes [Rasar98, Rasar99]. 2. BACKGROUND It is interesting to onsider the inherent aroriateness of this aroah for dislay surfaes. yially, finding feature orresondene using orrelation tehniques is less reliable for images or regions that la high-frequeny ontent. However, for our artiular aliation, it is OK to miss measurement oortunities in suh a region beause if there are no roblemati features for the system to observe, there are none for the human to observe either. When there are notieably distorted features, the user will see them, but so will the system, whih an then aount for them by adjusting the estimate of the dislay surfae. Given a suffiient variation of the rojeted image ontents over time, the system eventually onverges on the atual dislay surfae geometry. Beause our method is non-intrusive, the alibration roess an always been running to maintain an otimal alibration while the system is being used for real wor. Our simulation results (desribed later) redit a high degree of auray, and our atual imlementation aears to agree. Our aroah has the following ey advantages: One an ategorize different alibration methods as assive vs. ative, and online vs. off-line. Ative methods usually injet energy into the environment to aid in the estimation of the surfae roerties, while assive methods use only existing energy in the environment. Off-line alibration methods are erformed while the system is not being used, and on-line methods are erformed onurrently during normal use. See for examle the lassifiation in able 1. On-line Off-line Passive Stereo Mehanial alignment Ative Imeretible strutured light Laser san, Strutured light able 1. Different Calibration Methods he most ommon aroah is to use the off-line, assive aroah of mehanial alignment. In software the develoers assume some (usually simle) geometri model for the dislay surfae and rojetor arrangement. hey then attemt to ensure that the rojetors and dislay surfaes math the model by

3 onstruting and adjusting a rigid mehanial setu [Curz93]. In ratie the reise geometry is not nown to start, and worse it will liely hange over time with hysial erturbations of the environment (building vibrations from ventilation systems, slamming doors, nearby heavy vehiles, et.). Furthermore in some situations develoers want to relax the hysial onstraints of the setu [Rasar98, Rasar99]. In eah ase the reise geometry is not nown to start, and will liely hange over time. As suh we want a means to self or autoalibrate a dislay surfae, ontinuously while the system is in use. Ative means are the most attrative from a signal/noise standoint, however suh methods tyially interfere (visually) with the normal oeration. In [Rasar98, Fuhs99], the authors roosed a new on-line ative alibration method alled imeretible strutured light (ISL). hey desribe the use of time-division-multilexed (DM) digital rojetors that are able to modulate light at a very high rate (over 1000 Hz) to raidly rojet a strutured light attern and its omlement, embedded (in time) in normal DM imagery. he idea is that beause the swithing is so fast, the human visual system will not ereive the strutured light attern. (What a human sees is a normal image.) However a synhronized amera with a fast shutter seed is able to ature the strutured light attern embedded in the DM imagery. Using suh imeretible strutured atterns, the dislay surfae geometry an be estimated. Beause this method hides the atterns within the normal DM imagery, it an be used online while eole are using the system for every day wor. However there are two major disadvantages of this aroah. First, beause it relaes a ortion of the normal DM information with the strutured light attern, it redues the image quality (dynami range and ontrast). Seond, and most ritial, a full imlementation requires seialized hardware that we are unaware of anyone (inluding the authors of [Rasar98, Fuhs99]) having aess to. he basi roblem is that the develoers of the digital rojetor tehnology never intended it to be used in that way, and they have so far been unwilling to rovide the neessary aess to the hardware. Our roosed aroah for auto-alibration utilizes reursive estimation theory, in artiular the extended Kalman filter (EKF). he EKF aroah has roven to be very useful in reovery of rigid motion and struture from image sequenes. Early examles inlude [Matth88] and [Broid91]. A more reent examle is [Alon00], whih builds on the foundations of [Azarb95] but uses features on lanar surfaes to imrove robustness. Our aroah uses a similar arametri framewor to estimate the struture of the dislay surfae (in the absene of motion), using any features that are available in the rojeted imagery. In effet we use the imagery as "unstrutured light" to enable the estimation of the struture of an otherwise featureless surfae. Comared to the ISL method, our aroah enjoys both the signal/noise ratio of strutured light, and the unobtrusiveness of assive aroahes so that it an be used ontinuously without the user nowing, and without imating the quality of the imagery. It requires no seial hardware, and is relatively easy to imlement. 3. APPROACH We model the rojetor dislay surfae as a dense, regular, 3D mesh in the rojetor sae. We tyially use one 3D vertex (V) for eah 2D ixel (Z) inthe rojetor s image lane, although a less dense mesh ould be used if aroriate for the surfae. As deited in Figure 2, we osition the amera so that it an see the entire dislay surfae. During the normal ongoing oeration of the system we ontinuously hoose samle oints in the nown rojetor image and attemt to math them with the orresonding oints in the amera s image of the atual dislay surfae. We do this for one samle oint at eah frame, refining the overall mesh ontinuously over time. u~, ~ v [ ] Z t Projetor O Camera Figure 2. V is the oint where the ray O Z terminates on the dislay surfae. V is uniquely determined by a arametri value t. V s rojetion ( Z ' ) on the amera image lane is onstrained to lie on the eiolar line. Conetually, eah vertex V = [ x,y,z] of the mesh lies on a ray extending from the enter of rojetion of the rojetor (O ) through the orresonding 2D ixel Z in the rojetor's image lane. For eah 2D rojetor ixel Z we want to onverge on, and then ontinuously maintain an aurate estimate for the salar arameter t orresonding to the normalized distane along the ray where the ray terminates on the dislay surfae at V. Figure 2 deits the geometry of the setu. From [Fauge93], we now that for a given 3 4 V Z Eiolar Line u~, ~ v [ O ]

4 rojetion matrix (M ), and a samle oint Z = [ u ~, v~ ] on the rojetor s image lane, if we rewrite M as M [ ~ = ],where is a 3 3 matrix and ~ is a 3 1 vetor, V an be omuted as x u ~ 1 = ( ~ + ~ y t v ) Equation 1 z 1 where t is a arametri salar value. Eah samle oint Z when rojeted onto the dislay surfae has a orresonding rojetion Z in the amera's image sae. Given the rojetion matrix M of the amera we ould estimate t using a traditional orresondene aroah (a standard review is resented in [Dhond89]). Beause the observations of the feature ositions are noisy, and the various system arameters unertain, we use a Kalman filter [Brown97, Maybe79, Welh97a] (a reditororretor based minimum-variane stohasti estimator) to estimate the arametri value t for every rojetor ixel Z and orresonding mesh vertex V. We use the redition ste of the Kalman filter to estimate where the Z should aear on the dislay surfae, and subsequently to limit the feature searh in the amera image to a relatively small region. When a math is found, the differene between the redition and the atual math is used to orret the filter's estimate of the arameter t, and thus the 3D oint V. For eah 2D rojetor ixel Z we use an indeendent Kalman filter to estimate the arametri value t that orresonds to the rojetion of Z on the dislay surfae, as seen in the atual amera measurement. ogether with the rojetor arameters eah arameter t determines the 3D osition of eah mesh vertex V. We use a osition-only (no veloity) dynami model for eah arameter, i.e. we assume that the arameter is a onstant with only a small amount of zero-mean normally distributed white noise erturbing it over time. Beause the ersetive rojetion is not linear, we emloy an Extended Kalman filter (EKF) as desribed in [Brown97, Maybe79, Welh97a]. We use the following measurement model: [ u, v ] = h( M, x, y, z) = [, ] where [ = u s u, v, s] M [ x y z 1] v s Equation 2 Following onventional Kalman filter notation we denote the estimated 1D error ovariane of t as P, and the 2D measurement variane of [ u ~ ], v ~ as R. Assuming the measurement variane is equal and indeendent, we an write R as R r = 0 0 r So the time udate equations are filter are t + 1 P + 1 = t = P + Q And our measurement udate equations are K t = P H = t P = ( I K H ( H + K ( [ u~, v~ ] P ) P H + R ) [ u 1, v ] ) Equation 3 Where H is the Jaobian matrix of the measurement funtion with reset to t, H h( )[ u ] = t h( )[ v t Equation 4 ] he time udate (Equation 3) is used to rediate arametri value t and error ovariane P at the urrent time. he measurement udate (Equation 4) is used to orret the reditions based on the atual measurement. After eah time-measurement udate air, the 3D osition of V is udated orresonding to the new t using Equation 1. We initialize our algorithm with a rough estimate of the dislay surfae (some ratial onstraints), with every arameter t set to 0.5, orresonding to the middle oint between the far and near lane. hen we refine the estimates iteratively over time, measuring a distint oint at eah frame. For eah iteration we do the following: 1. ature an image of the dislay surfae, and mae a oy the ontents of the rojetor s frame buffer; 2. hoose a rojetor ixel V, selet a small samle of neighboring ixels, and searh for the samle in the neighborhood of the redited loation in the amera s image; 3. erform the Kalman Filter udate for the orresonding rojetor ixel; and 4. use 2D Delaunay riangulation [Delau34] to udate the mesh. We reeat this roess ontinuously while the system is being used. Notie that in the time udate (Equation 3) we add a small amount of roess variane Q to omensate for slow hanges in the system. Q should not be zero, or the Kalman filter will ease to udate its estimate of the dislay surfae after it has onverged to a solution. With the added roess variane the filter is never allowed to be.

5 absolutely ertain of the arameters, and so it ontinues to adjust the estimates very slowly, aounting for hanges due to drift or other fators. Seletion of Feature Points Beause of omutational onstraints we annot omute an entire feature set (all rojetor ixels) in one iteration. Instead we selet and udate sequentially a small number of feature oints at eah iteration, in a single-ixel-at-a-time fashion similar to [Welh97b]. he seletion roess has two arts: seudo-random seletion and distane-based seletion. In the seudo-random seletion, we first define a list of samle oints, and then ermute the list. At eah iteration, a number of onseutive oints in the ermuted list are seleted in a way that eah oint has equal lielihood of being udated. In the distane-based seletion, we want to identify ossible outlying oints (large unertainty and large residual) and orret them as soon as ossible. We found that in ratie, suh oints tend to be far away from the orret oints in 3D. We use a seletion roess based on Eulidean distane. We define a maximum neighborhood distane (MND). For every samle oint (Z) that has been udated, we find its losest neighbor (Z n ) that also has been udated at least one. If the distane between Z and Z n is greater than MND, this Z is onsidered a oint with higher unertainty, and added to the seleted oint list. One may argue that this distane-based seletion imoses an assumtion of the dislay surfae geometry no two neighbor oints an be farther than MND but in fat, this seletion only tries to identify ossible outlying oints. If a oint with high unertainty turns out to be a orret one, it will onverge to that osition in subsequent udates. In ratie, we set the MND to be twie the distane between two neighboring feature oints with the initial estimate (t = 0.5). We found this MND wors well for the variety of dislay surfaes we have exerimented with. Prediative Pattern Math One we have a seleted samle oint, we want to find its orresonding oint in the amera image. Using the urrent arametri value t and the estimated error variane P, we omute the losest oint ( Z min ) and the furthest oint ( Z max ), where Z min is omuted as t-sqrt(p ),andz max is omuted as t+sqrt(p ),wheresqrt is the square-root oeration. he two oints Z max and Z min are rojeted ba to the amera image lane, forming a line on the image lane along the eiolar line. We reate a bounding box around this line using the estimated error ovariane P, and then erform the searh within this box. he estimated error variane P will gradually derease as the Kalman filter onverges. he bounding box will orresondingly shrin as P dereases. Consequentially, the searh area will beome smaller and smaller until the system reahes a steady state. For our setu, we use a 16x16 blo around eah seleted samle oint as the orrelation temlate. We use the Matrox Imaging Library (MIL) to erform the attern mathing within the seified bounding box in the amera image. It returns a math with sub-ixel auray. In some ases, there are multile mathes returned by the MIL, all within the bounding box. In suh ase we omute the mean and the standard deviation of these mathes, and if the standard deviation is greater than the measurement variane R, the entire math set is disarded. Otherwise, we use the mean as the final result. Beause MIL's temlate mathing routine searhes within the entire bounding box, sometimes it will return a math that is not on the eiolar line. his is liely due to two fators: there is liely to be error in the arioriestimated internal and external rojetor and amera arameters, and there is some amount of noise (eletroni) in the digitized images. If suh a situation is enountered, we omute its distane to the eiolar line, and if the distane is greater than sqrt(r ) (the resumed measurement variane) the math is disarded. Figure 3 deits suh a situation. Image retifiation is widely used in stereo algorithms. It is a two-dimensional transformation that attemts to align the eiolar lines along (arallel to) image san lines, so that the searh for orresondene is omutationally more tratable. We do not retify our images beause the number of oints we omute at eah iteration is tyially small for our setu, and retifiation would ost more than the seedu it brings in the searh hase. Reall also that our rimary goal is not seed, as the oerations ontinue throughout normal oeration, not as an off-line alibration. We use the MIL library s temlate mathing funtion. Its hierarhial searh algorithm is very fast, and we have found no signifiant differene between a searh along a line or within a box. Finally we he the result returned by the MIL routine to see if it is within our estimated measurement deviation of the eiolar line.

6 Figure 3. A sreenshot of the system during autoalibration of a urved dislay surfae with video imagery. Here the system is udating 10 samle oints er frame. For illustrative uroses we modified the ode to render the bounding box (the searh area) and eiolar line orresonding to eah samle oint. wo andidate mathes are shown: the small square on the left indiates an aeted math, while the dot on the right indiates a rejeted one beause it is too far away from the eiolar line. Rendering Corret Image aeted math rejeted math We use a two-ass tehnique as desribed in [Rasar98] to render ersetively orret images using our ontinually udated estimate of the dislay surfae. o do this, we first need to reate a triangular mesh. We imlemented a san-line based triangulation routine to reate a omlete mesh and let it deform as its verties 3D ositions were being udated. In ratie we found that until the system reahed a steady state, this aroah reated notieable distortions if there was a hole in the mesh. (A hole exists where there are udated oints surrounding oints that have not yet been udated.) o address this we hose to erform a omlete triangulation in run-time. Assuming there is no self-intersetion of the dislay surfae, the triangulation an be erformed in rojetor s sreen sae using a 2D Delaunay triangulation method, whih is relatively simle to imlement, and more robust than its 3D ounterart. Beause rendering is not our rimary fous, we have not yet sent signifiant effort to ahieve fast rendering seeds. We have a very basi OenGL rogram that offers enough to demonstrate that the surfae estimate was orret. (his an be seen in video.) We have identified several laes the rendering ould be otimized, and we also have an eye on ontinually imroving grahis hardware. Fundamentally the rendering and surfae estimation are largely de-ouled in our method. 4. EXPERIMENS RESULS We imlemented our aroah using C++ under Windows N. We initially develoed and tested our algorithm in simulator where we ould erform ontrolled exeriments. We then transformed the simulator into a woring system. (he rimary differene between the simulator and the real setu is that in the real setu we initially have to estimate the external and internal arameters of the amera and the rojetor.) We first resent some results from our simulator, and then some results using our real setu. o mae our results more realisti in our simulation, we used the external and internal arameters of the amera and the rojetor, estimated from the real devies. All of our exeriments (simulated and atual) have the same basi setu: the rojetor is about one meter away from the dislay surfae, and the amera is about 0.6 meters u and to the right of the rojetor, ointing at the dislay surfae. In our simulations we set the roess noise ovariane Q to (1e-5) 2 and the initial error ovariane P to (5e-1) 2 (both have units of arameter t squared), and the measurement variane r to 3 2 (ixels squared). We used a 40 x 30 mesh of feature oints. We erformed two exeriments: one using a lanar dislay surfae with a shar disontinuity, and another using a urved (onave) surfae. o reinfore the indeendene of the aroah from the grahial ontent, we used a short sequene of video from a ommerial film. In ratie the imagery would be the ongoing stream of whatever the user was dislaying 2D windows or 3D grahis. We started the system with estimates that orresonded to the rough 3D bounding boxes of the surfaes, and let the system run about 45 minutes in eah exeriment. In simulation we were able to assess the absolute auray of our results, as shown in able 2.he estimated surfaes are shown in Figure 4 and Figure 5 resetively. Mean Error (mm) Max. Error 1 (mm) Planar Surfae Curved Surfae able 2. Auray of the Simulation 1 he results shown here do not inlude andidate outlier oints seleted by distane-based seletion routine.

7 Figure 6 shows the results for a urved surfae in a real setu. Panels (a) and (b) of Figure 1 show the differene between an unorreted view and a orreted view. (Note that in our real setu we had no aurate ground truth to omare our results with.) Notie how the wall is urved and the window has a disontinuity in Figure 1 (a), and how they aear straight and ontinuous in Figure 1 (b). More results are shown in the video. 5. CONCLUSION Figure 4. Planar surfae simulation. Blue surfae is the atual surfae; red dots are the estimated feature oints. Light blue dots (magnified for illustration urose) are seleted outlying oints deteted by our distane-based heuristi. Figure 5. A bird-eye view of the urved surfae simulation. Beyond large dislay systems suh as [Curz93, Li00] we are exited by the growing roset of grahial imagery dislayed on real surfaes around us [Rasar98, Under97, Under99]. We believe that our aroah to surfae estimation rovides an imortant iee of the uzzle. he aroah is aurate, robust, and an be imlemented in ratie with reasonably ommon omonents and minimal infrastruture. Furthermore, we have done some reliminary wor on extending our method to inlude ontinual estimation of other system arameters, suh as amera and rojetor oses, whih an vary over time. he rimary hallenge is the limited observablity [Soatt94] beause of the very onstrained motion of rojetors and ameras. We are lanning to imose metri onstraints on the state sae to redue the set of indistinguishable states. Beyond the algorithmi imrovements we resent here, we loo forward to imroved hardware. For examle, some day smart rojetors with built-in ameras will be ommon, enabling automati adjustments beyond simle eystone orretion. Some day grahis engines will suort more effiient rendering onto non-lanar (and non-retangular) surfaes, and maybe will even suort automati view-deendent orretion. ACKNOWLEDGEMENS Figure 6. he estimate of a urved dislay surfae after we run our algorithm for over one half hour in a real setu. his researh is suorted by the National Siene Foundation agreement ASC : "Siene and ehnology Center for Comuter Grahis and Sientifi Visualization", Intel Cororation, and the "National ele-immersion Initiative" sonsored by Advaned Networs & Servies, In. We would lie to than members of the Offie of the Future grou at UNC Chael Hill, and in artiular Prof. Henry Fuhs and Herman owles for useful disussion and suort.

8 REFERENCES [Alon00] Jonathan Alon and Stan Slaroff, Reursive Estimation of Motion and Planar Struture, Proeeding of IEEE Conferene on Comuter Vision and Pattern Reognition (CVPR 2000). [Azarb95] Azarbayejani, Ali, and Alex Pentland, June 1995, Reursive Estimation of Motion, Struture, and Foal Length. IEEE rans Pattern Analysis and Mahine Intelligene, June 1995, 17(6). [Broid91]. Broida and R. Chellaa. Estimating the inematis and struture of a rigid objet from a sequene of images. PAMI, 13(6): , [Brown97] Brown, R. G. and P. Y. C. Hwang Introdution to Random Signals and Alied Kalman Filtering, Seond Edition, John Wiley & Sons, In, 3rd edition. [Curz93] Curz-Neira, Carolina et al., 1993 Surround-Sreen Projetion-Based Virtual Reality: he Design and Imlementation of the CAVE, SIGGRAPH Conferene Proeedings, Annual Conferene Series, Addison-Wesley, July [Delau34] B. Delaunay. Sur la sh`ere vide. Izv. Aad. Nau SSSR, Otdelenie Matematihesii i Estestvennya Nau 7 (1934), [Dhond89] U. Dhond and J. Aggrawal. Struture from stereo: a review. IEEE ransations on Systems, Man, and Cyber-netis, 19(6): , [Fauge93] O. Faugeras, hree-dimensional Comuter Vision: A Geometri Viewoint,Cambridge, Massahusetts; MI Press, [Fuhs99] H. Fuhs, M. Livingston, G. Bisho, and G. Welh, "Dynami generation of imeretible strutured light for traing and aquisition of three dimensional sene geometry and surfae harateristis in interative three dimensional omuter grahis aliations". US Patent, US , 1999, [Humh99] Greg Humhreys and Pat Hanrahan, 1999, A Distributed Grahis System for Large iled Dislay, in the Proeeding of IEEE Visualization 1999.Ot [Li00] Kai Li, et al. Early Exerienes and Challenges in Building and Using A Salable Dislay Wall System. IEEE Comuter Grahis and Aliations, vol 20(4), , [Matth88] L. Matthies, R. Szelisi, and. Kanade, Kalman Filter-based Algorithms for Estimating Deth from Image Sequenes, eh. reort CMU-RI-R-88-01, Robotis Institute, Carnegie Mellon University, January 1988 [Maybe79] Maybe, Peter S Stohasti Models, Estima-tion, and Control,Volume1, Aademi Press, In. [Rasar98] Rasar, R. et al., 1998, he Offie of the Future: A Unified Aroah to Image-Based Modeling and Satially Immersive Dislays. SIGGRAPH Conferene Proeedings, Annual Conferene Series, Addison-Wesley, July [Rasar99] R. Rasar, M, Cutts, G. Welh, and W. Stürzlinger, 1999, Effiient Image Generation for Multirojetor and Multisurfae Dislay, in the Proeedings of the Ninth Eurograhis Worsho on Rendering, (Vienna, Austria), June 1998 [Saman99] R. Samanta et al., Load Balaning for Multi-Projetor Rendering Systems, Pro.1999 Eurograhis/Siggrah Worsho on Grahis Hardware, ACM Press, New Yor, Addison-Wesley, Reading, MA, Aug. 1999, (See also htt:// x.html, ited 12 Ot ) [Soatt94] Stefano Soatto. Observability/ identifiability of rigid motion under ersetive rojetion, Pro. of the 33rd IEEE Conf. on Deision and Control, CDC [Under97] J. Underofler, A View from the Luminous Room, Personal ehnologies, Vol.1, No.2, June 1997, [Under99] J. Underofler,B. Ullmer,and H.Ishii, Emaniated Pixels:Real-World Grahis In he Luminous Room, Comuter Grahis, Annual.Conferene.on Comuter Grahis and Interative ehniques, A.Rowood, ed., Siggrah Conferene Pro.,ACM Press, New Yor, Addison-Wesley, Reading, MA, 1999, [Welh97a] Greg Welh and Gary Bisho, An Introdution to Kalman Filter. ehnial Reort, R , Det. of Comuter Siene, University of North Carolina at Chael Hill, Chael Hill, NC [Welh97b] G. Welh and G. Bisho, "SCAA: Inremental raing with Inomlete Information," in Comuter Grahis, Annual Conferene on Comuter Grahis & Interative ehniques,. Whitted, Ed., SIGGRAPH 97 Conferene Proeedings ed. Los Angeles, CA, USA (August 3-8): ACM Press, Addison-Wesley, 1997,