Motion Tracking Systems

Transcription

1 Autonomous Systems Lab Prof. Roland Siegwart Studies on Mechatronics Motion Tracking Systems An overview of motion tracking methods Spring Term 2011 Supervised by: Cédric Pradalier Samir Bouabdallah Authors: Dominic Jud Andreas Michel

2

3 Contents Abstract iii 1 Introduction 1 2 Evaluation criteria Economic criteria Technical criteria Geometric criteria Motion tracking methods Inertial sensing Accelerometers Gyroscopes Inertial Measurement Unit Acoustic sensing Magnetic sensing Optical sensing Laser-based methods Vision methods Radio sensing Specific Motion Tracking Systems Hybrid Systems Inertial and Vision Tracking Inertial and ultrasound tracking Inertial and magnetic tracking The HiBall Tracking System Tracking targets with an array of multi laser sources Ultrasound positioning using transmitter arrays Cricket location-support system Passive RFID positioning system Results Evaluation of the introduced products How to choose the right tracking method Bibliography 29 i

4 ii

5 Abstract Motion tracking systems can trace any moving object. The perfect motion tracking system [2] is described as tiny, self-contained, complete, accurate, fast, immune to occlusions, robust, wireless and cheap. But in reality, such a tracking system is not feasible. We introduce the reader in some good, but not perfect methods using totally different physical basics. Furthermore we evaluate these methods and take a closer look at a few specific applications. iii

6 iv

7 Chapter 1 Introduction There is a lot of research available on the topic of motion tracking. The different methods are based on various physical principles, but nothing really sticks out as the perfect solution. A motion tracker can be based on inertial, acoustic, optical or radio and microwave frequency sensors. Why is that? Shouldn t there be one method that is actually the best choice and therefore has all the attention of the researchers? The answer is found in the broad field of applications for motion trackers. Different tasks need different solutions. No tracker can handle everything efficiently. It is therefore essential to know all the advantages and disadvantages of a system to be able to match them with the requirements, providing the reader with the optimal solution. Furthermore we will give a recommendation of a good tracking system for a few different purposes. These purposes will cover most of the application fields of motion tracking, i.e., virtual reality, navigation, object selection, instrument tracking and avatar animation. The goal of this report is to guide the reader through the process of understanding the physical principles, getting to know some available products and then most importantly choose the right tracking system for his or her project. Thus the structure of this report is as follows. Starting with the methods in chapter 3, the report will then deal with a few available products in chapter 4 and end with the evaluation and the discussion of the application fields in chapter 5. 1

8 Chapter 1. Introduction 2

9 Chapter 2 Evaluation criteria For an evaluation of the different motion tracking approaches described in this article some criteria are needed. Given that there is no perfect tracker available yet and probably never will be, every approach has its specific advantages and disadvantages. Thus not to disadvantage any of them, the criteria are weighted equally. The following criteria are used for the evaluation: Economic Cost Technical Accuracy Data rate Latency Range Reliability Number of tracked degrees of freedom (DOF) Power consumption Robustness Self-containing Ability for tracking multiple objects Geometric Weight Size 2.1 Economic criteria Cost The economical aspect of a project is very important because it often determines whether the project will be shut down or never even started. Thus we can not neglect this criterion, even though in research projects the technical improvement is the top priority. 3

10 Chapter 2. Evaluation criteria Technical criteria Accuracy Tracking an object is mostly used for control systems or an analysis of its path. So it is undeniable that a tracker needs a high accuracy in position and also in orientation. Data rate The speed of a tracking system is specified by the number of measurements it can take in a specific time. Latency The tracking latency is considered in this category. Latency is the time a tracker needs from the actual motion to the output, similar to delay. Range The range of a tracker should obviously be as wide as possible. Reliability Reliability is the ability of a system to perform correctly its required functions under varying conditions. A tracking device has to be reliable, otherwise it is simply useless. Number of tracked DOF The perfect tracker has to be able to track both position and orientation (6 DOF). Power Consumption This criterion is particularly important for mobile applications, because they rely on a battery and not a permanent power source. Robustness The different tracking systems rely on different physical phenomena. Thus they are sensitive to different influences from the environment. For example a camera based tracker will have problems with bright sunlight or reflections. Other external conditions are magnetic fields, acoustic waves etc.. Additionally we will include in this criterion the maximum speed an object can have before the tracking system fails. Self-containing How easy the system is to use and maintain, is strongly depending on how many parts have to be mounted in the environment at different positions. So a tracker without any cables or external cameras etc. would be the aim. Ability for multiple objects The possibility to track more than just one object is often needed. 2.3 Geometric criteria Weight and Size Again this criterion is mostly important for small and mobile applications, i.e., UAV s or head-mounted displays. Thus there is no sense in evaluating a static mounted tracking system by its weight, but the size will still be analysed.

11 Chapter 3 Motion tracking methods 3.1 Inertial sensing Accelerometers An accelerometer is simply a mass attached to a spring with the spring constant k. The displacement x of the mass m from its center position is then measured. Using Hooke s law combined with Newton s law we can derive the acceleration a: (Newton s law) F = ma (3.1) (Hooke s law) With (3.1) and (3.2) the acceleration is: F = kx (3.2) a = kx m (3.3) This sounds fairly simple, but the fact that the spring will only behave in a linear fashion (see equation 3.3) around its null position will give wrong acceleration readings as soon as the acting force on the spring is relatively big. In order to build a more accurate sensor an enhancement is needed. The goal has to be to keep the mass always close around its null position. This is accomplished by a closed-loop system with a forcer and an electromagnetic displacement pickoff. The acceleration can then be determined by the amount of power the forcer needs to hold the mass in place. This kind of approaches are often built using Micro-Electro-Mechanical- System (MEMS). [3] In figure 3.1 the position estimate and signal noise is plotted against the frequency. A high signal-to-noise ratio is desired, thus, looking at figure 3.1, an accelerometer will perform best at high frequencies and the worst at low frequencies. Which means that accelerometers can indicate a movement, when they are actually standing still, and vice versa. This so called drift error during small or no movement is caused by random biases and other phenomena described in section For their weakness at low frequencies, inertial tracking systems are often used in hybrid system as described in

12 Chapter 3. Motion tracking methods 6 Figure 3.1: Logarithmic plots of typical accelerometer-based position signal vs. noise for constant velocity [7] Gyroscopes The first real use for gyroscopes was to help navigate ships, submarines and aeroplanes. They used spinning wheels mounted on gimballed platforms to determine roll φ, pitch θ and yaw ψ from the angles of the gimbal s axe. But these kind of systems are very heavy and big, so there is no use for human or robot tracking. This all changed when MEMS became popular. It introduced a new class of smaller, lighter and cheaper gyroscopes called Coriolis Vibratory Gyroscopes (CVG). Instead of a spinning wheel CVG s use a mass that oscillates at a high frequency, usually in the tens of khz. A pickoff then measures the secondary vibration mode caused by the Coriolis force. Gyroscopes have a lot of advantages, they are self-contained, can be read out at thousands of Hz, have very low jitter and are not noisy at all. This sounds a lot like a perfect tracker, but gyroscopes have one big drawback. They drift when you integrate the angular rates. There are several reason for drift [3]: gyro bias gyro bias instability: Even if the gyro bias is compensated, it will change slowly over time and add additional drift. gyro white noise: When white noise is integrated, the result should be 0 (when integrated over a long enough time), but the mean squared error will grow linearly in time. calibration error: is caused by a wrong scale factor, alignment or linearity. The problem around drift can be minimized by using highly accurate sensors and/or intelligent algorithms which account for the drift. But there will possibly never be a completely drift free gyroscope, thus the user has to reset the sensor at a known position from time to time or a second sensor has to be included, e.g., a compass to correct the yaw drift. In Figure 3.2 Foxlin describes the drift of differently graded gyroscopes. He measures the drift from the different error possibilities by placing the sensors for 20 minutes on a not moving object. The three different types he used are commercial,

13 Inertial sensing Figure 3.2: Comparison of 1-σ random orientation drift performance of commercial (1500 /hr/ hr bias stability), tactical (15 /hr/ hr bias stability) and navigationgrade (0.015 /hr/ hr bias stability) gyros over a 20 minute covariance simulation. [3] tactical and navigation grade sensors and are defined by their different bias stabilities. Commercial grade gyroscopes are the ones used in everyday applications like phones or automobiles. They can only be used for about one minute for head tracking, before the estimates are drifted to far away. The tactical grade gyroscopes hold up for approximately 20 minutes, but they were developed for the use in short range missiles and due to their weight they can not be used for smaller applications. Despite that, the big potential in MEMS technologies will bring the commercial gyroscopes to a new level and they will pick up with the tactical grade ones in the next decade and provide us with a small, cheap and very accurate gyroscope. The main reason for drift is the initial bias uncertainty in the beginning, which is then overlaid by the bias instability. [3] [4] Inertial Measurement Unit An Inertial Measurement Unit (IMU) contains the two sensors described in the previous two sections and 3.1.2, which are placed on three perpendicular axes to keep track of the position and orientation. The IMU combines all the advantages of the two single systems described above, e.g., low latency, high frequency, selfcontained, small, light and robust, but an IMU also drifts. A basic algorithm to derive the pose of an object is shown in figure 3.3. At first the orientation is calculated by integrating the angular rates provided by the gyroscopes. Using roll, pitch and yaw from the this step, the accelerations are rotated from the body into the world frame. The position is then derived by a simple double integration of the world frame acceleration values. [2] [3] [4] The drift of the position can be explained by taking a close look at figure 3.3. As described by Welch et al. [2] the estimates would have drifted in 30 seconds by 4.5 meters, when the bias error of one accelerometer is just 1 milli-g. But where does this bias error come from? An orientation error of 1 milliradian from the integration of the gyroscopes would give a wrong rotation matrix and therefore a wrong gravity compensation on the vertical accelerometer, which results in exactly this m s 2 of acceleration drift.

14 Chapter 3. Motion tracking methods 8 Inputs: Gyroscopes Accelerometers φ, θ, ψ f ẍ, f ÿ, f z Integration φ, θ, ψ φ, θ, ψ Rotate the accelerometer into world frame Remove gravity from vertical accelerometer Double integration gẍ, g ÿ, g z gẍ, g ÿ, g z Orientation Outputs: Position gx, g y, g z Figure 3.3: Basic algorithm to derive the position and orientation from an IMU [2] [3] 3.2 Acoustic sensing Acoustic methods rely on transmission and sensing of sound waves. In general, all systems using sound waves work with time-delay based positioning which means measuring the flight duration of an ultrasonic pulse. There are approaches using a continuous-wave source to determine the distance by measuring the phase shift. But acoustic signals are extremely reflective on walls and other objects, thus the problem named multipath appears. The signal received is often the superposition of the directly sent and maybe several times reflected signal. One can avoid such multipath problems by working with impulses. The trick is to wait until the first impulse arrives and block the rest. This method works for acoustic systems, but not for radio or optical methods, because sound waves are significantly slower than the others. [2] [9] Ultrasound positioning systems can be classified whether they are hybrid systems

15 Magnetic sensing or not: 1) based on ultrasound alone 2) hybrid, combining ultrasound with Radio Frequency (RF) The system implementation of the first class mostly involves portable ultrasound transmitter. Several receivers are distributed in a room and a central processor collects and manipulates all the data to infer the position. In the second case, the receiver sensor is mounted on the portable device and determine its own positioning. Either for the positioning or only for the communication to others, it is required to add another data sending/receiving method, such as RF. A second classification distinguish the positioning principle: a) based on time-delay, angle-of-arrival or time-difference-of-arrival b) based on the ability to communicate, signal level and/or Doppler shift The first one is the dominant method. The properties of such systems are relatively high accuracy (1-30 cm), but low range. The second method identifies only if the portable device is reachable to communicate and/or analyse the remaining signalpower, e.g., to check if someone is in the room. An often used class is 1a. But some shortcomings must be pointed out. Primarily, the ultrasound communications channel works with a low bit rate. This results a low tracking frequency. An often needed application in positioning is to track humans. Hence, the ultrasound transmitter should not come to close to the ears not to exceed the maximum recommended level of ultrasound. Also common is the class 2a. A developed system is introduced in section 4.5. They typically have a high accuracy, low range and easily break down in practice when the range is more then a few meters. This is due to the background noise and the reduction of the received signal level. A classical test is key jingling which proved to be difficult for a lot of acoustic motion tracking systems. [9] 3.3 Magnetic sensing Magnetic systems measure the local magnetic field vector and its absolute value. The sensors achieve this differ in measuring a quasi-static direct current fields or a changing magnetic field produced by an active source. The first class of sensors is called magnetometer. The second types function by measuring the current induced in an electromagnetic coil. The magnetic field vector indicate the orientation of the object relatively to the excitation. To measure the orientation and the position, three orthogonal triaxial coils are used at both the transmitter and receiver. [2] [7] Magnetic tracking methods have certainly their disadvantages in producing such magnetic field strengths and dealing with distortion problems. But there are some essential advantages. The transmitter and the receiver do not have to be in sight. Hence you can track object through obstacles, e.g., humans. Another advantage is to track multiple targets at the same time with no further effort and the user-worn component can be quite small. [2] 3.4 Optical sensing Vision based tracking essentially consist out of two components, a sensor and an emitter or reflector. This systems theoretically work with all the wavelengths shown

16 Chapter 3. Motion tracking methods 10 Figure 3.4: Electromagnetic spectrum [3] in figure 3.4, but only a small section is practical in reality. X-rays have the advantage that they can penetrate non metallic objects like flesh and other light materials, but it is obviously not a good choice because of the radiation and the resulting health issues. The same counts for gamma rays. Ultraviolet (UV) light seems to be feasible for motion tracking purposes, because the ambient visible light can be filtered out, but UV light is blocked by glass and plastic lenses and therefore a UV tracker would need expensive quartz lenses. So all that is left is the visible and infrared (IR) light. But not every wavelength of IR light is suitable either. Mid IR, commonly known as heat radiation, also needs expensive thermal imaging cameras. For all these reasons, the preferred wavelengths for optical motion tracking lie between near IR and visible light. [3] As described above, an optical tracker consists of emitters and sensors. There are two ways of placing these two components, either the sensors are placed on the moving object and the emitters in the environment, or the other way round. The first one of those two approaches is called Inside-Looking-Out and the other one Outside-Looking-In. This description is also applicable to acoustic and microwave sensing. [2] [3] Laser-based methods As mentioned in 3.4, optical systems consist of a light source and an optical sensor. In the laser-based case, an active light source is used and the sensor is a 1D photo-detector. The principle of using a laser to track motions has really good properties: first pro to many other methods is the accuracy of such system, second is the robustness against external conditions (compare to 2.2). A con is the quickly happened occlusion of the laser beam. Depending on the sensing technology, they either could have a high data transfer rate due to the short response time of optronics devices (e.g. photo-detector as used in [1]) or could also have a really bad performance (such as a laser measurement device using an interferometer). The data transfer rate depends on the accuracy. [2] The property of high accuracy leads to quite expensive systems. However, an accuracy to subcentimeter is in robotic application mostly not needed Vision methods The discussion about the different wavelengths was conducted in section 3.4. That leaves us with the different kind of sensors and sources. One can distinguish between active and passive light sources. An active light source has the advantage that the user can choose what kind of light it should emit. For applications with an active light source, near infrared light is preferred as a result of the possibility that can be filtered out from the ambient visible light. Thus these systems will be more immune to influences from the environment and robust due to their own frequency band. Examples for active light sources are light-emitting diodes (LEDs) or simple light bulbs. Passive light sources are noticeable shapes or colors in your environment, e.g., edges and patterns. [2] [3] [8] The light sensors can be analog or digital. The simplest analog or also called

17 Radio sensing non-imaging device is a photo-sensor, which just changes its resistance linearly with the intensity of the light that hits it. By placing a few of these sensor in your environment it is possible to get the position, but it is clearly not the most accurate technique when there is just one sensor, despite of that it is fast and simple. Similar to this approach is the Position Sensing Detector (PSD). It is basically a semiconductor that provides different currents according to the center of the brightest points on the chip. Again using a multiple number of this sensors the position of the light source can be triangulated. The trick in using analog nonimaging sensors is to combine them with an active light source that is switched on and off at a known time to be sure that the bright spot on the sensor is definitively originated from the object we want to track. Or as described above, use IR LEDs as targets and a IR filter in front of the photo-sensor. Examples for digital sensors or also called imaging sensors are Complementary Metal Oxide Semiconductor (CMOS) or Charge-Coupled Device (CCD) cameras. They use dense arrays of pixels to convert light into an electrical charge. But as in every comparison between digital and analog, the digital cameras have the disadvantage that they are discretely sampled and therefore produce a delay. Additionally to this drawback, digital vison based motion tracking needs image processing to extract the motion and position from the image. As described by Murray and Basu in [8] one possibility for this extraction is to subtract the previous from the current image pixel by pixel, which will also remove all the background clutter. Another way is to keep track of edges, colors or patterns. It is now fairly clear that there is another disadvantage in digital sensors, they need a lot of processing power. [2] [3] But it is still recommended to use digital imaging sensors with passive targets for the following reasons introduced by Foxlin [3]: no need for external active light sources no wire, nor wireless data transfer needed (completely self-contained) targets can be uniquely identified works indoors and outdoors even works in unprepared and unknown environments depending on the sophistication of the tracking algorithm, it is possible to track multiple targets with one camera Additionally there is in every analog and digital sensor the trade-off between resolution and the maximal possible field of view (FOV). A higher resolution can be achieved with a sensor that has the pixels closer together. For a wider FOV one can use a wide-angle lens, but it will decrease the resolution. 3.5 Radio sensing In general, electromagnetic wave-based methods can provide long range motion tracking. Additionally, they suffer negligible absorption losses in air and are virtually unaffected by wind and air temperature. [2] The principle of operation usually works with measuring the time-of-flight (compare to acoustic methods 3.2). The big difference to sound waves is the wave velocity which is about a million times faster. This fact hinder the estimation of a sufficient precision. To point this out, we assume to reach a resolution of 1 mm. This would require a sensing process working with 300 GHz. That implies expensive and

18 Chapter 3. Motion tracking methods 12 power-consuming electronics.[2] There are software which provide high resolution without huge sensor counting. GPS 1 is such example. It uses a delay-locked loop (DLL) to keep adjusting the delay τ to minimize the correlation of the incoming signal with a locally generated replica of it. This methods works with sinusoidal signal. Hence the problem of this method is the uselessness in indoor positioning causing influences from multipath (see section 3.2). Reflection are unavoidable indoors. [2] Another method working with ultra-wideband (UWB) uses impulses instead of sinusoidal signals. Typically for UWB is the broad frequency spectrum. The big pro of this strategy is the ability to reject multipath signals (compare to 3.2). [2] UWB sensors are based on propagation and scattering of radio waves. They usually feature a stimulus wave generator for excitation of the test object. Coherent receivers capture the scattered waves and thereby indirectly obtain some information about the scatterers. [14] 1 Global Positioning System

19 Chapter 4 Specific Motion Tracking Systems 4.1 Hybrid Systems Every tracking system has its limitations and weaknesses. By combining two or more tracking devices to a hybrid system, the weakness of one single system can then be compensated by the other one. Producing a tracking system that has a performance over a wide spectrum of applications. Most hybrid systems are based on inertial tracking and extended by some kind of low frequency tracking system that provides absolute position data, e.g., optical or acoustic sensors. Figure 4.1: A qualitative comparison of the performance of inertial vs. optical and acoustic sensors at various motion speeds [7] As described in section 3.1, the performance of inertial tracking systems at high frequencies and fast movements is superior to every other tracker, but they do not provide any useful estimates at low frequencies. Thus they are combined with other sensor that can track slow movements to correct the drift error of the gyroscopes and accelerometers. [7] [4] In the following three subsection we will take a closer look at inertial tracking combined with vision tracking (section 4.1.1), ultrasound tracking (section 4.1.2) and magnetic tracking (section 4.1.3) 13

20 Chapter 4. Specific Motion Tracking Systems Inertial and Vision Tracking This section is based on You et al. [4] which is the continuation of their previous work [5]. This approach was specifically developed for augmented reality 1 (AR), but it could also be used to navigate robots etc. by replacing the recognition of landmarks with an algorithm that detects natural surfaces in the environment. Their system includes a CCD camera (see section 3.4.2) and three orthogonal rate gyroscopes (see section 3.1.2). The camera is sampled at 30 Hz, while the gyroscopes provide the angular rates in all three dimensions at 1 KHz. With this setup it is possible to track slow movements with the camera and fast movement with the inertial sensing component. But now, the hard part is to fuse the two data streams. One would think that an easy approach is a combination of a low-pass filter for the camera and a high-pass filter for the integrals of the gyro sensors. But how do you choose the cut-off frequency? This unanswered question and the fact that there is still an actual data fusion needed led You et al. to the conclusion that this is an unfeasible approach. The second approach, which in the end fits the desired criteria, is a two channel motion filter consisting out of two parallel Extended Kalman Filters (EKF) which share the same predictor but have separate correction channels (as described in figure 4.2). At every time step, the filter predicts the new states and then corrects them individually with the actual measurements from Figure 4.2: Fusion filter framework [5] the camera and the gyro sensors followed by the update of the states. This fusion filter has the following advantages. The two different sampling rates (the camera is sampled roughly thirty times slower than the gyros) can be taken into account by the two separate measurement corrections. Plus this filter can handle incomplete information, i.e., caused by occlusion of the camera, by only relying on the one channel that is still working. Which solves the big robustness issues that tracking systems with just a camera are suffering from. The fusion filter runs either with the angular rates or the integrated relative angles. After conducting a few dynamic test runs the tracking accuracies came out as almost Table 4.1: This table shows the maximum error, average error and error covariance when using angular rates or relative angles Angular rate Relative angle max. error 9.5 pixels 9.93 pixels average error 1.84 pixels 2.18 pixels error covariance 5.32 pixels 5.62 pixels 1 mixture of real and virtual scene elements

21 Hybrid Systems identical, but using the integrated angles is more efficient. The effect on the tracking accuracy of the different gyro measurements is described in table 4.1 and figure 4.3. Figure 4.3: The blue line describes the dynamic tracking error using gyro angular rates and the red line using the integrated relative angles [5] Overall this hybrid system achieves a high tracking stability and robustness that exceeds every single camera motion tracking system Inertial and ultrasound tracking The following extract refers to the paper Constellation T M : A Wide-Range Wireless Motion-Tracking System for Augmented Reality and Virtual Set Applications. [12] The introduced system is a hybrid system using inertial (see section 3.1) and ultrasound sensing (see section 3.2). An extended Kalman filter fuses this data and returns an error minimized estimation. The system (see general overview in figure 4.4) consists of transponder beacons mounted on the ceiling, three tracker units with ultrasonic rangefinder modules (URMs), an InertiaCube T M inertial sensing device and a basic processor collecting and processing all data. The beacons are activated one-at-a-time, controlled by the URMs communicating over infrared signals. Therefore, each beacon owns a unique code and responds to ultrasound pulse in the active mode case. The principle of measuring the time-of-flight is used to estimate the position. The number of transponder beacons should be four or more to guarantee an acceptable estimation. They decided to take three URMs to get all degrees of freedom of a tracked object (position and orientation). Because of the specific usage of the system, software for acquisition is required. Acquisition occurs whenever the powered-up tracker enters or re-enters a room that has transponder beacons. Once there is a successful acquisition, a second software part, the tracking software, takes over. The tracking algorithm is shown in figure 4.5 (compare to figure 3.3 in section 3.1). The most important point is the direct feedthrough of the IMU data to guarantee low latency. The orientation, the gyro biases, the position and the velocity are used as states of the Extended Kalman Filter (EKF) and are computed from the inertial sensing device. Also the covariance matrix of the EKF can be derived. For more information to the inertial part see section 3.1. Next step is to improve the estimated states with the acoustic range measurements. If the range measurement does not match within

22 Chapter 4. Specific Motion Tracking Systems 16 (a) General system idea [12] (b) Schematic overview of hardware [12] Figure 4.4: Setup of the system the tolerance computed from the covariance matrix, it can be rejected. This case occurs due to multipath (described in section 3.2). In fact, the acoustic sensing device listens only for that short time in which the sound wave should arrive. The first sound pulse that arrives is used for the estimation. In general, the assumption is right. Unfortunately, there is occasionally a random background noise or an echo from previous sampling period which arrives before the real pulse and causes a false time-of-flight. Figure 4.5: Software algorithm flow chart [12] Inertial and magnetic tracking We introduce here a system especially made for human motion tracking based on the paper A Real-Time Articulated Human Motion Tracking Using Tri-Axis Inertial/Magnetic Sensors Package. [13] The system connects magnetic with inertial sensing. In figure 4.6, the used integrated sensor pack (ISP) is introduced consisting of an IMU and a tri-axis micro magnetometer. The ISP estimates the pose filtering of the data of the IMU and the magnetometer which measures only the vector of earth s magnetic field and thus returns no information about the position. As filter, an extended kalman filter is used estimating the orientation and the position trough integration. In practice, the measured magnetic vector is affected by nearby ferrous materials in the environment. This distortions can be distinguished between hard iron and soft iron effects. The hard iron distortions arise from permanent magnets and mag-

23 The HiBall Tracking System netized substances fixed in a location relative to the sensors. This influences can be compensated by calibration methods. The hard to compensate distortions are the soft iron effects arising from any magnetically soft material in the environment varying from location to location. Figure 4.6: Hardware design [13] 4.2 The HiBall Tracking System This section is based on the paper High Performance Wide-Area Optical Tracking: The HiBall Tracking System by Welch et al. [6]. Figure 4.7: System overview of the HiBall Tracking System with the ceiling, interface and HiBall s [6] The HiBall Tracking Systems is what we call an Inside-Looking-Out System (see section 3.4 for further information). The sensor is mounted on the head of the user, looking out at the infrared LED s planted in the ceiling or the wall. The third part of the system is the interface, linking the ceiling and the HiBall s. The Inside- Looking-Out approach has the advantage that when the system evolves, the LED panels may be redundant and the HiBall s could track natural features. But in the same time it is a big challenge to build the HiBall as light and small as possible, given that it is worn by a user or put on a mobile application. The HiBall contains six Lateral Effect Photo-Diodes (LEPD) on the lower six faces of the hollow ball and six infrared lenses on the upper six faces. The LEPD is a non-imaging analog photo sensor. It provides the system with two current ratios which describe the position of the brightest centroid on the sensor, thus focus is not an issue and the lenses work over various distances. By flashing the diodes sequently, the position can be estimated from the LEPD s, but the orientation can not be determined equally. The orientation has to be derived from multiple fixed landmarks. The pose estimation is provided by a Kalman filter based prediction-correction approach known as Single-constraint-at-a-time (SCAAT) tracking with an autocalibration. For more detailed information about the recursive pose estimation look

24 Chapter 4. Specific Motion Tracking Systems 18 at chapter 5 of the paper cited at the top of this section. System characteristics (everywhere in a 4.5 by 8.5 meter room with a height variation of 2 meters): Frequency : 2000 pose estimates per second Latency : < 1 ms Accuracy : 0.5 mm and 0.03 of absolute error and noise HiBall s Weight : 300 g LED density : 65 LED/m 2 The robustness of the system is described as quite robust. Caused by the mechanical design trade-off, the FOV of a single sensor is less than 6 degrees. A small FOV increases the resolution, but there have to be enough markers (in this case LEDs with a density of 65 LED/m 2 ) visible for successfully tracking an object (refer to section 3.4.2). Hence, the system loses lock when the error of the pose estimation is even momentary small. Despite that it is possible to track fast maneuvers like jumping etc.. Also blocking parts of the HiBalls FOV by waving the hands in front of it does not force it to lose lock. These specifications are impressing, but we have to keep in mind that this tracking system only works in a specifically prepared environment. 4.3 Tracking targets with an array of multi laser sources This section refers to the paper A Novel Laser-based Tracking Approach for Wide Field of View for Robotics Applications. [1] Figure 4.8: Concept of the laser tracker system the. [1] Figure 4.9: Cross-section in the scanning head. [1] An overview of the set-up is shown in figure 4.8. The system consists of a scanning head, which is fixed on a motorised platform stage, and a reflective target. The scanning head (see figure 4.9) is composed of an array of VCSEL 2 laser diodes positioned around a non-imaging photo detector. The laser diodes are using a serial switching strategy for localising the target. The photo detector measures the 2 VCSEL stands for Vertical Cavity Surface-Emitting Laser

25 Ultrasound positioning using transmitter arrays reflected light and hence, notices if the sent laser beam hits the target. Doing this with each source, you can compute the relative position error (see figure 4.10). Figure 4.10: Computation of the position error vector. [1] To track the target, the system needs a control system, which minimizes the error vector. To achieve this task, a cascaded controller consisting of an angular speed and a position control loop is used. If the target is bigger then the laser circle, a side-effect occurs. Figure 4.11 points this out. The system does not notice whether the beams are at the left, the right or in the middle. Figure 4.11: Computation of the position error vector. [1] 4.4 Ultrasound positioning using transmitter arrays This extract is based on the article Robust ultrasonic indoor positioning using transmitter arrays. [9] The goal of the research reported in this paper is reliable positioning in rooms with a range of 10m and more. A location needing such systems is for example a multibed hospital room to know in which sector a patient or an instrument is located. The principle of this system is indicated in figure To have a more exact idea of the position, two array-based transmitter can be used. The array is configured to transmit data and steer its beam electronically in the array plane. The principle of this beamsteering is clarified in figure To send the beam into a specific sector, a relative delay between the individual elements of the array is applied (e.g. no delay would mean the sector orthogonal to the array). The data is first sent into sector 1 (compare to figure 4.12), then shifts further to sector 2 and so on. The transmitted data consists of the room ID plus the sector ID. The receiver may measure the RSSI-values 3 of the received superposed sound beam. If the receiver is located in the sector the transmitter is aiming to, it comes to a constructive 3 Received Signal Strength Indication

26 Chapter 4. Specific Motion Tracking Systems 20 interference of the beams. Hence the RSSI sensor detect the highest amplitude. Therefore, the receiver compares all different sector attitude and conclude in this way his actual sector. (a) Use of 1 array to divide into sections (b) Use of 2 arrays to position in smaller cells Figure 4.12: Setup: Divide a room into sections Figure 4.13: Generation of steered beam with delay. [9] 4.5 Cricket location-support system This section refers to the paper The Cricket Location-Support System. [10] Cricket is a hybrid system combining the ultrasound and RF methods (see section 3.2). The transmitter beacons are placed in the room publishing location information as RF and ultrasound signals. The listeners receive both two signal types correlating them to each other and infer their current position. One challenge is to overcome the effects of ultrasound multipath (see section 3.2) and RF interference. The principle to determine the position operates as follows: By measuring the oneway propagation time of the ultrasonic signals emitted by a beacon, taking the advantage of the fact that the speed of sound in air is much smaller than the speed of electromagnetic waves (RF) in air. The beacon concurrently transmits the RF signal implying information about its position together with the ultrasound pulse. Because of the different wave speeds, the RF signal is first received by the listener which turns on the ultrasonic receiver. Knowing the difference of arrival, you are able to determine the distance to the beacon. To reduce the effects of interference, a decentralized randomized transmission algorithm is used.

27 Passive RFID positioning system 4.6 Passive RFID positioning system This section is based on the paper Accurate Indoor Position Estimation by Swift- Communication Range Recognition (S-CRR) Method in Passive RFID systems. [11] The proposed method is based on RFID 4 technology (see section 3.5). RFID systems consist of RFID tags, RFID reader/writer and a server, e.g., PC. Each RFID tag is composed of IC memory with a Unique ID, which is related to infomation of the object, and an antenna. The RFID reader consists of a controller and antennas. You can group RFID tags in active and passive tags. The passive tags generate their energy from the electromagnetic waves radiated from RFID reader used for responding the reader. Active tags have their own energy source what the main difference to passive tags is. Hence active ones have the ability of much longer ranges. As in other physical methods, you have the choice of putting the sensing device (in this case the RFID reader) on the object and the transmitters (the RFID tags) on the wall or the RFID readers attached at a wall listening to the tag on the object. RFID readers can communicate with passive RFID tags by sending out a request to RFID tags which replies its object information. The working principle of RFID systems proposed here is simple: The RFID readers antenna is radiating the communication signals at regular intervals whether a RFID tag receiving the signal or not. Because the RFID reader is highly bounded by its communication range (see figure 4.14), he has to rotate searching for replying tags. As soon as you cross the borderline into the communication area (so you have found a RFID tag), you define your first communication boundary angle (see figure 4.14). The rotation of the RFID reader antenna is continued until the reader no longer recognize the tag. This is defined as the second communication boundary angle. Once you have both of them, it is easy to calculate an estimation of the relative difference of position (from reader to tag). There are some weak points of this method: While a RFID reader rotates to estimate the relative position to the tag, the relative position of the tag should not change during this scan. This brings a big time delay to such a positioning system. A further problem is that this system cannot estimate an accurate position without the rotation of the RFID reader. Figure 4.14: Working principle of RFID systems. [11] 4 Radio Frequency IDentification

28 Chapter 4. Specific Motion Tracking Systems 22

29 Chapter 5 Results 5.1 Evaluation of the introduced products We evaluate the introduced products from the previous chapter with the criteria described in chapter 2. To simplify the evaluation, we use the shortcuts as tabled in 5.1. Furthermore, there is somehow a reference needed to make a comparison. We cannot evaluate them by how they handle on one specific task, that would not be fair regarding the different advantages. Therefore the systems are compared to a perfect tracker. Welch et al. introduced a perfect tracker in [2], which they call a Tracker-on-a-Chip (ToC) and it will of course score ten out of ten points in every our criterion. Only one point means that it is completely insufficient. The specifications of a perfect tracker are as follows and mainly based on [2]: Cost Accuracy Data rate Latency Range Reliability DOF Power consumption Robustness Self-containing $1 in large quantities 1 mm in position and 0.1 in orientation Frequency at 1000 Hz Latency lower than 1 ms Tracking its target no matter how far it goes No restrictions in the objects speed or distance 6 (position and orientation) Running on a coin size battery for three years Not influenced by heat, light, sound, magnetic waves etc. No wires nor any additional parts in the environment 23

30 Chapter 5. Results 24 Ability for tracking multiple targets Possible when using multiple ToC s Weight and size 8-pin dual in-line package (1 2 cm 3 ) Table 5.1: Shortcuts of the introduced applications Shortcut In section Section title I/V Hybrid inertial and vision tracking I/U Hybrid inertial and ultrasound tracking I/M Hybrid inertial and magnetic tracking HiBall 4.2 The HiBall Tracking System Laser Array 4.3 Tracking spherical targets with an array of multi laser sources Ultrasound 4.4 Ultrasound positioning using transmitter arrays Cricket 4.5 Cricket location-support system RFID 4.6 Passive RFID positioning system There are some remarks to make, before looking at the evaluation table. First of all, not every criterion of every system can be evaluated (marked with a hyphen). An example for this would be the evaluation of a method that is plugged into a power socket and then evaluate the power consumption. I/V The inertial vison hybrid has the highest rating of all your evaluated systems, because it is probably the most complete tracking system we came across. It is not just small and self-contained, it also has a remarkable accuracy and data rate. Furthermore with its inside looking out approach, that does not rely on any other thing in the environment, it can basically track the motion of the object it is attached to everywhere. The only problem we see in this approach is, that when the camera is blocked for too long, the position estimates will be false. Thus the robustness is not very high. I/U The product is from the year 1998, from the technical point of view half an eternity. One of the goals of this approach was to create a completely untethered tracker, which can be mounted on the head. This seems not to be solved as elegantly as in other approaches. It is able to track multiple objects, but with some effort (e.g. an additional IMU). The system has a clever software, which overrides false acoustic information and makes it reliable. By using inertial sensing, a very high data rate is achieved. I/M The position is inaccurate, because it is only estimated through the integration of the IMU data. The approach is made for outdoor use, because indoor the magnetic field is highly disturbed. The magnetic sensor reacts very sensitive on external

31 Evaluation of the introduced products Table 5.2: Evaluation table Technologies Criteria ToC I/V I/U I/M HiBall Laser Array Ultrasound Cricket RFID Economical Cost Technical Accuracy Data rate Latency Range Reliability Tracked DOF Power Consumption Robustness Self-Containing Multiple Targets Geometric Weight Size Total Points Average

32 Chapter 5. Results 26 influences. The system is not prepared to any distortion coming from an unexpected magnetic object, hence it is not robust. The connection between the sensor pack and the computer is per cable. An advantage is the lightness of the sensor pack. HiBall The HiBall system is the one with the highest accuracy and data rate combined with a low latency. Unfortunately the high performance comes with a lot of disadvantages. It can only have this impressive specifications in a special room equipped with LED s on the ceiling and the interface it needs to communicate. Additionally with 300 grams of weight it is very heavy. Laser array The goal of this project was to reduce the costs at the expense of accuracy, because laser-based tracking systems are often very expensive. However, the accuracy may be smaller than in other laser-based systems, but it is still excellent if you pay attention to the side-effect described in section 4.3. This side-effect implies a special design of the target (most preferred is spherical). This approach also features the other advantages of laser-based products such as a fast data rate, small delay, robust against external influences and fast movements, but it can only keep track of one object at the time. Ultrasound This approach is useful for its specific application (e.g. in hospitals to position instruments or patients). Hence the accuracy is better than room-level, but not much more. Also the data rate is not really made for any other high performance demanding application. However, the important points are achieved, because the receiver is relatively small, light and completely untethered. The use of multiple receivers is also no problem to implement. Cricket One of the goals of this product was low costs. The system is not very exact, has a remarkable delay and a not very fast data rate. An advantage is that it can track multiple objects without much effort. RFID This system is similar to Ultrasound. It s about positioning an object or person in a room. As already described in section 4.6, the relative position can not change during the RFID reader is rotating. This implies a big delay and a low data rate. Although the RFID tag works passive, thus has a low power consumption, the whole system is not as power saving trough the permanent rotating of the RFID reader. An advantage is how simple the system can be extended to track multiple targets.

33 How to choose the right tracking method 5.2 How to choose the right tracking method The question discussed in this section is, which one is the most practical method for a specific application with the according requirements. There is certainly no all purpose tool. We therefore analyse the system requirements and match them to the key benefits of one of the methods described in chapter 3 and 4. Augmented and virtual reality The first task to discuss is head tracking for virtual and augmented reality. Augmented reality is adding a virtual component to the view of the user, e.g. a virtual cup of coffee on a real table. Virtual reality just replaces the whole view of the user with a virtual world. A motion tracking system is needed to recognize the motion of the users head and maybe also body and then change the view accordingly. The difficulties in this task lie in the sophisticated human recognition. Just a short delay or a small offset between the view and the actual movement will be detected by the user. Therefore the chosen tracking system needs to have a high accuracy, Figure 5.1: Amercian soldier training in virtual reality [16] low latency and a high update rate. Generally the motion tracker has to be at least as fast as the human eye, which recognises everything over 60 Hz as flickering free, therefore modern TV s have a frequency of 100 Hz or more. Additionally the user should not be disturbed by a heavy head tracker or restricted in his movements by wires or any other kind of constraint. A small and light solution integrated in the Head-mounted Display (HMD) is preferred. We propose an inertial and vison based hybrid approach as described by Suya et al. in [5]. The gyro sensor can be sampled at thousands of Hertz for a flickering free view. Additionally the extremely low latency of inertial sensors will be of advantage too. The camera is needed for drift correction and provides the system with an absolute position reading. Especially the fact that augmented reality applications need a camera either way makes this hybrid system highly feasible. A cheaper alternative to the vision based approach in virtual reality would be a magnetic sensor. Unfortunately a magnetic tracker has the requested performance only at very close range. Instrument tracking Another application area is instruments tracking. We will take a closer look at the tracking of surgical and industrial instruments. Surgeons could use the help of motion tracking systems for endoscopic surgery or any other high precision procedure, i.e. a hip replacement surgery. There are two possible methods. First for endoscopic surgery, one could think of an ultrasound based system. The advantage is obviously that ultrasound waves can penetrate flesh. Another possibility for any type of high precision surgery would be an optical solution, where LED s or other targets are mounted on the instrument and the patient. This approach fulfils all the requirements with only one drawback being possible occlusion. In the industry however, we propose another approach. The task is actually similar to the one surgeons have to deal with. An industrial worker may have to drill a hole with a hand-held power drill where no machine can be used. High accuracy will be the main criterion, most suitable for this is a laser based method. Furthermore such

34 Chapter 5. Results 28 firms often already have a laser measurement device to survey their work. With a sophisticated laser tracking system all the tasks could be done by only one device reducing the costs considerably. Tracking of people Tracking of people could be used in museums and hospitals for two completely different reasons, but with similar requirements regarding the applied motion tracking system. Doctors and nurses could be tracked in case of an emergency. Thus only calling the nearest person will reduce the response time. However in museums motion tracking is used for example in a audio tour guide application that provides the visitor with interesting informations about the piece of art they are stand near by. Obviously accuracy is in this applications for once not the most important criterion. It would be sufficient to know in which corner of the room a person or object is. The requirements for tracking people are rather a long battery life, price in high quantities, weight and size, than high performance tracking. We propose an acoustic or radio frequency based method, because they are the lightest, cheapest and most power saving options available. Avatar animation Avatar animation is based on motion capturing, where the motion of a person or an animal is recorded and then brought into a computer game or an animation film. This task is highly complex due to the number of joints a person has. About fifty targets are needed for fully capturing the motion. One could even record the mimic of a person. Therefore the main requirements for the motion tracking system we will choose is the possibility to track multiple targets at a high frequency and a high accuracy, both for smooth and flickering free motion in your virtual world of choice, e.g., a soccer player in a computer game or the characters in Figure 5.2: Motion capture of a person [15] an animation film. The most powerful tool for tracking multiple targets is undeniably a vision based approach. The most difficult part of this method is the digital image processing software, that extracts the position from the picture. For a full 3D motion capture an expensive multi camera set-up is needed. A cheaper approach would be a magnetic tracker, because it just needs one emitter due to the fact that magnetic waves can penetrate the human body and will not be blocked like light. Unfortunately the accuracy with magnetic sensors is not as high as with cameras.