Motion Tracking Systems

Autonomous Systems Lab Prof. Roland Siegwart Studies on Mechatronics Motion Tracking Systems An overview of motion tracking methods Spring Term 2011 Supervised by: Cédric Pradalier Samir Bouabdallah Authors: Dominic Jud Andreas Michel

Contents Abstract iii 1 Introduction 1 2 Evaluation criteria 3 2.1 Economic criteria............................. 3 2.2 Technical criteria............................. 4 2.3 Geometric criteria............................ 4 3 Motion tracking methods 5 3.1 Inertial sensing.............................. 5 3.1.1 Accelerometers.......................... 5 3.1.2 Gyroscopes............................ 6 3.1.3 Inertial Measurement Unit................... 7 3.2 Acoustic sensing............................. 8 3.3 Magnetic sensing............................. 9 3.4 Optical sensing.............................. 9 3.4.1 Laser-based methods....................... 10 3.4.2 Vision methods.......................... 10 3.5 Radio sensing............................... 11 4 Specific Motion Tracking Systems 13 4.1 Hybrid Systems.............................. 13 4.1.1 Inertial and Vision Tracking................... 14 4.1.2 Inertial and ultrasound tracking................ 15 4.1.3 Inertial and magnetic tracking................. 16 4.2 The HiBall Tracking System....................... 17 4.3 Tracking targets with an array of multi laser sources......... 18 4.4 Ultrasound positioning using transmitter arrays............ 19 4.5 Cricket location-support system..................... 20 4.6 Passive RFID positioning system.................... 21 5 Results 23 5.1 Evaluation of the introduced products................. 23 5.2 How to choose the right tracking method............... 27 Bibliography 29 i

ii

Abstract Motion tracking systems can trace any moving object. The perfect motion tracking system [2] is described as tiny, self-contained, complete, accurate, fast, immune to occlusions, robust, wireless and cheap. But in reality, such a tracking system is not feasible. We introduce the reader in some good, but not perfect methods using totally different physical basics. Furthermore we evaluate these methods and take a closer look at a few specific applications. iii

iv

Chapter 1 Introduction There is a lot of research available on the topic of motion tracking. The different methods are based on various physical principles, but nothing really sticks out as the perfect solution. A motion tracker can be based on inertial, acoustic, optical or radio and microwave frequency sensors. Why is that? Shouldn t there be one method that is actually the best choice and therefore has all the attention of the researchers? The answer is found in the broad field of applications for motion trackers. Different tasks need different solutions. No tracker can handle everything efficiently. It is therefore essential to know all the advantages and disadvantages of a system to be able to match them with the requirements, providing the reader with the optimal solution. Furthermore we will give a recommendation of a good tracking system for a few different purposes. These purposes will cover most of the application fields of motion tracking, i.e., virtual reality, navigation, object selection, instrument tracking and avatar animation. The goal of this report is to guide the reader through the process of understanding the physical principles, getting to know some available products and then most importantly choose the right tracking system for his or her project. Thus the structure of this report is as follows. Starting with the methods in chapter 3, the report will then deal with a few available products in chapter 4 and end with the evaluation and the discussion of the application fields in chapter 5. 1

Chapter 1. Introduction 2

Chapter 2 Evaluation criteria For an evaluation of the different motion tracking approaches described in this article some criteria are needed. Given that there is no perfect tracker available yet and probably never will be, every approach has its specific advantages and disadvantages. Thus not to disadvantage any of them, the criteria are weighted equally. The following criteria are used for the evaluation: Economic Cost Technical Accuracy Data rate Latency Range Reliability Number of tracked degrees of freedom (DOF) Power consumption Robustness Self-containing Ability for tracking multiple objects Geometric Weight Size 2.1 Economic criteria Cost The economical aspect of a project is very important because it often determines whether the project will be shut down or never even started. Thus we can not neglect this criterion, even though in research projects the technical improvement is the top priority. 3

Chapter 2. Evaluation criteria 4 2.2 Technical criteria Accuracy Tracking an object is mostly used for control systems or an analysis of its path. So it is undeniable that a tracker needs a high accuracy in position and also in orientation. Data rate The speed of a tracking system is specified by the number of measurements it can take in a specific time. Latency The tracking latency is considered in this category. Latency is the time a tracker needs from the actual motion to the output, similar to delay. Range The range of a tracker should obviously be as wide as possible. Reliability Reliability is the ability of a system to perform correctly its required functions under varying conditions. A tracking device has to be reliable, otherwise it is simply useless. Number of tracked DOF The perfect tracker has to be able to track both position and orientation (6 DOF). Power Consumption This criterion is particularly important for mobile applications, because they rely on a battery and not a permanent power source. Robustness The different tracking systems rely on different physical phenomena. Thus they are sensitive to different influences from the environment. For example a camera based tracker will have problems with bright sunlight or reflections. Other external conditions are magnetic fields, acoustic waves etc.. Additionally we will include in this criterion the maximum speed an object can have before the tracking system fails. Self-containing How easy the system is to use and maintain, is strongly depending on how many parts have to be mounted in the environment at different positions. So a tracker without any cables or external cameras etc. would be the aim. Ability for multiple objects The possibility to track more than just one object is often needed. 2.3 Geometric criteria Weight and Size Again this criterion is mostly important for small and mobile applications, i.e., UAV s or head-mounted displays. Thus there is no sense in evaluating a static mounted tracking system by its weight, but the size will still be analysed.

Chapter 3 Motion tracking methods 3.1 Inertial sensing 3.1.1 Accelerometers An accelerometer is simply a mass attached to a spring with the spring constant k. The displacement x of the mass m from its center position is then measured. Using Hooke s law combined with Newton s law we can derive the acceleration a: (Newton s law) F = ma (3.1) (Hooke s law) With (3.1) and (3.2) the acceleration is: F = kx (3.2) a = kx m (3.3) This sounds fairly simple, but the fact that the spring will only behave in a linear fashion (see equation 3.3) around its null position will give wrong acceleration readings as soon as the acting force on the spring is relatively big. In order to build a more accurate sensor an enhancement is needed. The goal has to be to keep the mass always close around its null position. This is accomplished by a closed-loop system with a forcer and an electromagnetic displacement pickoff. The acceleration can then be determined by the amount of power the forcer needs to hold the mass in place. This kind of approaches are often built using Micro-Electro-Mechanical- System (MEMS). [3] In figure 3.1 the position estimate and signal noise is plotted against the frequency. A high signal-to-noise ratio is desired, thus, looking at figure 3.1, an accelerometer will perform best at high frequencies and the worst at low frequencies. Which means that accelerometers can indicate a movement, when they are actually standing still, and vice versa. This so called drift error during small or no movement is caused by random biases and other phenomena described in section 3.1.3. For their weakness at low frequencies, inertial tracking systems are often used in hybrid system as described in 4.1. 5

Chapter 3. Motion tracking methods 6 Figure 3.1: Logarithmic plots of typical accelerometer-based position signal vs. noise for constant velocity [7] 3.1.2 Gyroscopes The first real use for gyroscopes was to help navigate ships, submarines and aeroplanes. They used spinning wheels mounted on gimballed platforms to determine roll φ, pitch θ and yaw ψ from the angles of the gimbal s axe. But these kind of systems are very heavy and big, so there is no use for human or robot tracking. This all changed when MEMS became popular. It introduced a new class of smaller, lighter and cheaper gyroscopes called Coriolis Vibratory Gyroscopes (CVG). Instead of a spinning wheel CVG s use a mass that oscillates at a high frequency, usually in the tens of khz. A pickoff then measures the secondary vibration mode caused by the Coriolis force. Gyroscopes have a lot of advantages, they are self-contained, can be read out at thousands of Hz, have very low jitter and are not noisy at all. This sounds a lot like a perfect tracker, but gyroscopes have one big drawback. They drift when you integrate the angular rates. There are several reason for drift [3]: gyro bias gyro bias instability: Even if the gyro bias is compensated, it will change slowly over time and add additional drift. gyro white noise: When white noise is integrated, the result should be 0 (when integrated over a long enough time), but the mean squared error will grow linearly in time. calibration error: is caused by a wrong scale factor, alignment or linearity. The problem around drift can be minimized by using highly accurate sensors and/or intelligent algorithms which account for the drift. But there will possibly never be a completely drift free gyroscope, thus the user has to reset the sensor at a known position from time to time or a second sensor has to be included, e.g., a compass to correct the yaw drift. In Figure 3.2 Foxlin describes the drift of differently graded gyroscopes. He measures the drift from the different error possibilities by placing the sensors for 20 minutes on a not moving object. The three different types he used are commercial,

7 3.1. Inertial sensing Figure 3.2: Comparison of 1-σ random orientation drift performance of commercial (1500 /hr/ hr bias stability), tactical (15 /hr/ hr bias stability) and navigationgrade (0.015 /hr/ hr bias stability) gyros over a 20 minute covariance simulation. [3] tactical and navigation grade sensors and are defined by their different bias stabilities. Commercial grade gyroscopes are the ones used in everyday applications like phones or automobiles. They can only be used for about one minute for head tracking, before the estimates are drifted to far away. The tactical grade gyroscopes hold up for approximately 20 minutes, but they were developed for the use in short range missiles and due to their weight they can not be used for smaller applications. Despite that, the big potential in MEMS technologies will bring the commercial gyroscopes to a new level and they will pick up with the tactical grade ones in the next decade and provide us with a small, cheap and very accurate gyroscope. The main reason for drift is the initial bias uncertainty in the beginning, which is then overlaid by the bias instability. [3] [4] 3.1.3 Inertial Measurement Unit An Inertial Measurement Unit (IMU) contains the two sensors described in the previous two sections 3.1.1 and 3.1.2, which are placed on three perpendicular axes to keep track of the position and orientation. The IMU combines all the advantages of the two single systems described above, e.g., low latency, high frequency, selfcontained, small, light and robust, but an IMU also drifts. A basic algorithm to derive the pose of an object is shown in figure 3.3. At first the orientation is calculated by integrating the angular rates provided by the gyroscopes. Using roll, pitch and yaw from the this step, the accelerations are rotated from the body into the world frame. The position is then derived by a simple double integration of the world frame acceleration values. [2] [3] [4] The drift of the position can be explained by taking a close look at figure 3.3. As described by Welch et al. [2] the estimates would have drifted in 30 seconds by 4.5 meters, when the bias error of one accelerometer is just 1 milli-g. But where does this bias error come from? An orientation error of 1 milliradian from the integration of the gyroscopes would give a wrong rotation matrix and therefore a wrong gravity compensation on the vertical accelerometer, which results in exactly this 0.0098 m s 2 of acceleration drift.

Chapter 3. Motion tracking methods 8 Inputs: Gyroscopes Accelerometers φ, θ, ψ f ẍ, f ÿ, f z Integration φ, θ, ψ φ, θ, ψ Rotate the accelerometer into world frame Remove gravity from vertical accelerometer Double integration gẍ, g ÿ, g z gẍ, g ÿ, g z Orientation Outputs: Position gx, g y, g z Figure 3.3: Basic algorithm to derive the position and orientation from an IMU [2] [3] 3.2 Acoustic sensing Acoustic methods rely on transmission and sensing of sound waves. In general, all systems using sound waves work with time-delay based positioning which means measuring the flight duration of an ultrasonic pulse. There are approaches using a continuous-wave source to determine the distance by measuring the phase shift. But acoustic signals are extremely reflective on walls and other objects, thus the problem named multipath appears. The signal received is often the superposition of the directly sent and maybe several times reflected signal. One can avoid such multipath problems by working with impulses. The trick is to wait until the first impulse arrives and block the rest. This method works for acoustic systems, but not for radio or optical methods, because sound waves are significantly slower than the others. [2] [9] Ultrasound positioning systems can be classified whether they are hybrid systems

9 3.3. Magnetic sensing or not: 1) based on ultrasound alone 2) hybrid, combining ultrasound with Radio Frequency (RF) The system implementation of the first class mostly involves portable ultrasound transmitter. Several receivers are distributed in a room and a central processor collects and manipulates all the data to infer the position. In the second case, the receiver sensor is mounted on the portable device and determine its own positioning. Either for the positioning or only for the communication to others, it is required to add another data sending/receiving method, such as RF. A second classification distinguish the positioning principle: a) based on time-delay, angle-of-arrival or time-difference-of-arrival b) based on the ability to communicate, signal level and/or Doppler shift The first one is the dominant method. The properties of such systems are relatively high accuracy (1-30 cm), but low range. The second method identifies only if the portable device is reachable to communicate and/or analyse the remaining signalpower, e.g., to check if someone is in the room. An often used class is 1a. But some shortcomings must be pointed out. Primarily, the ultrasound communications channel works with a low bit rate. This results a low tracking frequency. An often needed application in positioning is to track humans. Hence, the ultrasound transmitter should not come to close to the ears not to exceed the maximum recommended level of ultrasound. Also common is the class 2a. A developed system is introduced in section 4.5. They typically have a high accuracy, low range and easily break down in practice when the range is more then a few meters. This is due to the background noise and the reduction of the received signal level. A classical test is key jingling which proved to be difficult for a lot of acoustic motion tracking systems. [9] 3.3 Magnetic sensing Magnetic systems measure the local magnetic field vector and its absolute value. The sensors achieve this differ in measuring a quasi-static direct current fields or a changing magnetic field produced by an active source. The first class of sensors is called magnetometer. The second types function by measuring the current induced in an electromagnetic coil. The magnetic field vector indicate the orientation of the object relatively to the excitation. To measure the orientation and the position, three orthogonal triaxial coils are used at both the transmitter and receiver. [2] [7] Magnetic tracking methods have certainly their disadvantages in producing such magnetic field strengths and dealing with distortion problems. But there are some essential advantages. The transmitter and the receiver do not have to be in sight. Hence you can track object through obstacles, e.g., humans. Another advantage is to track multiple targets at the same time with no further effort and the user-worn component can be quite small. [2] 3.4 Optical sensing Vision based tracking essentially consist out of two components, a sensor and an emitter or reflector. This systems theoretically work with all the wavelengths shown

Chapter 3. Motion tracking methods 10 Figure 3.4: Electromagnetic spectrum [3] in figure 3.4, but only a small section is practical in reality. X-rays have the advantage that they can penetrate non metallic objects like flesh and other light materials, but it is obviously not a good choice because of the radiation and the resulting health issues. The same counts for gamma rays. Ultraviolet (UV) light seems to be feasible for motion tracking purposes, because the ambient visible light can be filtered out, but UV light is blocked by glass and plastic lenses and therefore a UV tracker would need expensive quartz lenses. So all that is left is the visible and infrared (IR) light. But not every wavelength of IR light is suitable either. Mid IR, commonly known as heat radiation, also needs expensive thermal imaging cameras. For all these reasons, the preferred wavelengths for optical motion tracking lie between near IR and visible light. [3] As described above, an optical tracker consists of emitters and sensors. There are two ways of placing these two components, either the sensors are placed on the moving object and the emitters in the environment, or the other way round. The first one of those two approaches is called Inside-Looking-Out and the other one Outside-Looking-In. This description is also applicable to acoustic and microwave sensing. [2] [3] 3.4.1 Laser-based methods As mentioned in 3.4, optical systems consist of a light source and an optical sensor. In the laser-based case, an active light source is used and the sensor is a 1D photo-detector. The principle of using a laser to track motions has really good properties: first pro to many other methods is the accuracy of such system, second is the robustness against external conditions (compare to 2.2). A con is the quickly happened occlusion of the laser beam. Depending on the sensing technology, they either could have a high data transfer rate due to the short response time of optronics devices (e.g. photo-detector as used in [1]) or could also have a really bad performance (such as a laser measurement device using an interferometer). The data transfer rate depends on the accuracy. [2] The property of high accuracy leads to quite expensive systems. However, an accuracy to subcentimeter is in robotic application mostly not needed. 3.4.2 Vision methods The discussion about the different wavelengths was conducted in section 3.4. That leaves us with the different kind of sensors and sources. One can distinguish between active and passive light sources. An active light source has the advantage that the user can choose what kind of light it should emit. For applications with an active light source, near infrared light is preferred as a result of the possibility that can be filtered out from the ambient visible light. Thus these systems will be more immune to influences from the environment and robust due to their own frequency band. Examples for active light sources are light-emitting diodes (LEDs) or simple light bulbs. Passive light sources are noticeable shapes or colors in your environment, e.g., edges and patterns. [2] [3] [8] The light sensors can be analog or digital. The simplest analog or also called

11 3.5. Radio sensing non-imaging device is a photo-sensor, which just changes its resistance linearly with the intensity of the light that hits it. By placing a few of these sensor in your environment it is possible to get the position, but it is clearly not the most accurate technique when there is just one sensor, despite of that it is fast and simple. Similar to this approach is the Position Sensing Detector (PSD). It is basically a semiconductor that provides different currents according to the center of the brightest points on the chip. Again using a multiple number of this sensors the position of the light source can be triangulated. The trick in using analog nonimaging sensors is to combine them with an active light source that is switched on and off at a known time to be sure that the bright spot on the sensor is definitively originated from the object we want to track. Or as described above, use IR LEDs as targets and a IR filter in front of the photo-sensor. Examples for digital sensors or also called imaging sensors are Complementary Metal Oxide Semiconductor (CMOS) or Charge-Coupled Device (CCD) cameras. They use dense arrays of pixels to convert light into an electrical charge. But as in every comparison between digital and analog, the digital cameras have the disadvantage that they are discretely sampled and therefore produce a delay. Additionally to this drawback, digital vison based motion tracking needs image processing to extract the motion and position from the image. As described by Murray and Basu in [8] one possibility for this extraction is to subtract the previous from the current image pixel by pixel, which will also remove all the background clutter. Another way is to keep track of edges, colors or patterns. It is now fairly clear that there is another disadvantage in digital sensors, they need a lot of processing power. [2] [3] But it is still recommended to use digital imaging sensors with passive targets for the following reasons introduced by Foxlin [3]: no need for external active light sources no wire, nor wireless data transfer needed (completely self-contained) targets can be uniquely identified works indoors and outdoors even works in unprepared and unknown environments depending on the sophistication of the tracking algorithm, it is possible to track multiple targets with one camera Additionally there is in every analog and digital sensor the trade-off between resolution and the maximal possible field of view (FOV). A higher resolution can be achieved with a sensor that has the pixels closer together. For a wider FOV one can use a wide-angle lens, but it will decrease the resolution. 3.5 Radio sensing In general, electromagnetic wave-based methods can provide long range motion tracking. Additionally, they suffer negligible absorption losses in air and are virtually unaffected by wind and air temperature. [2] The principle of operation usually works with measuring the time-of-flight (compare to acoustic methods 3.2). The big difference to sound waves is the wave velocity which is about a million times faster. This fact hinder the estimation of a sufficient precision. To point this out, we assume to reach a resolution of 1 mm. This would require a sensing process working with 300 GHz. That implies expensive and

Chapter 3. Motion tracking methods 12 power-consuming electronics.[2] There are software which provide high resolution without huge sensor counting. GPS 1 is such example. It uses a delay-locked loop (DLL) to keep adjusting the delay τ to minimize the correlation of the incoming signal with a locally generated replica of it. This methods works with sinusoidal signal. Hence the problem of this method is the uselessness in indoor positioning causing influences from multipath (see section 3.2). Reflection are unavoidable indoors. [2] Another method working with ultra-wideband (UWB) uses impulses instead of sinusoidal signals. Typically for UWB is the broad frequency spectrum. The big pro of this strategy is the ability to reject multipath signals (compare to 3.2). [2] UWB sensors are based on propagation and scattering of radio waves. They usually feature a stimulus wave generator for excitation of the test object. Coherent receivers capture the scattered waves and thereby indirectly obtain some information about the scatterers. [14] 1 Global Positioning System

Chapter 4 Specific Motion Tracking Systems 4.1 Hybrid Systems Every tracking system has its limitations and weaknesses. By combining two or more tracking devices to a hybrid system, the weakness of one single system can then be compensated by the other one. Producing a tracking system that has a performance over a wide spectrum of applications. Most hybrid systems are based on inertial tracking and extended by some kind of low frequency tracking system that provides absolute position data, e.g., optical or acoustic sensors. Figure 4.1: A qualitative comparison of the performance of inertial vs. optical and acoustic sensors at various motion speeds [7] As described in section 3.1, the performance of inertial tracking systems at high frequencies and fast movements is superior to every other tracker, but they do not provide any useful estimates at low frequencies. Thus they are combined with other sensor that can track slow movements to correct the drift error of the gyroscopes and accelerometers. [7] [4] In the following three subsection we will take a closer look at inertial tracking combined with vision tracking (section 4.1.1), ultrasound tracking (section 4.1.2) and magnetic tracking (section 4.1.3) 13

Chapter 4. Specific Motion Tracking Systems 14 4.1.1 Inertial and Vision Tracking This section is based on You et al. [4] which is the continuation of their previous work [5]. This approach was specifically developed for augmented reality 1 (AR), but it could also be used to navigate robots etc. by replacing the recognition of landmarks with an algorithm that detects natural surfaces in the environment. Their system includes a CCD camera (see section 3.4.2) and three orthogonal rate gyroscopes (see section 3.1.2). The camera is sampled at 30 Hz, while the gyroscopes provide the angular rates in all three dimensions at 1 KHz. With this setup it is possible to track slow movements with the camera and fast movement with the inertial sensing component. But now, the hard part is to fuse the two data streams. One would think that an easy approach is a combination of a low-pass filter for the camera and a high-pass filter for the integrals of the gyro sensors. But how do you choose the cut-off frequency? This unanswered question and the fact that there is still an actual data fusion needed led You et al. to the conclusion that this is an unfeasible approach. The second approach, which in the end fits the desired criteria, is a two channel motion filter consisting out of two parallel Extended Kalman Filters (EKF) which share the same predictor but have separate correction channels (as described in figure 4.2). At every time step, the filter predicts the new states and then corrects them individually with the actual measurements from Figure 4.2: Fusion filter framework [5] the camera and the gyro sensors followed by the update of the states. This fusion filter has the following advantages. The two different sampling rates (the camera is sampled roughly thirty times slower than the gyros) can be taken into account by the two separate measurement corrections. Plus this filter can handle incomplete information, i.e., caused by occlusion of the camera, by only relying on the one channel that is still working. Which solves the big robustness issues that tracking systems with just a camera are suffering from. The fusion filter runs either with the angular rates or the integrated relative angles. After conducting a few dynamic test runs the tracking accuracies came out as almost Table 4.1: This table shows the maximum error, average error and error covariance when using angular rates or relative angles Angular rate Relative angle max. error 9.5 pixels 9.93 pixels average error 1.84 pixels 2.18 pixels error covariance 5.32 pixels 5.62 pixels 1 mixture of real and virtual scene elements

15 4.1. Hybrid Systems identical, but using the integrated angles is more efficient. The effect on the tracking accuracy of the different gyro measurements is described in table 4.1 and figure 4.3. Figure 4.3: The blue line describes the dynamic tracking error using gyro angular rates and the red line using the integrated relative angles [5] Overall this hybrid system achieves a high tracking stability and robustness that exceeds every single camera motion tracking system. 4.1.2 Inertial and ultrasound tracking The following extract refers to the paper Constellation T M : A Wide-Range Wireless Motion-Tracking System for Augmented Reality and Virtual Set Applications. [12] The introduced system is a hybrid system using inertial (see section 3.1) and ultrasound sensing (see section 3.2). An extended Kalman filter fuses this data and returns an error minimized estimation. The system (see general overview in figure 4.4) consists of transponder beacons mounted on the ceiling, three tracker units with ultrasonic rangefinder modules (URMs), an InertiaCube T M inertial sensing device and a basic processor collecting and processing all data. The beacons are activated one-at-a-time, controlled by the URMs communicating over infrared signals. Therefore, each beacon owns a unique code and responds to ultrasound pulse in the active mode case. The principle of measuring the time-of-flight is used to estimate the position. The number of transponder beacons should be four or more to guarantee an acceptable estimation. They decided to take three URMs to get all degrees of freedom of a tracked object (position and orientation). Because of the specific usage of the system, software for acquisition is required. Acquisition occurs whenever the powered-up tracker enters or re-enters a room that has transponder beacons. Once there is a successful acquisition, a second software part, the tracking software, takes over. The tracking algorithm is shown in figure 4.5 (compare to figure 3.3 in section 3.1). The most important point is the direct feedthrough of the IMU data to guarantee low latency. The orientation, the gyro biases, the position and the velocity are used as states of the Extended Kalman Filter (EKF) and are computed from the inertial sensing device. Also the covariance matrix of the EKF can be derived. For more information to the inertial part see section 3.1. Next step is to improve the estimated states with the acoustic range measurements. If the range measurement does not match within

Chapter 4. Specific Motion Tracking Systems 16 (a) General system idea [12] (b) Schematic overview of hardware [12] Figure 4.4: Setup of the system the tolerance computed from the covariance matrix, it can be rejected. This case occurs due to multipath (described in section 3.2). In fact, the acoustic sensing device listens only for that short time in which the sound wave should arrive. The first sound pulse that arrives is used for the estimation. In general, the assumption is right. Unfortunately, there is occasionally a random background noise or an echo from previous sampling period which arrives before the real pulse and causes a false time-of-flight. Figure 4.5: Software algorithm flow chart [12] 4.1.3 Inertial and magnetic tracking We introduce here a system especially made for human motion tracking based on the paper A Real-Time Articulated Human Motion Tracking Using Tri-Axis Inertial/Magnetic Sensors Package. [13] The system connects magnetic with inertial sensing. In figure 4.6, the used integrated sensor pack (ISP) is introduced consisting of an IMU and a tri-axis micro magnetometer. The ISP estimates the pose filtering of the data of the IMU and the magnetometer which measures only the vector of earth s magnetic field and thus returns no information about the position. As filter, an extended kalman filter is used estimating the orientation and the position trough integration. In practice, the measured magnetic vector is affected by nearby ferrous materials in the environment. This distortions can be distinguished between hard iron and soft iron effects. The hard iron distortions arise from permanent magnets and mag-

17 4.2. The HiBall Tracking System netized substances fixed in a location relative to the sensors. This influences can be compensated by calibration methods. The hard to compensate distortions are the soft iron effects arising from any magnetically soft material in the environment varying from location to location. Figure 4.6: Hardware design [13] 4.2 The HiBall Tracking System This section is based on the paper High Performance Wide-Area Optical Tracking: The HiBall Tracking System by Welch et al. [6]. Figure 4.7: System overview of the HiBall Tracking System with the ceiling, interface and HiBall s [6] The HiBall Tracking Systems is what we call an Inside-Looking-Out System (see section 3.4 for further information). The sensor is mounted on the head of the user, looking out at the infrared LED s planted in the ceiling or the wall. The third part of the system is the interface, linking the ceiling and the HiBall s. The Inside- Looking-Out approach has the advantage that when the system evolves, the LED panels may be redundant and the HiBall s could track natural features. But in the same time it is a big challenge to build the HiBall as light and small as possible, given that it is worn by a user or put on a mobile application. The HiBall contains six Lateral Effect Photo-Diodes (LEPD) on the lower six faces of the hollow ball and six infrared lenses on the upper six faces. The LEPD is a non-imaging analog photo sensor. It provides the system with two current ratios which describe the position of the brightest centroid on the sensor, thus focus is not an issue and the lenses work over various distances. By flashing the diodes sequently, the position can be estimated from the LEPD s, but the orientation can not be determined equally. The orientation has to be derived from multiple fixed landmarks. The pose estimation is provided by a Kalman filter based prediction-correction approach known as Single-constraint-at-a-time (SCAAT) tracking with an autocalibration. For more detailed information about the recursive pose estimation look

Chapter 4. Specific Motion Tracking Systems 18 at chapter 5 of the paper cited at the top of this section. System characteristics (everywhere in a 4.5 by 8.5 meter room with a height variation of 2 meters): Frequency : 2000 pose estimates per second Latency : < 1 ms Accuracy : 0.5 mm and 0.03 of absolute error and noise HiBall s Weight : 300 g LED density : 65 LED/m 2 The robustness of the system is described as quite robust. Caused by the mechanical design trade-off, the FOV of a single sensor is less than 6 degrees. A small FOV increases the resolution, but there have to be enough markers (in this case LEDs with a density of 65 LED/m 2 ) visible for successfully tracking an object (refer to section 3.4.2). Hence, the system loses lock when the error of the pose estimation is even momentary small. Despite that it is possible to track fast maneuvers like jumping etc.. Also blocking parts of the HiBalls FOV by waving the hands in front of it does not force it to lose lock. These specifications are impressing, but we have to keep in mind that this tracking system only works in a specifically prepared environment. 4.3 Tracking targets with an array of multi laser sources This section refers to the paper A Novel Laser-based Tracking Approach for Wide Field of View for Robotics Applications. [1] Figure 4.8: Concept of the laser tracker system the. [1] Figure 4.9: Cross-section in the scanning head. [1] An overview of the set-up is shown in figure 4.8. The system consists of a scanning head, which is fixed on a motorised platform stage, and a reflective target. The scanning head (see figure 4.9) is composed of an array of VCSEL 2 laser diodes positioned around a non-imaging photo detector. The laser diodes are using a serial switching strategy for localising the target. The photo detector measures the 2 VCSEL stands for Vertical Cavity Surface-Emitting Laser

19 4.4. Ultrasound positioning using transmitter arrays reflected light and hence, notices if the sent laser beam hits the target. Doing this with each source, you can compute the relative position error (see figure 4.10). Figure 4.10: Computation of the position error vector. [1] To track the target, the system needs a control system, which minimizes the error vector. To achieve this task, a cascaded controller consisting of an angular speed and a position control loop is used. If the target is bigger then the laser circle, a side-effect occurs. Figure 4.11 points this out. The system does not notice whether the beams are at the left, the right or in the middle. Figure 4.11: Computation of the position error vector. [1] 4.4 Ultrasound positioning using transmitter arrays This extract is based on the article Robust ultrasonic indoor positioning using transmitter arrays. [9] The goal of the research reported in this paper is reliable positioning in rooms with a range of 10m and more. A location needing such systems is for example a multibed hospital room to know in which sector a patient or an instrument is located. The principle of this system is indicated in figure 4.12. To have a more exact idea of the position, two array-based transmitter can be used. The array is configured to transmit data and steer its beam electronically in the array plane. The principle of this beamsteering is clarified in figure 4.13. To send the beam into a specific sector, a relative delay between the individual elements of the array is applied (e.g. no delay would mean the sector orthogonal to the array). The data is first sent into sector 1 (compare to figure 4.12), then shifts further to sector 2 and so on. The transmitted data consists of the room ID plus the sector ID. The receiver may measure the RSSI-values 3 of the received superposed sound beam. If the receiver is located in the sector the transmitter is aiming to, it comes to a constructive 3 Received Signal Strength Indication

Chapter 4. Specific Motion Tracking Systems 20 interference of the beams. Hence the RSSI sensor detect the highest amplitude. Therefore, the receiver compares all different sector attitude and conclude in this way his actual sector. (a) Use of 1 array to divide into sections (b) Use of 2 arrays to position in smaller cells Figure 4.12: Setup: Divide a room into sections Figure 4.13: Generation of steered beam with delay. [9] 4.5 Cricket location-support system This section refers to the paper The Cricket Location-Support System. [10] Cricket is a hybrid system combining the ultrasound and RF methods (see section 3.2). The transmitter beacons are placed in the room publishing location information as RF and ultrasound signals. The listeners receive both two signal types correlating them to each other and infer their current position. One challenge is to overcome the effects of ultrasound multipath (see section 3.2) and RF interference. The principle to determine the position operates as follows: By measuring the oneway propagation time of the ultrasonic signals emitted by a beacon, taking the advantage of the fact that the speed of sound in air is much smaller than the speed of electromagnetic waves (RF) in air. The beacon concurrently transmits the RF signal implying information about its position together with the ultrasound pulse. Because of the different wave speeds, the RF signal is first received by the listener which turns on the ultrasonic receiver. Knowing the difference of arrival, you are able to determine the distance to the beacon. To reduce the effects of interference, a decentralized randomized transmission algorithm is used.

21 4.6. Passive RFID positioning system 4.6 Passive RFID positioning system This section is based on the paper Accurate Indoor Position Estimation by Swift- Communication Range Recognition (S-CRR) Method in Passive RFID systems. [11] The proposed method is based on RFID 4 technology (see section 3.5). RFID systems consist of RFID tags, RFID reader/writer and a server, e.g., PC. Each RFID tag is composed of IC memory with a Unique ID, which is related to infomation of the object, and an antenna. The RFID reader consists of a controller and antennas. You can group RFID tags in active and passive tags. The passive tags generate their energy from the electromagnetic waves radiated from RFID reader used for responding the reader. Active tags have their own energy source what the main difference to passive tags is. Hence active ones have the ability of much longer ranges. As in other physical methods, you have the choice of putting the sensing device (in this case the RFID reader) on the object and the transmitters (the RFID tags) on the wall or the RFID readers attached at a wall listening to the tag on the object. RFID readers can communicate with passive RFID tags by sending out a request to RFID tags which replies its object information. The working principle of RFID systems proposed here is simple: The RFID readers antenna is radiating the communication signals at regular intervals whether a RFID tag receiving the signal or not. Because the RFID reader is highly bounded by its communication range (see figure 4.14), he has to rotate searching for replying tags. As soon as you cross the borderline into the communication area (so you have found a RFID tag), you define your first communication boundary angle (see figure 4.14). The rotation of the RFID reader antenna is continued until the reader no longer recognize the tag. This is defined as the second communication boundary angle. Once you have both of them, it is easy to calculate an estimation of the relative difference of position (from reader to tag). There are some weak points of this method: While a RFID reader rotates to estimate the relative position to the tag, the relative position of the tag should not change during this scan. This brings a big time delay to such a positioning system. A further problem is that this system cannot estimate an accurate position without the rotation of the RFID reader. Figure 4.14: Working principle of RFID systems. [11] 4 Radio Frequency IDentification

Chapter 4. Specific Motion Tracking Systems 22

Chapter 5 Results 5.1 Evaluation of the introduced products We evaluate the introduced products from the previous chapter with the criteria described in chapter 2. To simplify the evaluation, we use the shortcuts as tabled in 5.1. Furthermore, there is somehow a reference needed to make a comparison. We cannot evaluate them by how they handle on one specific task, that would not be fair regarding the different advantages. Therefore the systems are compared to a perfect tracker. Welch et al. introduced a perfect tracker in [2], which they call a Tracker-on-a-Chip (ToC) and it will of course score ten out of ten points in every our criterion. Only one point means that it is completely insufficient. The specifications of a perfect tracker are as follows and mainly based on [2]: Cost Accuracy Data rate Latency Range Reliability DOF Power consumption Robustness Self-containing $1 in large quantities 1 mm in position and 0.1 in orientation Frequency at 1000 Hz Latency lower than 1 ms Tracking its target no matter how far it goes No restrictions in the objects speed or distance 6 (position and orientation) Running on a coin size battery for three years Not influenced by heat, light, sound, magnetic waves etc. No wires nor any additional parts in the environment 23

Chapter 5. Results 24 Ability for tracking multiple targets Possible when using multiple ToC s Weight and size 8-pin dual in-line package (1 2 cm 3 ) Table 5.1: Shortcuts of the introduced applications Shortcut In section Section title I/V 4.1.1 Hybrid inertial and vision tracking I/U 4.1.2 Hybrid inertial and ultrasound tracking I/M 4.1.3 Hybrid inertial and magnetic tracking HiBall 4.2 The HiBall Tracking System Laser Array 4.3 Tracking spherical targets with an array of multi laser sources Ultrasound 4.4 Ultrasound positioning using transmitter arrays Cricket 4.5 Cricket location-support system RFID 4.6 Passive RFID positioning system There are some remarks to make, before looking at the evaluation table. First of all, not every criterion of every system can be evaluated (marked with a hyphen). An example for this would be the evaluation of a method that is plugged into a power socket and then evaluate the power consumption. I/V The inertial vison hybrid has the highest rating of all your evaluated systems, because it is probably the most complete tracking system we came across. It is not just small and self-contained, it also has a remarkable accuracy and data rate. Furthermore with its inside looking out approach, that does not rely on any other thing in the environment, it can basically track the motion of the object it is attached to everywhere. The only problem we see in this approach is, that when the camera is blocked for too long, the position estimates will be false. Thus the robustness is not very high. I/U The product is from the year 1998, from the technical point of view half an eternity. One of the goals of this approach was to create a completely untethered tracker, which can be mounted on the head. This seems not to be solved as elegantly as in other approaches. It is able to track multiple objects, but with some effort (e.g. an additional IMU). The system has a clever software, which overrides false acoustic information and makes it reliable. By using inertial sensing, a very high data rate is achieved. I/M The position is inaccurate, because it is only estimated through the integration of the IMU data. The approach is made for outdoor use, because indoor the magnetic field is highly disturbed. The magnetic sensor reacts very sensitive on external

25 5.1. Evaluation of the introduced products Table 5.2: Evaluation table Technologies Criteria ToC I/V I/U I/M HiBall Laser Array Ultrasound Cricket RFID Economical Cost 10 7 - - - - - 8 - Technical Accuracy 10 8 5 3 10 8 1 6 6 Data rate 10 8 9 7 8 10 1 6 1 Latency 10 8 6 7 8 10 1 6 2 Range 10 10 3 7 3 6 4 5 6 Reliability 10 6 8 5 7 6 6 6 7 Tracked DOF 10 10 10 10 10 5 5 5 5 Power Consumption 10 5 7 - - - 9-6 Robustness 10 6 5 2 7 8 3 4 8 Self-Containing 10 10 7 4 3 9 7 7 7 Multiple Targets 10 1 5 5 5 1 8 9 9 Geometric Weight 10 8 4 8 5 4 7-8 Size 10 8 4 7 6 7 7-7 Total Points 130 98 73 65 72 74 59 62 72 Average 10 7.54 6.08 5.91 6.55 6.73 4.92 6.2 6

Chapter 5. Results 26 influences. The system is not prepared to any distortion coming from an unexpected magnetic object, hence it is not robust. The connection between the sensor pack and the computer is per cable. An advantage is the lightness of the sensor pack. HiBall The HiBall system is the one with the highest accuracy and data rate combined with a low latency. Unfortunately the high performance comes with a lot of disadvantages. It can only have this impressive specifications in a special room equipped with LED s on the ceiling and the interface it needs to communicate. Additionally with 300 grams of weight it is very heavy. Laser array The goal of this project was to reduce the costs at the expense of accuracy, because laser-based tracking systems are often very expensive. However, the accuracy may be smaller than in other laser-based systems, but it is still excellent if you pay attention to the side-effect described in section 4.3. This side-effect implies a special design of the target (most preferred is spherical). This approach also features the other advantages of laser-based products such as a fast data rate, small delay, robust against external influences and fast movements, but it can only keep track of one object at the time. Ultrasound This approach is useful for its specific application (e.g. in hospitals to position instruments or patients). Hence the accuracy is better than room-level, but not much more. Also the data rate is not really made for any other high performance demanding application. However, the important points are achieved, because the receiver is relatively small, light and completely untethered. The use of multiple receivers is also no problem to implement. Cricket One of the goals of this product was low costs. The system is not very exact, has a remarkable delay and a not very fast data rate. An advantage is that it can track multiple objects without much effort. RFID This system is similar to Ultrasound. It s about positioning an object or person in a room. As already described in section 4.6, the relative position can not change during the RFID reader is rotating. This implies a big delay and a low data rate. Although the RFID tag works passive, thus has a low power consumption, the whole system is not as power saving trough the permanent rotating of the RFID reader. An advantage is how simple the system can be extended to track multiple targets.

27 5.2. How to choose the right tracking method 5.2 How to choose the right tracking method The question discussed in this section is, which one is the most practical method for a specific application with the according requirements. There is certainly no all purpose tool. We therefore analyse the system requirements and match them to the key benefits of one of the methods described in chapter 3 and 4. Augmented and virtual reality The first task to discuss is head tracking for virtual and augmented reality. Augmented reality is adding a virtual component to the view of the user, e.g. a virtual cup of coffee on a real table. Virtual reality just replaces the whole view of the user with a virtual world. A motion tracking system is needed to recognize the motion of the users head and maybe also body and then change the view accordingly. The difficulties in this task lie in the sophisticated human recognition. Just a short delay or a small offset between the view and the actual movement will be detected by the user. Therefore the chosen tracking system needs to have a high accuracy, Figure 5.1: Amercian soldier training in virtual reality [16] low latency and a high update rate. Generally the motion tracker has to be at least as fast as the human eye, which recognises everything over 60 Hz as flickering free, therefore modern TV s have a frequency of 100 Hz or more. Additionally the user should not be disturbed by a heavy head tracker or restricted in his movements by wires or any other kind of constraint. A small and light solution integrated in the Head-mounted Display (HMD) is preferred. We propose an inertial and vison based hybrid approach as described by Suya et al. in [5]. The gyro sensor can be sampled at thousands of Hertz for a flickering free view. Additionally the extremely low latency of inertial sensors will be of advantage too. The camera is needed for drift correction and provides the system with an absolute position reading. Especially the fact that augmented reality applications need a camera either way makes this hybrid system highly feasible. A cheaper alternative to the vision based approach in virtual reality would be a magnetic sensor. Unfortunately a magnetic tracker has the requested performance only at very close range. Instrument tracking Another application area is instruments tracking. We will take a closer look at the tracking of surgical and industrial instruments. Surgeons could use the help of motion tracking systems for endoscopic surgery or any other high precision procedure, i.e. a hip replacement surgery. There are two possible methods. First for endoscopic surgery, one could think of an ultrasound based system. The advantage is obviously that ultrasound waves can penetrate flesh. Another possibility for any type of high precision surgery would be an optical solution, where LED s or other targets are mounted on the instrument and the patient. This approach fulfils all the requirements with only one drawback being possible occlusion. In the industry however, we propose another approach. The task is actually similar to the one surgeons have to deal with. An industrial worker may have to drill a hole with a hand-held power drill where no machine can be used. High accuracy will be the main criterion, most suitable for this is a laser based method. Furthermore such

Chapter 5. Results 28 firms often already have a laser measurement device to survey their work. With a sophisticated laser tracking system all the tasks could be done by only one device reducing the costs considerably. Tracking of people Tracking of people could be used in museums and hospitals for two completely different reasons, but with similar requirements regarding the applied motion tracking system. Doctors and nurses could be tracked in case of an emergency. Thus only calling the nearest person will reduce the response time. However in museums motion tracking is used for example in a audio tour guide application that provides the visitor with interesting informations about the piece of art they are stand near by. Obviously accuracy is in this applications for once not the most important criterion. It would be sufficient to know in which corner of the room a person or object is. The requirements for tracking people are rather a long battery life, price in high quantities, weight and size, than high performance tracking. We propose an acoustic or radio frequency based method, because they are the lightest, cheapest and most power saving options available. Avatar animation Avatar animation is based on motion capturing, where the motion of a person or an animal is recorded and then brought into a computer game or an animation film. This task is highly complex due to the number of joints a person has. About fifty targets are needed for fully capturing the motion. One could even record the mimic of a person. Therefore the main requirements for the motion tracking system we will choose is the possibility to track multiple targets at a high frequency and a high accuracy, both for smooth and flickering free motion in your virtual world of choice, e.g., a soccer player in a computer game or the characters in Figure 5.2: Motion capture of a person [15] an animation film. The most powerful tool for tracking multiple targets is undeniably a vision based approach. The most difficult part of this method is the digital image processing software, that extracts the position from the picture. For a full 3D motion capture an expensive multi camera set-up is needed. A cheaper approach would be a magnetic tracker, because it just needs one emitter due to the fact that magnetic waves can penetrate the human body and will not be blocked like light. Unfortunately the accuracy with magnetic sensors is not as high as with cameras.

Bibliography [1] F. Souvestre, M. Hafez, S. Regnier: A Novel Laser-based Tracking Approach for Wide Field of View for Robotics Applications. France, 2009. [2] G. Welch, E. Foxlin: Motion Tracking: No Silver Bullet, but a Respactable Arsenal. IEEE Computer Graphics and Applications, November/December, 2002. [3] E. Foxlin: Chapter 7. Motion Tracking Requirements and Technologies. 2002. [4] S. You, U. Neumann, R. Azuma: Hybrid Inertial and Vision Tracking for Augmented Reality Registration. 1999. [5] S. You, U. Neumann: Fusion of Vision and Gyro Tracking for Robust Augmented Reality Registration. IEEE, Proceedings of the Virtual Reality Conference, 2001. [6] G. Welch, G. Bishop, L. Vicci, S. Brumback, K. Keller: High- Performance Wide-Area Optical Tracking: The HiBall Tracking System. Presence: Teleoperators and Virtual Environments, Volume 10, Number 1, February, 2001. [7] B.D. Allen, G. Bishop, G. Welch: Tracking: Beyond 15 Minutes of Thought: SIGGRAPH 2001 Course 11. Course Notes, Ann. Conf. Computer Graphics and Interactive Techniques (Siggraph 2001), ACM Press, New York, 2001. [8] D. Murray, A. Basu: Motion Tracking with an Active Camera. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 16, No. 5, May, 1994. [9] S. Holm, C. Nilsen: Robust ultrasonic indoor positioning using transmitter arrays. Department of Informatics, University of Oslo, P. O. Box 1080, NO- 0316 Oslo, Norway. [10] N. Priyntha, A. Chakraborty, H. Balakrishnan: The Cricket Localion- Support System. MIT Laboratory for Computer Science, Cambridge, MA 02139. [11] N. Uchitomi, A. Inada, M. Wada, K. Mutsuura, H. Okada: Accurate Indoor Position Estimation by Swift-Communication Range Recognition (S-CRR) Method in Passive RFID systems. 2010 International Conference on Indoor Positioning and Indoor Navigation (IPIN), 15-17 Semptember 2010, ZÃ 1 4rich, Switzerland. [12] E. Foxlin, M. Harrington, G. Pfeifer: Constellation T M : A Wide-Range Wireless Motion-Tracking System for Augmented Reality and Virtual Set Applications. Orlando, Florida, July 19-24, 1998. 29

Bibliography 30 [13] R. Zhu, Z. Zhou: A Real-Time Articulated Human Motion Tracking Using Tri-Axis Inertial/Magnetic Sensors Package. Department of Precision Instruments and Mechanology, Tsinghua University, Beijing 100084, China, 2003. [14] R. Herrmann, J. Sachs, F. Bonitz: On benefits and challenges of person localization using ultra-wideband sensors. 2010 International Conference on Indoor Positioning and Indoor Navigation (IPIN), 15-17 Semptember 2010, ZÃ 1 4rich, Switzerland. [15] http://cse.csusb.edu/tong/courses/cs520/images/intro/mocap2.png, accessed 01.06.2011 [16] http://thearmyadapts.blogspot.com/2011 02 01 archive.html, accessed 01.06.2011

Autonomous Systems Lab Prof. Roland Siegwart Bachelor-Thesis Control of a Micro UAV Using an External Laser Tracker Spring Term 2011 Supervised by: Cédric Pradalier Samir Bouabdallah Authors: Dominic Jud Andreas Michel

Contents Abstract Symbols iii v 1 Introduction 1 1.1 General product idea........................... 1 1.2 Goal of the project............................ 2 2 Topology of the system 3 2.1 Initial conditions and requirements................... 3 2.2 Morphological Box............................ 3 2.3 Set-up overview.............................. 6 3 Subsystems 9 3.1 Leica total station............................ 10 3.1.1 Description TS15......................... 10 3.1.2 ROS Node - Leica Interface................... 11 3.2 Mikrokopter................................ 11 3.2.1 Mikrokopter interface...................... 11 3.2.2 Prism fixation........................... 12 3.2.3 Kalman Filter........................... 13 3.2.4 Trajectory following controller................. 14 3.3 AscTec hexacopter............................ 15 3.3.1 Software.............................. 15 3.3.2 Test flight setup......................... 16 3.3.3 Alignment of the coordinate system.............. 17 4 Tests and evaluation 19 4.1 Motion tracking of the leica system................... 19 4.1.1 Performance........................... 19 4.1.2 Interface.............................. 21 4.1.3 Prism position for optimized tracking............. 22 4.1.4 Influences on Kalman filter................... 25 4.2 Mikrokopter vs. AscTec......................... 26 4.3 Flight performance............................ 26 4.3.1 Outdoor test........................... 27 4.3.2 Indoor test............................ 29 4.3.3 Evaluation and Comparison of outdoor and indoor...... 31 5 Conclusion 35 5.1 Achievements............................... 35 5.2 Outlook.................................. 35 i

Bibliography 37 ii

Abstract This study provides an evaluation of the Leica TS15 total station for controlling a micro UAV. First of all, all the needed software nodes and subsystems to fly autonomously with a stationary laser tracking system are described. Secondly, the evaluation is conducted by comparing the Leica laser tracker with the Vicon motion capture system. The resulting flight characteristics of the UAV is stable, but with slow oscillations around the set-point. These oscillations are caused by the delay of the Leica tracker, being a measurement device for static points and not an actual motion tracking system. iii

iv

Symbols Symbols r = x y z position v, a velocity and acceleration φ, θ, ψ roll, pitch and yaw angle φ ω = θ ψ accelerometer and gyroscope values q b rotation as quaternion bias Acronyms and Abbreviations ETH UAV ROS IMU DOF MEMS Eidgenössische Technische Hochschule Unmanned Aerial Vehicle Robot Operating System Inertial Measurement Unit Degree of Freedom Microelectromechanical System v

vi

Chapter 1 Introduction This report deals with the performance evaluation of the total station 1, Leica Viva TS15 [1], used for controlling a micro UAV. Such a laser measurement device is made for static measurements with a high accuracy of buildings or bridges. Additionally to its static use, the total station we tested has an integrated tracking ability working with specific reflectors. This motion tracking system could have many advantages like a high accuracy, mobility or the long range. We will analyse whether this advantages are realisable in our application or not. 1.1 General product idea A laser-based measurement device is probably the most accurate way to capture an object. But not all objects are easy to scan. For example a wing of an aircraft is not easy to reach with such a laser system. The goal of the project is to integrate a subsystem to the given laser measurement device, which is movable and preferably unbounded. A highly dynamic UAV will be our choice. The laser measurement device tracks the motion of the UAV and returns its position. On-board of this micro UAV, a 1D range sensor will be mounted for scanning objects. With this system, you are able to take the measurements of a nearly non-reachable object such as ships or tall buildings. The ambitious plan consists of the following steps. First of all, one has to integrate the Leica total station and close the control loop. The second step is to implement a function to follow a trajectory or fly through waypoints. Finally the data from the arbitrary device mounted on the UAV has to be evaluated. Thus, the main task could be split into two parts: 1. Closing the control loop: Therefore, establishing the whole communication and stabilize the micro UAV. 2. Following a given flight trajectory over an object, scanning it and returning a 3D-model. We focused our work on the communication, control and flying through waypoints. Any device mounted on the UAV could be the content of future work. 1 term for laser measurement device 1

Chapter 1. Introduction 2 1.2 Goal of the project As the title indicates, the goal of our project is the control of a micro UAV using an external laser tracker. Therefore we evaluate the dynamics of the resulting flight characteristics. We will put the focus on the performance of the total station for controlling an UAV and the influences it has on the control loop. Optimizing the implemented control and filtering is not an element of this project.

Chapter 2 Topology of the system 2.1 Initial conditions and requirements Initial conditions Leica TS15 is providing us with position information, but without any indication of the orientation. Thus only three degrees of freedom are tracked. In this project, the UAVs provided by Mikrokopter [4] and Ascending Technologies [6] are used. The UAV from Mikrokopter needs to be assembled first and is then ready to fly with the remote control. The Ascending Technologies UAV is already built and equipped with a position controller. Requirements The whole system itself should be as self-contained as possible and independent from its environment, because the usage will be mostly outdoors. The UAV has to be able to carry the weight of a prism and an additional arbitrary device, e.g., a laser range sensor or a camera. Furthermore the prism has to be mounted in a way that an occlusion of the laser beam and a resulting crash is avoided. For any application it is beneficial that the hovering oscillations are small and the controller can accurately hold its setpoint. 2.2 Morphological Box To get a first idea of the whole project and its problems, a morphological analysis is conducted. The first question is what one needs to select. The system consists of three subsystem: a total station, a micro UAV and a computer. Thus, we have to define the different possibilities of the devices. Further, the interfaces between them have to be chosen. The whole list of points is shown in table 2.1. Total station Our total station is called Leica Viva TS15. Its technical specifications are elaborated in section 3.1 and the complete datasheet can be found at [1]. Micro UAV Basically, there are two different providers offering a potential micro UAV for our purpose. The manufacturer is not our choice, we only take the platforms into account on which our institute has experience on. The first and mostly used platform is called AscTec [6]. The second is an open source project named Mikrokopter [4]. Thus, there is a remarkable variety of quadro- (see figure 2.1), hexa- and oktocopters. Also coaxial helicopters provided from another company. In general, the 3

Chapter 2. Topology of the system 4 Table 2.1: Morphological Box Function Solution 1 Solution 2 Solution 3 Solution 4 Total station Leica Viva TS15 Micro UAV Quadrocopter Hexacopter Coaxial helicopter Reflector Round prism 360 Mini prism 360 prism Reflective tape Reflector position Top Bottom Side Communication ROS Additional sensing method Inertial Optical Magnetic Ultrasound Processing of laser and inertial data Kalman filter Recursive total least squares Double exponential smoothing Controller On-board Off-board Interface total station/computer USB-mini RS-232 Bluetooth Interface micro UAV/computer Bluetooth XBee Cable Wlan

5 2.2. Morphological Box Figure 2.1: Quadrocopter provided by Mikrokopter, fixation of the reflector on the top needed requirements are: big payload agile, thus more accurate Reflector The total station from Leica Geosystems AG is able to track the listed reflective objects which are also provided by the same firm. We chose the 360 prism (see figure 2.2) because with this choice the orientation of the UAV does not matter. The round prism is just to heavy for our purpose. That leaves the two 360 prisms. The main advantage of the bigger one is its ability to use a certain function called power search which can search and focus on the prism automatically. Additionally it brings more precise measurements but is also heavier. Figure 2.2: Leica prism GRZ4 [2] Reflector position Given that the take-off and landing is the most difficult part in flying, it s crucial that the reflector will not be blocked by any parts of the UAV. Mounting the reflector on top of the UAV (see figure 2.1) will fix this problem best. Mounting on the side will affect the flight performance. For more detail about the fixation of the reflector see sections 3.2.2 and 4.1.3. Communication To collect all received data over the interfaces and manipulate them, we needed a tool to communicate. The best way to keep clear view is to use ROS. Additional sensing method The initial situation using a laser tracker device with a low measuring frequency requires us to attach another motion tracking method improving the position data

Chapter 2. Topology of the system 6 to a higher frequency. The list of motion tracking technologies is endless. An obvious approach is to use the inertial sensor (IMU) already mounted on every UAV. Another proposal could be to utilise an optical system, e.g. Vicon 1 [7]. But such systems are not self-contained 2 at all and therefore useless for our outdoor project. Processing of laser and inertial data There are some linear and nonlinear procedure dealing with this problem. A common method is a Kalman filter to fuse the data received from the total station with the one of the IMU returning linearly optimized states. More information about Kalman filtering is in section 3.2.3. Controller We have two possibilities on the location of the controller, on-board which means on the microcontroller of the UAV or off-board on the computer. The native approach would be to handle it on-board inducing less delay into the whole system. Then, you only send your position signal of the laser device up to the UAV and the whole computation including the Kalman filter is placed there. By using the UAV from Mikrokopter, we have to merge the laser signal with the IMU s output off-board because of the limited performance of the microcontroller. In this case, we also take the controller off-board for exactly the same reason. Workin with the UAV from AscTec, there are no such limitations. Interface total station/computer The decision to go forward with the serial port is based on its reliability and simplicity to program. Interface micro UAV/computer The preferred way to communicate is obiously wireless. Possible choices could be Bluetooth or XBee. The decision to chose Bluetooth is based on the fact that it checks if all the data packages have arrived or not. Furthermore our data packages will not be as big that they could exceed the transmission speed. 2.3 Set-up overview We are now able to state more precisely the setup of our system (see figure 2.3). The leica laser measurement device has an implemented motion tracking function for special prisms also provided by Leica. Therefore, such a prism is mounted on the UAV. The laser device measures the position of the prism at regular intervals. The position is transmitted to the a computer. On the computer, the communication between all subsystems is made by ROS (further information in the next chapter). Because of the low input frequency of the Leica tracker, we have to use another sensing method to improve this rate and make the whole system control feasible. Additionally, the laser tracker returns only information about the position, but nothing about the attitude of the UAV. This problems are solved through inertial sensing with the IMU. Hence, we have to arrange the communication from the computer to the UAV. The controller and filter could be on- or off-board. This is not decided yet. Thus, the control outputs (when off-board) or the position measurements (when off-board) require a communication channel to the UAV. 1 Precise and fast system provided in our institute 2 with no other parts to be mounted in the environment or on the UAV

7 2.3. Set-up overview Figure 2.3: Setup overview

Chapter 2. Topology of the system 8

Chapter 3 Subsystems In this chapter, the hard- and software components are described. We want to underline once more, that the goal of this project is not to optimize the controller etc., but rather to build a good system to evaluate the Leica total station for controlling a micro UAV. So the following mentioned system components are required to complete a working application. The subsystems are: Leica tracker UAV Computer The first thing is to organize the whole communication. An appropriate solution is to use ROS. In ROS, you are able to split the different challenges. Thus, there are a few nodes operating independently and communicate among each other over ROS. Figure 3.1: ROS nodes See figure 3.1 for a graphical description of the ROS nodes. The ROS nodes are: Leica interface (see section 3.1.2) UAV interface (see section 3.2.1) Filter (see section 3.2.3) Controller (see section 3.2.4) 9

Chapter 3. Subsystems 10 3.1 Leica total station 3.1.1 Description TS15 The Leica totalstation TS-15 is typically used to stake-out an object on a construction site or to take accurate measurements of a tunnel or bridge. However, it can not only measure static points, it has also the unique ability to lock on to prisms and follow them, providing continuous position measurements. Therefore using the TS-15 as a motion tracker suggests itself. Additionally to the specifications mentioned above, the TS-15 has a function called PowerSearch, which can find prisms everywhere up to 300 meters away and is based on digital image processing and a rotating laser fan. This makes working with the TS-15 very comfortable, easy and productive. The important specifications regarding this project are mentioned in table 3.1. Table 3.1: Important specifications of the TS-15 for tracking a micro UAV [1] Distance Measurement with Prism General Range 360 prism 2000 m 360 mini prism 1000 m Accuracy Continuous (< 0.15 s) 3 mm + 1.5 ppm Weight and Dimensions Weight of Total Station 4.9 5.5 kg Height / Width / Length 345 mm/226 mm/203 mm Motorization Rotation Speed 45 /s Range in Lock Mode 360 Automatic prism 600 m 360 Target Aiming mini prism 300 m Maximum Speed in Lock Mode (ATR) Tangential 5 m/s at 20 m 25 m/s at 100 m Radial 4 m/s Power Search (PS) Range 360 prism 300 m The TS-15 has different connection possibilities to offer, from Bluetooth to serial connection (RS-232) and USB-mini. The currently measured point is published in the following format: point ID, north, east, elevation, date, time, <CR/LF> Field point ID north east elevation date time <CR/LF> Table 3.2: Description of the data fields Description identifies every point with an incrementing number north coordinate east coordinate height coordinate date of measurement time of measurement Carriage Return/Line Feed

11 3.2. Mikrokopter Examples: 1234,12.3456,78.9012,34.5678,09/10/2010,16:34:12.2 1235,11.3836,80.8825,32.6109,09/10/2010,16:35:09.4 1236,11.2821,79.4112,32.5832,09/10/2010,16:35:95.1 3.1.2 ROS Node - Leica Interface The ROS node which is responsible for the data management over the serial port is called Leica Node and it is implemented in Python. The Leica Node listens to the serial port, reads out the position measuremnets from the string and writes the position into the geometry message PoseWithCovarianceStamped from ROS. Because the coordinate system of the Leica totalstation is a North-East-Up, there is also a coordinate system transformation needed into the North-East-Down coordinate system commonly-used by any kind of aerial vehicle. The message contains the position in x, y and z direction. In addition there is also a time stamp from ROS. Unfortunately it is not possible to include the time from the totalstation when the measurement was taken due to a synchronisation problem as described in section 4.1. The reason why it also has the orientation and covariance, which are set two zero, is simply because the input of the Kalman filter demands it. PoseWithCovarianceStamped Message Definition [3] Header header uint32 seq time stamp string frame ID PoseWithCovariance pose geometry msgs/pose pose geometry msgs/point position float64 x float64 y float64 z geometry msgs/quaternion orientation float64 x float64 y float64 z float64 w float64[36] covariance 3.2 Mikrokopter 3.2.1 Mikrokopter interface Our interface is based on the QMK-Tools Version 1.3.1 from the Mikrokopter repository [5]. It is compatible with the Flight Control code version 0.82. The program has a server application called QMK-Communicator which is responsible for the communication with the Mikrokopter. Additionally to this server a few client applications are available, i.e., QMK-Maps, QMK-Settings and QMK-Scope. The last one is of particular significance for our purpose, because it receives all the debug data and plots it simultaneously. That means the communication from the Mikrokopter

Chapter 3. Subsystems 12 to the computer is already working and all that has to be done, is publishing the received values into ROS. The second a lot harder part is the communication from the PC to the Mikrokopter to send out the stick commands calculated in the control loop. This send-out function is called whenever a new stick command is calculated. Qt with its signals and slots concept is a good and clear way to achieve this. 3.2.2 Prism fixation The 360 prism has a fairly big housing. Since a UAV is a light weight construction it is important that you can save weight where ever it is possible. Therefore the prism is stripped from its original housing and put into an optimized support construction. The design criteria are weight, size, mounting points and field of view of the prism. With respect to these guidelines, our housing consists of a quadratic bottom plate with the same holes as the flight control board and a triangular cover plate which are both produced by a 3D printer as shown in figure 3.3. This two parts are hold together by a screw in the center. Furthermore a notch on both parts keeps the prism in position. Additionally there are two sets of four aluminium poles attached to the screws of the flight control. In between them is the compass. The additional height of the compass reduces the influences of any electric current on the measured magnetic field. Thus aluminium is not only a good choice because of the low weight but also because of its non-ferromagnetic characteristics. A further discussion of this topic is found in section 4.1.3. (a) (b) Figure 3.2: Comparison between the original housing [2] (a) and our light weight alternative (b) (a) (b) Figure 3.3: CAD model of the bottom plate (a) and the top plate upside down (b)

13 3.2. Mikrokopter 3.2.3 Kalman Filter To control an unstable system like a quadrocopter, one is required to have a reasonably high tracking frequency. Thus, only the frequency given from the laser tracker system, is not good enough to stabilize the system. You also have information just about the position, but nothing about the attitude. The simplest way to get more information about the motion of the system and not to include another motion tracking system, which is not self-contained, is to use the inertial sensors (IMU). The IMU includes an accelerometer and a gyroscope. In theory, you can integrate the values of the accelerometer and gyroscope to receive the position and attitude of the helicopter. But in reality, the IMU suffers typically from noise error and especially the gyro, from bias. The best solution to handle this noise is to use a Kalman Filter. You take the IMU data as input to estimate the position and orientation and fuse it with the measurement from the laser tracker. Principle In this section, the needed mathematics is shortly introduced. For further informations read paper [9]. The two main parts of the Kalman Filter are: the prediction and the update In the prediction stage, an estimate of the state variables is determined. In our case, the state variables are the position g r, the velocity g v (both in world frame) and the attitude as quaternion q att. We additionally estimate the bias of the acceleration sensors b a and gyroscopes b ω to improve the filter. Thus, the state vector x is defined. g r g v x = q att ba bω The sensor values from the IMU are used as input u into the Kalman Filter. As mentioned, the IMU provides us with the acceleration in body frame f a and the gyroscope values ω, which describe the rotation between body and world frame. For a simpler handling with this rotation, it is converted into a quaternion q ω. ( u = f a q ω The position measurement of the laser tracker g r is then used to correct the estimate in the update stage. y = g r Next step is to arrange the system equations. The rotation matrix R q is given by q att. Ω is given by q ω. Furthermore the derivatives of the biases are set to zero, because they change very slowly in time. x = ) g v R q ( f a b a ) g 1 2 Ωq att 0 0 Having all the necessary mathematics, the Kalman Filter can be implemented. (3.1)

Chapter 3. Subsystems 14 3.2.4 Trajectory following controller In this section, we describe the control system responsible for the trajectory following. The real flight controller stabilising the UAV is already given on both platforms. There are different opinions to arrange a control system. The simplest way is to work with a PID approach. Another idea would be a controller with observer, but then a model of the system is required. Hence, we introduce here a simple idea of PID control approach. Basic features At the end, our controller should be able to follow a given trajectory. A trajectory following is nothing more then passing given points at a given time. To elaborate the principle of the controller, we take the example to control the UAV from a starting point A to an end point B. The goal is to avoid overshoot as shown in figure 3.4 on the top. The simplest solution would be to take only a P part, thus just to multiply the position error vector with the P value. But as you could expect and is clarified in figure 3.4 on the bottom, the UAV would not reduce the velocity, thus compensate its declination until it overshoots point B. Therefore, you have to account for the velocity. Figure 3.4: with and without velocity control A first draft you can see in figure 3.5. The output of the Kalman filter consists of the actual position g r, the velocity g v and the attitude q att of the UAV. The block FLIGHT POINTS publishes the reference position and is therefore marked with a star, thus g r. The first controller box CTRL 1 consist of a P and an I control term. Our approach should be flexible for every environment, thus for outdoor- and indoor-use. An outdoor application implies more critical external influences such as wind. If we assume a constant wind affecting our UAV, only a P would leave an offset. Hence, an I part is placed to avoid a steady-state error. Summarised, CTRL 1 takes the position error ( g r g r), affects it by the PI terms and results an advisable velocity vector g v. g v enters the second control loop CTRL 2 consisting only of a P term. The actual velocity g v and attitude as quaternion q att, one gets from the Kalman Filter. However, the velocity error ( g v g v) shows the direction in which the UAV has to be piloted. To derive how much the UAV has to roll and/or pitch to fly in the calculated direction, you have to identify the yaw angle between the world and the UAV frame. This is given by the quaternion q att. Therefore, the CTRL 2 outputs the control signals roll R, pitch P, thrust T and if necessary yaw Y.

15 3.3. AscTec hexacopter KALMAN FILTER g r g v, q FLIGHT POINTS g r CTRL 1 PI SAT g v CTRL 2 P SAT R, P, T, Y 3.3 AscTec hexacopter Figure 3.5: Controller This hexacopter is one of the UAV s provided by Ascending Technologies. At the ASL ETH Zurich, a group of Ph.D. students (M. Achtelik, S. Weiss et al.) already did a lot of work on this particular hexacopter 1, so that the control loop already exists and just needs an input providing the system with an absolute position. [8] Figure 3.6: Hexacopter from Ascending Technologies with considerable changes 3.3.1 Software The software on board includes a position controller and a filter for fusing the low frequency position signal with the integrated acceleration values and angular rates. The x and y position are controlled by a PD controller, whereas the altitude in z direction is controlled by a PID controller. The I term in x and y is missing because it could induce additional instabilities. The filter has static gains due to the limited 1 The project is called sfly done at ASL ETHZ

Chapter 3. Subsystems 16 processing power of the micro controller. The communication is established over WLAN, because the sfly team normally transmit the streaming of the camera on board which needs a high bandwidth. [8] The software is not dealt with any further and is simply taken as it is, because our interest lies in the behaviour and performance analysis of an UAV controlled with an external laser tracker and not in the controller or filter itself. 3.3.2 Test flight setup Our application is in general made for outdoor use with the setup as described in figure 3.7. As always, the position is measured by Leica TS15, then received on the computer and published into ROS by the Leica Node. To be able to control the UAV, we need information about the attitude, in fact only the yaw angle is needed. Outdoor, it is convenient to get the yaw angle from a magnetometer, i.e., a compass, because it is an absolute value without drift and accurate enough for the slowly changing orientation. Indoor, earth s magnetic field is corrupted and you have to find another way to control the orientation. One way would be to take the estimated attitude out of the filter, but that is not a good idea because the values are highly inaccurate. The second possibility at the ASL is using the Flight Room with the Vicon tracking system. Therefore in the node Fusion of Position and Attitude the leica position is fused with the attitude measured by Vicon as drawn in figure 3.8. The influences on the system by taking the much more accurate orientation from the Vicon instead of the compass is negligible from the system dynamics point of view due to the non-sensitive dynamic behaviour of the yaw orientation. The resulting pose is then sent per WLAN to the Hexacopter. All the important subfunctions provided by the hexacopter are pictured in the top right corner of the figure 3.7. This interacting subfunctions are the Controller, the WLAN device, the IMU and the Filter. Environment: Leica TS15 position pose Leica Node Fusion of Position and Attitude Hexacopter ROS Nodes: yaw angle from compass Figure 3.7: Setup outdoor Indoor use is not a native application area of our system due to the fact that UAV s only make sense in a big and open environment and also the application area of the Leica total station is highly limited indoors.

17 3.3. AscTec hexacopter Environment: Leica TS15 Vicon position attitude Leica Node Fusion of Position and Attitude pose Hexacopter ROS Nodes: Figure 3.8: Setup indoor 3.3.3 Alignment of the coordinate system The initialization of the two coordinate systems, i.e. the body frame of the UAV and the world frame of the total station, has to be as accurate as possible. A small deviation of the two will lead to a circular movement of the UAV in the x and y plane with rising amplitude and eventually cause a crash. By simply setting up a few markers in the Vicon, it is not especially difficult to align the laser s coordinate system with the one of Vicon. However the set-up outdoors is a completely different story, because the reference for the coordinate systems is now the magnetic north pole. Therefore we moved the hexacopter 20 meters away from the total station and aligned it in a way that its positive x axis points to north and the negative x axis points to the total station. We then lock on to the prism with the total station and set the pointing direction of the laser beam as the positive x direction of the total station. The resulting two coordinate systems are not perfectly in line due to the inaccurate compass value and the tedious search for the north pole, but it is accurate enough if you do it carefully. Some kind of initialization that automatically corrects the rotation between the two coordinate systems would be preferred and could be the task of a future work.

Chapter 3. Subsystems 18

Chapter 4 Tests and evaluation 4.1 Motion tracking of the leica system 4.1.1 Performance To evaluate the overall performance of the Leica total station, a second tracking system, in our case the Vicon motion tracking system, is needed as a comparison. The total station has of course a very high accuracy in measuring static points, since it is built for exactly this purpose. Therefore the main interest of this evaluation lies in the behaviour of tracking dynamic objects at the highest frequency possible. The first test is a simple comparison between Vicon and the Leica total station. Figure 4.1: Comparison of the measured z-position of the Vicon (blue) and Leica (red) tracking system The Vicon tracking system measures the pose with a subcentimeter accuracy at roughly 100 Hz and a delay less than 10 Milliseconds. By comparing the two plots in figure 4.1 one can estimate the relative delay as approximately 200 Milliseconds and it also seems to have some jitter in it. This jitter meaning an inconstant delay between Leica and Vicon has also its bad influence on further processing. As in- 19

Chapter 4. Tests and evaluation 20 troduced in section 3.1, the Leica total station transmits also the timestamp. This timestamp is unfortunately not taken at the time of the measurement, but instead at the time of the transmission. Further details of the influences on the filter are mentioned in section 4.1.4. Despite this considerably big delay, the accuracy is nearly as good for dynamic objects as it is for static points. Another issue noticeable on the plot 4.1 and especially on plot 4.2 is the big interrupt all few seconds. This phenomenon is caused by calibration interrupts of the Leica laser device. In figure 4.2 the coordinate systems of Vicon and Leica are not perfectly aligned resulting in a wrong impression of the Leica delay, but it still shows the bad effect of these interrupts. Figure 4.2: Comparison of the measured y-position of the Vicon (blue) and Leica (red) tracking system In the second test we will take a look at a histogram of the position output of the Leica total station. This histogram contains about 3336 points at a frequency of 7.5 Hz and shows the time interval between two measurements. The histogram in figure 4.3 shows a big cluster of measurements at 140 milliseconds. This test was conducted with the highest possible frequency the total station allowed, i.e., 10 Hz. But there is a second clearly smaller cluster of points at roughly 700 milliseconds with a significant amount of measurements. This can be explained by the fact that the total station takes a long break every few seconds to recalibrate. This jitter will be a big challenge for any filter and controller. Furthermore the frequency will not be at the promised 10 Hz, instead it will more likely be around 7.5 Hz. The maximal possible rotation speed is 45 /s according to their data sheet [1]. This then results in a maximum possible speed for a translation in 10 meters distance to the total station of around 7.8 meters per second when the movement is orthogonal to one of the two rotation axis of the laser tracker. This speeds are the physical maxima. The fastest speed the Leica s control system can handle will obviously be much smaller. The situation is not as bad as it sounds, because by placing the total station twice as far away the maximum speed will double too. Furthermore we never had any problems with the tracker losing lock during the flight. The

21 4.1. Motion tracking of the leica system Figure 4.3: Histogram of the position measurements provided by the Leica totalstation, 3336 points at a freqeuncy of 7.6 Hz only cases where it happened were due to occlusion or hard and abrupt manual landings. Overall the robustness of the Leica total station for tracking dynamic objects is quite good and should not be an issue as long as the controller and the flight manoeuvres are not too aggressive and far enough away from the laser station. Looking at the three figures above, it is clear that the Leica total station is meant to measure single points with a high accuracy at a low frequency and is not build for high speed motion tracking. Nevertheless the right countermeasures will make the TS15 a viable motion tracker. These are the specifications of the Leica total station for dynamic tracking shortly summarized. For further information take a look at the datasheet [1]: Frequency : 7.5 Hz with significant jitter Delay : 200 ms Accuracy (continuous) : 3 mm + 1.5 ppm 4.1.2 Interface The Leica total station has a few different interfaces to offer [1]. Our choice is the serial link, because it is the most simple, reliable and fastest connection from

Chapter 4. Tests and evaluation 22 the repertory. Furthermore Bluetooth is not a good choice, when there is another Bluetooth connection from the computer to the helicopter active. The total station then just publishes the position and time information in a string on to the serial port. Unfortunately the time the total station provides is useless, because there is no possibility to synchronize the clock with the computer. A very exact clock is just not needed for the purposes of a surveyor. Otherwise it could have been used to measure the delay of every single measurement and therefore reduce the effects of the delay in the control loop. That is why all we have is an educated guess of the delay. 4.1.3 Prism position for optimized tracking There are several different possibilities to mount the prism on the helicopter. The crucial criterion for a wise position is the visibility of the prism from different angles. In addition the resulting weight distribution and restriction of flight is analysed too. Therefore four different approaches, which are top, bottom, side and a multi prism set-up, are discussed in this section. For every evaluation the position of the total station is assumed to be one and a half meters off the ground (h T S ). Figure 4.4: Different positions for the prism: top, side and bottom This evaluation takes only the 360 prisms in account, because the bigger round prism is just too heavy for any micro UAV application. Reflector tape is not a good choice either, due to the fact that every other reflective object will disturb the tracker. Furthermore any other prism that does not cover the whole 360 will restrict the flight pose. An approximation of the UAV s shape is needed for a mathematical analysis of the different mounting points. The most obvious choice is a circular plate with the radius r UAV being the distance between the center and the outermost fix point of the UAV, which in most cases would be the motors. A short test in the laboratory showed that the laser beam is not occluded by a fast spinning propeller, therefore they can be disregarded. Most of the UAVs are light weight constructions and thus do not fill the whole space our mathematical approximation will cover. However it is not a good idea to track the prism with the laser beam going between the motor supports, because a small change of the yaw angle could occlude the beam resulting in a crash. And a system that could eventually fail, will fail.

23 4.1. Motion tracking of the leica system hsafe = 1m ϕ safe flight zone h TS = 1.5 m d min = 10 m Figure 4.5: The safe flight zone is mainly dependant on the maximum angle Φ and the safety distance d min Used variables: h T S : Height of total station over the ground h safe : Safety margin for controller overshoot or external influences z prism : Vertical distance between the prism and the center of the approximation model Φ : Maximum angle of the laser beam d : Horizontal distance between the total station and the UAV d min : Minimum horizontal distance between the total station and the UAV, when no other constraints: d min = 10 m r UAV : Radius of the approximation model, i.e., circular plate Top Advantages: + prism is on the center of gravity no counter weight is needed the measured position is just the actual position of the UAV plus a static height offset + not restricting the heading

Chapter 4. Tests and evaluation 24 + visibility of the prism is guaranteed while taking off and landing + visibility from 360 degrees + enough space for any desired support construction for the prism + hard landings will not damage the prism Disadvantages: - maximum altitude is limited due to occlusion The maximum angle Φ is: Φ = arctan z prism r UAV (4.1) The maximum possible altitude h max is given by the following equation: h max (d) = h T S + z prism d r UAV h safe (4.2) Example: h T S = 1.5 m, z prism = 0.2 m, d = 20 m, r UAV = 0.3 m, h safe = 1 m h max = 13.8 m Bottom Advantages: + prism is on the center of gravity no counter weight is needed the measured position is just the actual position of the UAV minus a static height offset + not restricting the heading + visibility from 360 degrees + maximum altitude is only limited by the angle between the laser beam and prism when the prism can not reflect the light any more Disadvantages: - prism can be blocked by grass, the motor support etc. while landing and taking off - prism fixation has to be strong (therefore heavy) in case of hard landings The minimum possible distance of the UAV to the total station d min is given by the following equation: d min = h T S r UAV z prism d min = 10 m for for h T S r UAV z prism > 10 m (4.3) h T S r UAV z prism 10 m (4.4)

25 4.1. Motion tracking of the leica system Example: h T S = 1.5 m, z prism = 0.1 m, r UAV = 0.3 m = 4.5 m and therefore d min = 10 m h T S r UAV z prism Side Advantages: + maximum altitude is only limited by the angle of the laser beam to the prism Disadvantages: - restricting the heading - just visible from a certain direction - prism is not on the center of gravity in case of a heavy prism a counter weight is needed, which doubles the weight the actual position is dependant on the measured position plus the horizontal and vertical offset combined with the yaw angle Multi prism set-up A multi prism set-up would have a lot of advantages like visibility from all directions. Unfortunately the total station TS15 can not handle more than one prism efficiently, because it just locks on to the first one it finds and can then not skip to another one during the flight. Looking at all this advantages and disadvantages we decided to mount the prism on top. The main drawback with this choice is the altitude. However, this is an experimental set-up and there will be hard landings and maybe even crashes. Thus we want to protect the expensive prism as good as we can. To mount the prism underneath the UAV will result in the smallest restriction of the flight zone. Therefore when the whole system will work safely, the prism can easily be switched from top to bottom. This will only implicate minor changes, i.e., a different constant value. 4.1.4 Influences on Kalman filter A correct implementation of the Kalman filter proved to be more difficult than expected. With the mathematical characterisation of the system, it seems to be simple to implement a correct Kalman filter, but the testing of such an implementation brought some problems along. First of all, you don t have much influence on is the correctness of the IMU data and its noise sensitivity. In our case, a further difficulty appeared. The laser device has a unexpected big delay for measuring a position. It comes from the complexity of the device using the principle of an interferometer. This shortcoming has its logical source. Such laser measurement devices are made for static use. You don t have to expect any motion of a building. However, there is a chance to consider such big delays in your Kalman filter, but for this, you must know the exact delay. As shown in section 4.1, the delay is around 200 ms, but unfortunately

Chapter 4. Tests and evaluation 26 inconstant. The problem would be solved, if we got the timestamps of the exact measurement. As already mentioned, it s not made for such an application, so it does not matter taking the timestamps at the time when the data is transmitted. To make it appropriate for our use, little software editing in the Leica total station would be necessary. That is unfortunately beyond our feasibility. The delay issue causes square signals with picks between more or less every sampling time of the Leica data. 4.2 Mikrokopter vs. AscTec Including a joystick into your ROS setup allowed us to test the software and the connection stability between the computer and the Mikrokopter. As mentioned earlier a long range Bluetooth device or a simple serial cable connected to the serial port of the micro controller could be used for the data transmission. The joystick is an easy way to simulate the stick commands the controller puts out. The test routine consists of constantly sending stick commands and receiving the IMU data plus a few other important values, i.e., the battery voltage, at the highest frequency possible. Testing the transmission with a sending frequency of 20 Hz and receiving the IMU data at roughly 10 Hz exposed the weaknesses of this setup. This test was conducted with the serial cable which has a faster and more stable data rate then the bluetooth we need to use in the end. Furthermore the above chosen frequencies are definitely too slow for any flight manoeuvres. The IMU should actually be sampled at around 50 Hz. An inconstant transmission frequency, hangup s and delays up to one second were the results of this test. Obviously such a system behaviour is useless for our purpose. Reasons for this could be buffering either on the side of the computer or Mikrokopter, an inefficient communication program, the limited processing power of the micro controller or the badly written flight control code. The debugging of this problem would have taken us to much time, we therefore decided to replace Mikrokopter by a platform from Ascending Technologies (AscTec). This turned out to be the right decision, regarding the transmission performance and stability of the AscTec UAV. 4.3 Flight performance To evaluate a controller, you have to fix the testing methods. An obvious test procedure is holding a set point and then characterise the resulting behaviour. We also did two tests with a step input showing the sensitivity of the Leica-based control to dynamic effects and delays. Additionally to only move trough two waypoints, a trajectory following (moving through a list of waypoints) makes also sense. All this tests have to be distinguished between indoor and outdoor, which could imply greatly changed conditions. Data filtering First, we shortly show the filtered data of the Leica position data and the IMU data. This process results in the desired higher data frequency. The output of the filter consisting of the position, velocity and the orientation and is then transmitted to the controller. The delay between the Leica signal and the filtered data occurs on account of data transmission and processing. All the data is recorded on the computer, therefore the Leica signal is recorded as soon as received from the total station. Whereas the filtered data has to be sent back from the UAV to be recorded.

27 4.3. Flight performance Figure 4.6: Filtered vs. unfiltered position data in x 4.3.1 Outdoor test Test conditions We tested outdoors in a park. There was little wind mainly in y direction. The distance between the hexacopter and the Leica total station was arround 10m. The coordinate systems were aligned as described in section 3.3.3. Reference tracking In this section, we present the recorded outdoor-flight data. In figure 4.7(a), the reference (in red) and the actual from the Leica laser tracker measured position (in blue) is graphed. As shown in the drawing, the hexacopter is hovering around the reference x coordinate. In figure 4.7(b) and 4.7(c), the same is drawn for y- and z-direction. A step in the x-y-plane is done, after raising the altitude in z direction. The y-coordinate has a remarkable offset to the set point. This can be explained by the wind influence as mentioned in the initial conditions. An I-term should be added to the controller to prevent offsets, however this can also lead to more instability. In the y-graph, you get the feeling that the UAV reacts before the step of the reference signal. The problem comes from incorrect time synchronisation of the hexacopter s CPU and the computer. Therefore, no conclusion about the response time can be done in this test. An offset is also visible in the graph of the z component. Notice that the coordinate system is north-eastdown. Hence, the dead weight is to little compensated. Additionally, there is a I-term for z-direction in the controller, but it is too small.

Chapter 4. Tests and evaluation 28 (a) Reference and Leica signal in x (b) Reference and Leica signal in y (c) Reference and Leica signal in z Figure 4.7: Reference tracking of an outdoor flight

29 4.3. Flight performance 4.3.2 Indoor test Test conditions We tested indoors in a room made for flight test. The room is equipped with the optical tracking system Vicon. Thus, we can additionally prove the correctness of the Leica position data. The distance between the hexacopter and the Leica total station was arround 5m. This is a really short distance for the Leica tracker (see section 4.1.3). The coordinate systems between the hexacopter and the total station were aligned with the help of a few markers. Because the compass fails indoors, we take the orientation of the UAV with the Vicon. Reference tracking The figures 4.8(a), 4.8(b) and 4.8(c) represent the recorded indoor data in x -, y - and z - direction. Two bidirectional steps in y - direction are forced. The interpretation is similar to the outdoor test. An offset in all the coordinates are remarkable. For x and y, this are signs for an asymmetrical weight distribution. For z, it is the same as outdoor, the compensation of the dead weight. The big overshooting in the y-plot indicates a calibration break of the Leica tracker. In this second test we properly synchronised the clock of the UAV and the computer. We now can interpret the reaction time of the UAV to the reference it should follow. As plotted in 4.8(b) in y-direction the reaction time is remarkable. Another interesting observation in this three plots is, that even with this aggressive step in y-direction there is no change in z or x. This leads us to the conclusion that our Leica-controlled UAV is highly stable, robust and follows its reference very fast.

Chapter 4. Tests and evaluation 30 (a) Reference and Leica signal in x (b) Reference and Leica signal in y (c) Reference and Leica signal in z Figure 4.8: Reference tracking of an indoor flight

31 4.3. Flight performance 4.3.3 Evaluation and Comparison of outdoor and indoor To evaluate the Leica total station as a tracking device, we compare it with another tracking system, the Vicon [7]. Therefore, a similar reference tracking test is conducted with the position and orientation completely from Vicon without Leica at a data rate of 51.4 Hz. The delay of Vicon is less than 10 Milliseconds and it works with a subcentimeter accuracy. Figure 4.9 shows this flight in detail. It shows clearly the significant smaller amplitude of the oscillations and thus a smoother attitude of flight. Figure 4.9: x position of the flight with the Vicon data input Standard error analysis By holding the same set-point, an analysis can be done to compare the different conditions. As a characterisation of the offset and the oscillations, the mean offset and the standard deviation is computed (see table 4.1). Some sectors of the previous graphs (figure 4.7(a): Outdoor test from 17.5s to 33.5s, Figure 4.8(a): Indoor test from 60s to 130s and figure 4.9: Test only with Vicon from 23s to 36s ) are chosen to evaluate the reference holding (see table 4.1). Thereby, one should wait until the additional oscillations caused by a step input fade away. The evaluation gives a first impression, but the results are affected by random error which should cancel out if you look at a long enough time period. Mean offset The offset in x and y for indoor flights results from an asymmetrical weight distribution. Hence, there should be no offset difference between the two indoor test runs with the Leica tracker and the Vicon. In x-direction, the Vicon-only offset amounts to about 19% less than with Leica tracker. Concurrently in y, it is about 13% more only with Vicon. We conclude that the assumption has its correctness. In comparison to the outdoor test, it differs highly. This is caused by the wind which mainly blew in negative y-direction, also visible in the table. There should be no influences of wind in z-direction. The big differences in the z offsets is caused by a change in the controller parameters. As we mentioned earlier, the compensation of the dead weight was too little, therefore we increased the accountable parameter after the first test.

Chapter 4. Tests and evaluation 32 Table 4.1: Standard error analysis of an outdoor, an indoor and a Vicon-only flight Direction Mean offset [m] Standard dev. [m] RMS [m] x 0.0501 0.0926 0.1053 Outdoor y -0.1685 0.1406 0.2195 z 0.3799 0.0135 0.3801 x 0.1288 0.0760 0.1496 Indoor y 0.1006 0.0729 0.1243 z 0.2741 0.0093 0.2743 x 0.1045 0.0098 0.1050 Vicon y 0.1160 0.0154 0.1170 z 0.2731 0.0084 0.2732 Standard deviation As expected and easy visible on the plots, the flight taking the pose out of the Vicon system has a significantly smaller standard deviation. By reason of symmetry, the standard deviation in x- and y-axis should equal. The values of the Leica indoor test are nearly the same. The Vicon-only data are bad due to the short evaluation time. In the outdoor case, both the x- and y-standard deviation values exceed the indoor values. This can be traced back to inconstant wind conditions, again mainly in y-direction. The system dynamic in the z-axis is significantly different compared with the other two axis. It is much stabler in its characteristics. Hence, it is less vulnerable to the delay of the Leica total station. This can be recognised by comparing the difference between the two indoor standard deviations in z-direction of the Vicon and Leica flight, which is really small. The Root Mean Square (RMS) values are tabled for completeness. They are derived from the mean values and the standard deviation and thus give no new information about the flight characteristics we have not already mentioned with the two other values. Oscillation comparison In the following figures 4.10(a)(b)(c), two full oscillations are drawn. The axes scales are intentionally kept constant to illustrate the amplitude and period differences. The oscillations using the Leica tracker indoor and outdoor are similar in amplitude and period, which is expected. The reason for these oscillations is as follows. No matter how good the position input into the x and y controller is, the controller still has to control an unstable system and it will therefore oscillate. We can now use the amplitude and period of these oscillations to evaluate the performance of the tracking system. It is needless to say that the controller has to be the same in each test. In the plot, where only the Vicon data was used to fly, the amplitude is very small and also the period is a little smaller. The interpretation of these observations is that the oscillations are mainly caused by the big delay the Leica total station has. Sure, the lower frequency also affects these oscillations, but the influences of the frequency are reduced by the filter.

33 4.3. Flight performance (a) Oscillations in x during the outdoor test (b) Oscillations in x during the indoor test (c) Oscillation in x during the flight with Vicon Figure 4.10: Oscillation in x in the different environment

Chapter 4. Tests and evaluation 34

Chapter 5 Conclusion 5.1 Achievements We successfully integrated a laser tracker from Leica Geosystems AG in a ROS environment and used it to provide accurate position measurements for an autonomously flying UAV. In the end the hexacopter from Ascending Technologies was flying stable but with slow oscillations around its setpoint. In addition, the performance of the Leica TS15 has been evaluated for this specific application. With the results that it actually is a viable motion tracking system but still has some drawbacks when using it with a highly dynamic system like a micro UAV. The mentioned drawbacks are a low frequency of 7.5 Hertz and a inconstant high delay of around 200 Milliseconds. 5.2 Outlook On the side of the UAV and the associated software, the control has to be optimized to reduce the oscillations and lower the offset. An I-term in the control loop would reduce the offset significantly. Additionally a softer position control could reduce the oscillations, but this would just fight the symptoms and not the actual cause, i.e., the delay from Leica. These oscillations, caused by the delay the Leica system adds to the position measurements, have to be accounted for in a sophisticated filter that fuses the low frequency position data with the integrated accelerations and angular rates from the IMU. Stephan Weiss from the Autonomous Systems Lab (ASL) at ETH Zurich is currently working on such an approach. We see great potential in his future work for a smoother position control regardless of the used tracking system. The evaluation of the Leica TS15 for tracking an UAV belongs to the most important parts of our work. Therefore we introduce a few improvement possibilities for this specific application. First of all, this total station is still built to be a measurement device for static points like bridges or buildings. Thus for our purpose adjustments towards a better dynamic tracking system would be preferred. Those improvements could be accomplished by raising the frequency and reducing the delay preferably without lowering the excellent accuracy. Another problem we came across is that the time is stamped at the time of transmission and not the measurement. There is some software development of Leica Geosystems AG needed to remove this imperfection. Furthermore, it is not possible to synchronize the clocks of the computer and the total station. If this two problems were solved, we could derive the delay of every single measurement and therefore implement a more powerful filter. 35

Chapter 5. Conclusion 36

Bibliography [1] Leica Viva TS-15 Datasheet. Leica Geosystems AG, Heerbrugg, Switzerland, 2010, http://www.leica-geosystems.com/de/leica-viva-ts15 86198.htm. [2] http://www.leica-geosystems.com/images/new/product solution/grz4-pic- 500x500.jpg, 01.06.2011 [3] http://www.ros.org/doc/api/geometry msgs/html/msg/posewithcovariance Stamped.html, 01.06.2011 [4] Mikrokopter, http://www.mikrokopter.de. [5] QMK-Tools, http://www.mikrokopter.de/ucwiki/qmk-tools. [6] Ascending Technologies GmbH, http://www.asctec.de. [7] Vicon Motion Systems, http://www.vicon.com/. [8] Markus Achtelik, Michael Achtelik, Stephan Weiss, and Roland Siegwart: Onboard IMU and Monocular Vision Based Control for MAVs in Unknown In- and Outdoor Environments. Accepted for The IEEE International Conference on Robotics and Automation. Shanghai, China, 2011. [9] Stephan Weiss and Roland Siegwart: Real-Time Metric State Estimation for Modular Vision-Inertial Systems. Accepted for The IEEE International Conference on Robotics and Automation. Shanghai, China, 2011. 37