A Low-Cost High Precision Time Measurement Infrastructure for Embedded Mobile Systems Kemal Köker, Kai-Steffen Hielscher, and Reinhard German Department of Computer Science, Computer Networks and Communication Systems, University of Erlangen-Nuremberg, Germany {koeker, ksjh, german}@informatik.uni-erlangen.de 1 Introduction and Overview Simulations are frequently used to test the performance of embedded systems in different operating scenarios and are usually based on a system model that focuses the relevant parts e.g. for the response time. The accuracy of the system model is very important to gain minimum divergences to the real world and therefore it has to be calibrated and validated multiple times. The quality of the calibration and validation process depends on the accuracy of the monitored data of a real system. As example, hardware monitoring requires a specialized device with a built-in clock, which generates timestamps for signals observed at some system interfaces. This solution may provide precise timestamps but can only be applied to systems located in one place like in [1]. The software method generates required time stamps in software, but the accuracy is depended of the systems load [2]. Monitoring distributed mobile systems is a challenge to keep the accuracy and requires synchronisation of all systems clocks or a global master clock for the reference to collect and compare time related data. The possibilities for synchronisation differ by the medium for the transmission and the protocols for it, whereby the focus is primarily on the accuracy for the reception of the time signal according to a master reference clock like in [6]. Wireless dissemination of time signals in TCP/IP networks (e.g. IEEE 802.11a/b/g) makes it hard to keep certain accuracy because of the back off times in case of data collision. A further drawback for the medium access in wireless networks is the CSMA/CA mode which disallows to gain a high accuracy for a long time period. Higher accuracy gained with other methods like in [8] requires additional expensive hardware components.
2 Kemal Köker, Kai-Steffen Hielscher, and Reinhard German We analysed the feasibility of transmitting reliable high precision time signals for indoor measurements and time synchronisation of mobile embedded systems like soccer robots. This work deals especially with the extension of the timing infrastructure based on the Global Positioning System (GPS) for indoor mobile applications. The extension should however keep the benefit of the high accuracy of the GPS system for time synchronisation and should also provide a reliable solution in a cost efficient manner. The work is primarily based on the software monitoring infrastructure for distributed web servers in [5] for precise measurements of one-way delays of TCP packets, to parameterize, validate and verify simulation and state-space based models of the system. Our solution uses this infrastructure in combination with a FM-receiver module and the PPS-API for the time synchronisation of all object systems clocks with a precision in the range of one microsecond. In addition, example measurements for the timing signal generated with our infrastructure are presented to confirm the usability for distributed mobile embedded systems. 2 Architecture of the Application We selected a modified Robocup [3] scenario for the labour setting with mobile soccer playing robots similar to the F-180 Robocup. All mobile robots in the test bed are equipped with embedded systems using Linux and provided with commonly used FM receiver. Furthermore the PPS- API for the operating system is used which is specified in RFC 2783 [7]. The PPS-API kernel patch for the Linux system not only provides a mechanism for detecting external PPS pulses but also improves the timing resolution from one microsecond to one nanosecond using the timestamp counter (TSC) which is available in most modern processor architectures. 2.1 Robocup F-180 Small Size League In a common RoboCup small size league system, an off-field host computer performs remote control of each player according to the results of the strategy software. The architecture of a soccer robot in the F- 180 league usually consists of the mechanical and electronic part. The mechanical part covers the body and the actors. The electronic part consists of a micro controller for the actors, a dedicated FM-module to receive action commands triggered from the host, and the power supply with batteries. Our modification of the system splits the robots
Low-Cost Time Measurement Infrastructure for Mobile Systems 3 Fig. 1. F-180 Soccer robot with embedded system in an upper layer with an embedded system, and the lower layer covering the mechanical and electronic part for the actors and sensors. We provided each player with a PC/104 [4] as an embedded system for the upper part and installed embedded Linux with kernel 2.6.9 as the operating system. The embedded system of the upper layer has capabilities to communicate with the host over WLAN (IEEE 802.11b) and furthermore each robot is equipped with a FM-receiver module. Figure 1 displays our architecture for the soccer robots using the PC/104 embedded system. 3 Time Synchronisation with GPS The precision of the GPS makes it an impressive technique for any imaginable application that requires the determination of positions (positioning) and direction of motion (navigation) anywhere on earth. The technique is based on measurements of code-phase arrival time from the satellites with high accurate reference clocks on board, to gain positions in three dimensions and GPS time. This facility is often used as potential time synchronisation to provide a globally accurate clock to each receiver. Because of reasons of cost and convenience, it is not possible to equip every computer system with a GPS-receiver to keep a high accurate system clock. But instead of, it is possible to equip some number of computers and let them act as primary time servers. 3.1 Existing Infrastructure The infrastructure we used in our application is based on the infrastructure for the web cluster project in combination of NTP with a master GPS receiver and the PPS-API. Main part of the infrastructure is the possibility to generate high resolution timestamps derived from
4 Kemal Köker, Kai-Steffen Hielscher, and Reinhard German the TSC. The usual solution for synchronizing the clocks of a number of systems is by using NTP over a network connection. The accuracy of this method is limited to a few milliseconds on LANs and up to a few tens of milliseconds on WANs and is relative to the Universal Time Coordinated (UTC). Another possibility would be to equip each of the desired system with one cost effective GPS receiver module (e.g. the price for a common GPS timing module including antenna is listed with more than 100 US$) [9]. The disadvantage of the idea providing the embedded systems with GPS receiver modules is the need of so many modules as soccer robots are deployed. A further drawback is the inherent weakness of GPS requires line-of-sight to orbiting satellites, and hence this method is inappropriate for indoor applications (e.g. wireless time dissemination). 3.2 PPS-API The PPS-API (pulse per second) provides a facility to timestamp external events delivered to a system with high resolution. It is intended for connecting external time sources like GPS receivers to a stratum-1 NTP server. The GPS receiver cards used in the web cluster project have a PPS signal output at TTL level that marks the beginning of a second with an uncertainty below 500 ns with respect to UTC. In Linux, the recognition of the PPS pulse is done by instructing the hardware to generate an interrupt in case of a signal transition on the DCD pin of the serial port. The interrupt handling routine in the serial port driver is modified by the patch to timestamp every invocation. We patched the Linux kernel with the PPS-API patch to use this facility. This patch modifies the serial port driver for detecting and time stamping signals delivered to the DCD pin of the serial port. In addition to the PPS recognition, the kernel patch extends the timekeeping resolution of the Linux kernel to one nanosecond by utilizing the TSC, whereby the TSC is incremented with every tick of the internal clock. The timestamps of the PPS pulses can be used in two ways to discipline the kernel clock: either by using the hardpps() kernel consumer or by using a user level NTP daemon. Both of them make use of the kernel model for precision timekeeping as specified in RFC 1589 [10] and estimate the frequency error of the local oscillator by averaging the measured time interval between successive PPS pulses. Since the PPS pulse just marks the beginning of an arbitrary second, but does not contain information on the absolute time (second, minute, hour, day, moth, year), all clocks of the receivers must be set to have offsets less than 500 ms. Usually
Low-Cost Time Measurement Infrastructure for Mobile Systems 5 Fig. 2. Wireless signal dissemination based on GPS this can be achieved using a standard NTP server on a wireless network and using the ntpdate command before starting a synchronisation with PPS or by using the NTP daemon with a configuration file that contains two time sources, the PPS clock driver and the NTP server. 3.3 Wireless Dissemination of Time Signal Obtaining signals based on a GPS receiver infrastructure may enables to use high accurate pulses for time stamping in wired networks, but this infrastructure is inappropriate for wireless systems. The most possibilities for wireless signal circulation lead to non deterministic response time. The WLAN (IEEE 802.11b) e.g. uses CSMA/CA for the medium access to avoid data collision. Each time before the client starts to send, the clearance of the medium has to be checked, which leads to non deterministic time consumption for collision free data reception. This drawback leads to the idea to transmit the timing signal pulse of the mentioned GPS infrastructure regardless of the medium clearance without affecting other clients and keep the mechanism as easy as possible. For this purpose, we connected the TTL output of the GPS timing architecture to the input of an widely spread FM-transmitter module to broadcast the GPS second pulses on the FM-band 433,92MHz. To receive the signal, we connected an appropriate FM receiver module on the same FM band to the serial port of the robots PC/104 system. The costs for both modules e.g. from [11]are listed below 20US$. Figure 2 illustrates our infrastructure for the wireless signal dissemination. 4 Measurements We started a test run transmitting the PPS-signal from the GPS based infrastructure to the robots PC/104 system and simultaneously tracked the signal reception with an analogue oscilloscope. The signal course of the first test run resulted a lot of interference, which would lead the PPS-API facility assume to a wrong offset of the systems clock.
6 Kemal Köker, Kai-Steffen Hielscher, and Reinhard German We started a long term test run to assure, that the interference is not temporarily and we record the internal system clock, the second pulse recognized by the PPS-API and the jitter. 4.1 Results of First Test Run The left side of figure 3 shows the plot of the results of a continuous measurement for three days. The plotted dots are representing the difference of two back-to-back seconds. The y-axis of the figure is scaled linearly and is indicating the duration of the systems second. We can match this with the deviation of the second pulse registered by the PPS-API from the true (exact) second. The x-axis is also scaled linearly and is counting the triggered pulses. Considering the results, we detect a lot of high ratio jitters that disallow using the modified GPS timing architecture in this mode and therefore it is not useful to discipline the kernel clock in a high accurate way. Primary sources for the interferences can be found in different electronic devices with spurious radiations (i.e. power supply of running computers), and also atmospheric disturbances. Another reason for the deviation of the exact time second is the technical design of the used FM receiver module with a fast-acting carrier detect and power-up implementation. This implementation may allow effective duty cycle power saving and a - 107dBm sensitivity but takes about 1 ms for enabling the device and is an disadvantage for our application. According the time measuring infrastructure of the web-cluster, the GPS cards deliver a PPS signal output that marks the beginning of a second with a very high accuracy (below 500ns) with respect to UTC. Therefore the maximum deviation of a time second between the systems clock - using the FM module - and the GPS subsystem should amount not more than 1.5 ms considering the reception delay of the module. A further disadvantage is the sensitivity of the module for captured undefined foreign signals which just lead to enable the FM module and then again shutting down because of missing signals. 4.2 Techniques for Improvement To use however the accurate GPS timing infrastructure and handle the disadvantages of the FM modules, we switched the GPS card to generate a 100Hz PPS signal output. In this operating mode, the output delivers 98 TTL high level signals with a length less than 10ms and a longer 99th high level signal to mark the beginning of a second with by the next pulse. This operating mode of the GPS card also leads the
Low-Cost Time Measurement Infrastructure for Mobile Systems 7 Fig. 3. Scatter plot for different driving mode of the GPS cards FM receiver module to maintain the enabled mode for signal reception and therefore no additional enable time is necessary. But to use the PPS-API facility, the signal input to the system has to be exactly one pulse per second. To utilize the 100Hz PPS signal for the PPS-API, we build a time measuring circuit with a micro controller on the receiver side. The function of the circuit is to monitor the width of a signal pulse delivered by the GPS card output. The circuit enables an incoming signal to be forwarded with the next pulse if and only if a longer high level signal is received. This method assures to forward the correct signal pulse for the beginning of a second and also decreases interference. We ensured our considerations by again tracking with the analogue oscilloscope and monitored fewer disturbances. Ongoing we started a new test run for 3 days and again recorded the internal system clock, the second pulse recognized by the PPS-API and the jitter for both back-to-back seconds. The right part of figure 3 shows the measurements for the test run, whereby the number of pulses for the x-axis is abbreviated to compare the results of the first test run. We can identify a smoother scatter plot with more precise boundaries for the duration of a second. These boundaries should be considered for the system design and monitoring method. 5 Conclusion We analysed the feasibility and performance of an indoor facility for a high precision time synchronisation for mobile devices. The facility is intended to build a simulation model for embedded mobile systems. For this purpose we extended an existing high accurate GPS based infrastructure and introduced commonly used FM modules to transmit
8 Kemal Köker, Kai-Steffen Hielscher, and Reinhard German and receive a pulse per second signal to the embedded systems of mobile soccer robots. The simply usage of the FM modules to transmit and receive signals led to higher interference ratio, also caused from the entire design and enable delay of the FM receiver. To handle the interferences and profit the accuracy of the GPS time synchronizing infrastructure, we switched the GPS output to transmit a 100Hz signal and introduced a circuit to generate a pulse at the end of the 100th signal on the receiver side. Using this method significantly improved the results and allows transmitting high accurate pulses for a second gained from the GPS in an appropriate way for performing measurements on mobile embedded systems. References 1. Kemal Köker, Richard Membarth, Reinhard German, Performance analyses of embedded real-time operating systems using high-precision counters, Proceedings of 3rd International Conference on Autonomous Robots and Agents, New Zealand, pp 485-490, (2006) 2. Ojha, A.K., Techniques in least-intrusive computer system performance monitoring, Proceedings of SoutheastCon 2001, Clemson, SC, USA, pp 150-154, (2001) 3. ROBOCUP SMALL SIZE LEAGUE, http://www.robocup.org/02.html, visited 14/01/2007 4. PC/104 EMBEDDED CONSORTIUM, PC/104-Specification, http://www.pc104.org, visited 12/06/2004 5. Hielscher, Kai-Steffen Jens ; German, Reinhard, A Low-Cost Infrastructure for High Precision High Volume Performance Measurements of Web Clusters, Proc. of 13th Conf. on Computer Performance Evaluations, Modelling Techniques and Tools, Urbana, IL, USA, (2003) 6. D. Mills. Internet time synchronization: the Network Time Protocol. IEEE Trans. Communications, 39(10):1482-1493, October 1991 7. J. Mogul, D. Mills, J. Brittenson, J. Stone, and U.Windl. Pulse-persecond API for Unix-like operating systems, Version 1. Request for Comments RFC-2783, Internet Engineering Task Force, March 2000 8. H. Dai, R. Han, TSync: a lightweight bidirectional time synchronization service for wireless sensor networks, ACM SIGMOBILE Mobile Computing and Communications Review, Vol. 8, Is.1, pp. 125-139, (2004) 9. Synergy Systems, GPS Timing Modules, http://www.synergygps.com/content/view/20/34, visited 15/12/2006 10. D. Mills. A Kernel Model for Precision Timekeeping. Request for Comments RFC-1589, Internet Engineering Task Force, March 1994 11. Radiometrix radio modules, http://www.radiometrix.com, visited 12/12/2005