Real Time Target Tracking with Binary Sensor Networks and Parallel Computing

Transcription

1 Real Tie Target Tracking with Binary Sensor Networks and Parallel Coputing Hong Lin, John Rushing, Sara J. Graves, Steve Tanner, and Evans Criswell Abstract A parallel real tie data fusion and target tracking algorith for very large binary sensor networks is presented. A binary sensor can give an on or off signal to indicate the presence or absence of targets within its range, but it cannot tell how any targets are present, where the targets are, how fast they are oving, or which direction they are heading. In order to detect and track targets using these sensors, it is necessary to fuse inforation fro ore than one sensor. A parallel data fusion process based on siulated annealing is used to identify and locate targets. Processing is perfored on a coodity Linux cluster with counication between nodes facilitated by the Message Passing Interface (MPI). The fusion and tracking algorith is tested with a wide variety of sensor network paraeters using target track data fro a theater level air cobat siulation. It is deonstrated that very accurate target detection and localization are possible even though the binary sensors theselves provide little inforation and have high error rates. Real tie tracking is perfored on a network with 2.5 illion sensors on a coodity cluster with only 50 processors. Index Ters sensor network, binary sensor, data fusion, siulated annealing, target tracking, real tie, parallel coputing, cluster I. INTRODUCTION ENSOR networks are systes with devices deployed over Sa geospatial area to onitor objects of interest in both ilitary and civilian applications, such as integrated air defense and intrusion detection. Large networks of priitive sensors are becoing ore attractive due to the low cost deployent, sall size, low energy consuption and siple operation and data counication of the sensors [-2]. These networks can continuously onitor a uch larger geographic area than would be possible using ore expensive sensors. Sensors can be classified according to their capabilities. This research explores ethods for tracking targets using very priitive binary sensors. A binary sensor can give an on or off signal to indicate the presence or absence of targets within its range. Binary sensors cannot tell how any targets are present, where the targets are, how fast they are oving, or which direction they are heading. Binary sensors ay not be fully reliable, so the sensor odel includes both a false positive rate and a false negative rate. Here, a false positive rate is the This work was supported by the Dept. of Defense under GS-23F-0062P. All authors are with Inforation Technology and Systes Center, University of Alabaa in Huntsville, Huntsville, AL probability that the sensor issues an on signal when there is no target around, and a false negative rate is the probability that a sensor issues an off signal when there are targets within its field of view. Target tracking is a process of state estiation of ore than one object over a region of interest during a period of tie. Since binary sensors can only provide an on or off signal, the target positions can only be deterined through a data fusion process. Siple filtering and search based data fusion [3-4] approaches are coon. The input to a data fusion algorith is a set of noisy sensor observations, and the output is a set of target positions. In a siple filtering approach, the network is divided into sall grids and the relative likelihood that a target is in a given grid is coputed based on the readings of its surrounding sensors. A siple center-surround filter can be used to detect targets. This approach can be unreliable because the quality of results is sensitive to the grid resolution and the threshold for filtering. Search based data fusion uses an optiization process to find a set of target positions that is consistent with the sensor network observations. In general, the process achieves better quality of results but with high coputation cost, especially if the nuber of sensors is in the order of illions. Increasing coputing power and iproving data fusion efficiency are crucial in designing a high perforance real tie tracking algorith for a binary sensor network. Clusters coposed of coodity processing nodes provide a eans for constructing econoical, powerful, robust and scalable high-perforance coputing systes. There are any possible ways to ipleent a parallel target tracking algorith on a cluster, such as assigning one track to one processor [5], assigning one sensor to one processor, or using a spatial parallelization approach [6]. Assigning one track to each processing node and processing in parallel has the advantage of load balancing on each node, but it has the disadvantage of significant counication overhead. Assigning each sensor to a processing unit is ore suitable to a network with a sall nuber of sophisticated sensors. With a spatial parallelization approach, also called geographic tessellation [6], the area of interest is divided into sall regions and each region is assigned to one coputing unit. Kalan filtering [7] and nearest neighbor data association (NNDA) are coonly used in ulti-target tracking [8-9]. With NNDA, the tracks are built by connecting the detected targets to their closest tracks if the distance is less than a threshold. A new track is created if no such previous track exists X/06/$ /06/$ IEEE 2

2 In the binary sensor networks considered here, the nuber of targets is far less than the nuber of sensors. The vast ajority of coputation tie is devoted to data fusion. Therefore, iproving data fusion efficiency and in turn reducing latency is the focus of this paper. A parallel siulated annealing algorith with efficient search strategy is proposed and tested. The rest of this paper is organized as the follows: section II gives an overview of the binary sensor network odel. Section III presents the search based data fusion algorith used in the network siulations. Section IV outlines the strategy for parallel coputing. Siulation results with the proposed algorith are presented in Section V, followed by conclusions in Section VI. II. SENSOR NETWORK MODEL AND SIMULATION DATA Real tie tracking of aircraft and issiles within a theater of operations is a proble of significant interest. This research considers the possibility of accurately tracking a large nuber of air targets over a very wide area using very large nubers of priitive sensors. Target tracks for the siulation are generated using Modern Air Power, a theater level air cobat siulation used by the USAF Squadron Officers College (SOC) for training. The sensor networks cover an area of 800 iles by 800 iles, and are equally divided into sall regions for parallel coputing. The binary sensors have identification and position inforation, and report on when there are targets within range. A 0% false positive rate and 0% false negative rate are used for the binary sensors. The set of network paraeters in the siulations are as the follows: Nuber of sensors: varied between 250,000 and 2,500,000 Sensor range: varied fro 3 to 0 iles Sapling rate: every 0, 5, 3, 2,.5, and 0.5 seconds Nuber of divided regions (slave processors): 49, 36, 25, 6, 9, 4 and Because of the liited inforation provided by each sensor, large nubers of sensors are needed to cover the desired region. This is not ipractical considering the low cost of such sensors. Siulations are perfored on a Linux cluster where each node ay have ore than one processor. The sensor network is divided spatially, and each region is assigned to a processor ipleenting data fusion for that region. During siulations, the aster process reads the target track inforation and passes it to the slave processes via MPI. Each slave process coputes the siulated response of each of the binary sensors in its doain, and then perfors Siulated Annealing (SA) based data fusion to identify and locate targets within its region of interest. At end, the slave process passes those target locations back to the aster process via MPI. When the aster process obtains inforation fro all slave processes, it resolves any possible conflicts in target locations and perfors data association and tracking using Nearest Neighbor Data Association (NNDA) and a Kalan filter. The flow chart of siulation odel is shown in Fig. Input Read Tracks Master Processor Fig.. Flow chart of target tracking siulation High probability of target detection and low false alar rate are desired for a reliable sensor network. The average distance error of the detected targets and how these errors distribute reflect the accuracy of the detection. Hence, the following paraeters are used in evaluating the sensor network perforance: Probability Of Detection (POD): Percentage of the targets actually detected during the entire siulation, False Alar Rate (FAR): Percentage of detects that are false; Average Deviation (AVD): Average position error for the detected targets. The total nuber of tie steps is deterined by the tie interval for the 5 inutes siulation. If the total nuber of tie steps is, POD, FAR and AVD are coputed as the following: POD = NuDetectedTrueTargets FAR = NuDetectedFalseTargets AVD = NuTrueTargets ; NuDetectedTargets ni 2 2 ( ( x x ) + ( y y ) n. k = Net Net2 Netn- Netn Slave Processors Resolve Conflicts at Boundaries Next Tie Step d, k t, k d, k t, k ) Kalan Filter and NNDA Master Processor i Output Tracks Here, TrueTargets refers to the targets fro input track data, and n i refers to the nuber of true targets at tie step i. III. DATA FUSION WITH SIMULATED ANNEALING ALGORITHM A data fusion process is used to cobine inforation fro individual binary sensors to identify and locate targets. The input to a data fusion algorith is a set of noisy sensor observations, and the output is a set of target positions. In this paper, a search based data fusion approach with siulated annealing algorith was used. The Siulated Annealing (SA) algorith proposed by Kirkpatrick [0] exploits an analogy between the way in which a etal cools and freezes into a iniu energy crystalline structure (the annealing process) and the search for a iniu in a ore general syste. SA has an advantage over greedy search ethods because it is less likely to becoe trapped at a local iniu. In target tracking, the constraint for the optiization is the highest consistency of network sensor ; X/06/$ IEEE 3

3 readings. Here, a consistent reading refers to a sensor which is on when the target is within its detection range and off otherwise. In standard SA approach, the algorith begins with a particular solution and applies a series of transfors to it. After each transfor, a fitness function is used to evaluate the new solution, and a decision is ade whether to accept the new solution or retain the previous one. In a real tie application, the process is repeated for as any iterations as possible given the tie constraints. The transfor will always be accepted if it results in an iproveent to the objective function. Transfors are soeties accepted even if they do not yield iproveent. This is done in an attept to avoid local inia in the search space. The decision to accept a new solution is based on the current annealing teperature. If the teperature is high, siulated annealing accepts nearly any transfor and essentially behaves like a rando walk. If the teperature is low, solutions are accepted only if they result in iproveent. In the ipleentation, a rando nuber r between 0 and is generated and copared with a probability p deterined by the iproveent and the teperature at that tie. The transfor will be accepted if r p. The probability p is coputed as the following: p = exp( δf / T). Here δf is the change to the objective function between the current and new solutions, and T is the current teperature. The teperature T is controlled by the algorith paraeters T ax and T in. The teperature starts at T ax on the first iteration, and ends at T in on the final iteration. Begin with an epty target set For each tie step Set teperature T to Tax Calculate cooling rate R based on Tax, Tin, NuTransfors For count = to NuTransfors Choose a transfor at rando (Add,Move,Delete) Apply the transfor to the odel Copute consistency for the resulting transfor Accept the transfor if it results in iproveent OR if it is close enough to the current solution quality based on T Reduce teperature T by cooling rate R End For End For Fig. 2. Siulated annealing algorith For the target location application, the SA search begins with an epty target set and applies a series of transfors until the target set achieves axiu consistency with the sensor observations. The SA search applies three types of transfors to the target set: Add Target: add a target to the set Delete Target: delete a target fro the set Move Target: change position of a target Any target currently in the set is a candidate for the delete or ove transfor. Candidates for add transfors are selected by finding sensors with on readings that do not have a target fro the set in their range. Appropriate values of T ax and T in were deterined experientally for the application. It was found that an initial teperature of e-7 and final teperature of zero yielded the best results. The SA algorith for target identification is shown in Fig. 2. As the siulation proceeds, aircraft enter and leave the area of interest, take off and land (or crash), and ove within the area of interest. At each step in the siulation, a siulated annealing search is perfored to identify and locate the targets based on current sensor readings. Target locations fro the previous step in the siulation are used to seed the search in the current step. IV. REAL TIME SIMULATION For the application described above, the nuber of targets is typically uch saller than the nuber of sensors. Since the tie coplexity of the tracking and association steps depend only on the nuber of targets, and the tie coplexity of the data fusion process depends priarily on the nuber of sensors, the fusion step consues the ajority of coputation tie. This is illustrated in Fig. 3. Latency is defined as the tie delay between the data being received at the input and the results being available. For a real tie tracking application, the latency ust be less than or equal to the period between data saples. Parallel coputing together with a sart search strategy are used to reduce latency. Transit Target Inforation to Child Processors Using MPI Collect Results fro Child Processors Using MPI Resolve Conflicts Perfor Data Fusion at Child Processors in Parallel Siulation Tie Interval Data Association Fig. 3. Coputation tie illustration at each tie step Tracking A. Parallel Coputing Parallel coputing is used to distribute coputation to any processors, reduce latency, and increase throughput. Coputing clusters are widely used in high perforance coputing applications. A coputer cluster with Red Hat Enterprise Linux AS Release 3 and la-oscar is used for the sensor network siulation. The cluster has total of 25 nodes, and each node has two SMP CPUs of 2.6 GHz. The Open Source Cluster Application Resources (OSCAR) package is a fully integrated software bundle designed for high perforance cluster coputing. OSCAR provides the standard Message Passing Interface (MPI) for the counications between the parallel coputing processes. A spatial decoposition technique is used to parallelize the data fusion process. The sensor network is divided into sall regions and each region is assigned to one processor of the cluster. Data fusion is perfored on these sall networks by each process in parallel. Spatial decoposition has been widely used in coputational fluid dynaics (CFD) research as well as in other sensor network studies. This approach is referred to as tessellation sensor data fusion (TSDF) [6]. At the end of data fusion step, the slave processors send their list of X/06/$ IEEE 4

4 identified targets to the aster processor, which erges the observations to get the overall set of target locations. Since only the observed target inforation is sent to the aster processor through MPI, this overhead is relatively sall. A potential proble arises if there are targets close to the border of a region. Soe sensors that could see those targets ay actually be in another region. To handle this boundary proble, the sensor networks are divided into regions with slight overlap. It is therefore possible that two neighboring regions ay report soe of the sae targets. This internal boundary conflict is handled by the aster process after it receives the target location inforation fro all the slave process. B. Sart Search with Update Windows In search based data fusion, a set of target locations that axiizes consistency with sensor network siulations is sought. The search process involves testing any possible sets of target locations. Iproving the efficiency in coputing the odel sensor consistency and increentally re-coputing it after each transfor are crucial in reducing the latency. For the types of networks under consideration here, naely networks of priitive short range sensors, the effects of a particular transfor are local to the area where the transfor occurs. There is no reason to check every sensor within the entire network, but only those close by. These considerations dictate the level of granularity at which the consistency coputation should be perfored. Hence, we further divide the network using regular grids. Each sensor is assigned to a particular grid location, and the consistency counts are aintained for each grid. When a transfor is ade, only the grids within sensor range need to be updated. Fig. 4 illustrates the relation of whole network, sub region and update windows. Sub-Region Boundary Update Window Overlapped Window Fig. 4. Sensor network configuration illustration: the network is divided into 6 sub regions with overlap for parallel coputing, each region is further divided into sall windows to expedite data fusion. Since there ay be single oves that span ore than one window, and since sensors in one window ay see into neighboring windows, it is necessary to ake sure that each processor sees the windows surrounding its area of interest. This eans that the processor sees areas that overlap by one update window. The update window size is chosen such that it is not possible for a single sensor s range to be less than the size of the window and is set to 0 iles in the siulations. Since the nuber of sensors in each region is approxiately the sae, and the data fusion tie priarily depends on the nuber of sensors, a spatial division of the processing does a reasonably good job of balancing the load between processors. In other applications where the nuber of targets is close to the nuber of sensors, a different parallelization approach such as distributing one target to one processor ight be a better approach. In order to ake the siulation as close to real tie as possible, the allowable data fusion tie is adjusted at each tie step based on processing tie on the aster processor, which includes data counication overhead, tracking and I/O. The allowable data fusion tie is the difference of the tie interval and the tie used on the aster processor at the last tie step. Data fusion will stop when the allowable fusion tie is reached and the best network state at that oent will be returned. The algorith for real tie sensor network siulation is outlined in Fig. 5. Begin with an epty target set For each tie step Get latest target locations Send latest target locations to slave processors (MPI) On each slave processor (in parallel) Identify targets with SA algorith Transit fusion results to the aster End of this processor Resolve network boundary conflicts Perfor data association and tracking End For Fig. 5. Real tie siulation algorith V. NETWORK PERFORMANCE ANALYSIS The perforance of a binary sensor network depends on several factors, including the nuber of sensors, the range of the sensors, the sapling rate, and the nuber of regions into which the network is divided. The effects of these paraeters are investigated through a series of siulations with different network settings. A. Nuber of Regions The nuber of regions a network needs to be divided into depends on any factors, such as siulation tie step, the nuber of sensors and the available processors of the coputing cluster. Norally, high sapling frequency leads to better target tracking as the changes of target state are saller within shorter periods of tie. However, the tie interval has to be long enough so that the SA search process can find a good solution. Table I shows the siulation results of a network with 2.5 illion binary sensors each with a range of 3 iles, with the network divided into 4 regions. One can see that even with tie interval of 0 seconds, the network only has about 94% target detection rate (POD). This indicates that the optial solution could not be reached within the tie constraint. The network perforance gets worse when the tie interval is reduced further. If the network is divided equally into 9 regions, the perforance iproves significantly as the results in Table II show. Even with tie interval of 0.5 seconds, the network can X/06/$ IEEE 5

5 still have up to 96% of target detection rate and less than % false alar rate. TABLE I Perforance Metrics of Sensor Network with 2,500,000 Binary Sensors Divided into 4 Regions Tie Interval (seconds) POD FAR AVD (iles) % 0.08% % 0.5% % 0.29% % 0.83% % 6.78% % 44.5% TABLE II Perforance Metrics of Sensor Network with 2,500,000 Binary Sensors Divided into 9 Regions Tie Interval (seconds) POD FAR AVD (iles) % 0.% % 0.22% % 0.00% % 0.0% % 0.06% % 0.63% iles, the siulation results show that 750,000+ sensors are needed for the area of 800 iles by 800 iles in order to have a higher than 99% POD and lower than % FAR. Table IV shows the results of a network with different nuber of binary sensors and a tie interval of 5 seconds. The network is divided into 49 sall regions. The high FAR in network with 250,000 sensors is due to the fact that there are any areas that are covered by only one or two sensors, and the sensors have a high false positive rate. It takes a higher sensor density to reliably eliinate the noise. TABLE IV Perforance Coparison of Networks with Different Nuber of Sensors, Tie Interval 5 Seconds, Sensor Range 3iles, 49 Sub Regions Nuber of Sensors POD FAR AVD (iles) 250, % 69.32% , % 3.7% , % 2.4% 0.220,000, % 0.62% 0.76,250, % 0.24% 0.54,500, % 0.09% 0.42,750, % 0.02% ,000, % 0.05% ,500, % 0.09% 0.9 In both Table I and Table II, one can see that the average distance error (AVD) increases with the decrease of tie interval. This is also an indication of non-optial data fusion results due to the lack of fusion tie. TABLE III Perforance Metrics of Sensor Network with 2,500,000 Binary Sensors Divided into 49 Regions Tie Interval (seconds) POD FAR AVD (iles) % 0.33% % 0.36% % 0.2% % 0.02% % 0.03% % 0.02% 0.26 By further dividing the network into ore regions, the data fusion tie for each region will reduce ore due to the saller nuber of sensors. Table III shows the siulation results when the network was divided into 49 regions. The average distance errors are only around 0.2 iles for all the different tie intervals with alost 00% POD and less than 0.4% FAR. This indicates that optial solutions are reached even for a tie interval of 0.5 seconds. In general, one should choose the iniu nuber of regions such that the search converges, since each region requires its own processor. B. Nuber of Sensors Since binary sensors norally have short sensing range, the coverage of sensors in the network affects the target detection rate. The better the sensor network coverage, the higher the target detection rate of the network. For a sensor range of 3 Fig.6. Average distance error of tracking with networks of different nubers of sensors and different tie intervals Fig. 6 shows the average distance errors (AVD) of networks with different nubers of sensors and different tie intervals. There is little effect on AVD with the variation of tie interval, an indication that data fusion is finished at every tie step during siulations. Once illion sensors are present (a density of.56 sensors per square ile), a point of diinishing returns is reached in ters of target localization. Fig. 7 illustrates how the distance errors are distributed. Distance errors for the siulations with a tie interval of 5 seconds are discretized using a 0.05 ile increent up to 2 iles, and the density functions of distance errors are plotted for varying nuber of sensors. One can see that for networks with,000,000+ sensors, ore than 95% of the detected targets are within the range of 0.4 iles. C. Sensor Range In general, decreasing sensor range will iprove localization X/06/$ IEEE 6

6 accuracy, provided there are enough sensors to cover the area of interest. This is illustrated in Fig. 8, which shows siulation results for a network of 2,000,000 sensors in an 800 x 800 ile area. Decreasing sensor range results in higher detection and lower false alar rates. Fig. 7. Density functions of the distance errors for networks of different nubers of sensors. insufficient density to cover the area. VI. CONCLUSIONS A parallel real tie data fusion and target tracking algorith for very large binary sensor networks is presented. The parallel data fusion algorith proposed in this paper is efficient and highly scalable. More processing units can be added as network size increases with no significant increase of counication overhead. There is no counication between slave processors, and the nuber of targets in the siulation deterines the counication overhead between the aster and slave processors. The algorith is tested with a wide variety of sensor network paraeters using siulation data fro a theater level air cobat siulation. It is deonstrated that very accurate target detection and localization are possible even though the binary sensors theselves provide little inforation and have high error rates. Real tie tracking is perfored on a network with 2.5 illion sensors using a tie interval of 0.5 seconds on a coodity cluster with only 50 processors. Tracking accuracy iproves as the range of the binary sensors decreases, provided there is sufficient density to eliinate the noise. Perforance is iproved by using a fine level of granularity to perfor sensor consistency updates after each transfor. Fig. 8. Siulation results of networks of 2,000,000 sensors with different sensor ranges. Fig. 9. Siulation results of networks of 250,000 sensors with different sensor ranges. Fig. 9 shows siulation results for a network of 250,000 sensors in the sae area. In this case results iprove until the sensor range reaches 5 iles. Below that point, there is REFERENCES [] L. Doherty, B.A. Warneke, B.E. Boser, and K.S.J. Pister, Energy and perforance considerations for sart dust, International J. of Parallel and Distributed Systes and Networks, Vol. 4, pp. 2-33, 200. [2] S. Sandeep Pradhan, Julius Kusua, and Kannan Rachandran, Distributed copression in a dense icrosensor network, IEEE Signal Processing Magazine, pp. 5-60, March [3] Javed Asla, Zack Butler, Florin Constantin, Valentino Crespi, George Cybenko, Daniela Rus, Tracking a oving object with a binary sensor network, in Proc. Of ACM SenSys. ACM Press, Nov [4] Volkan Isler, Sanjeev Khanna, John Spletzer, Caillo J. Taylor, Target tracking with distributed sensors: The focus of attention proble, Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systes (IROS), [5] Charles Pedersen, Architecture and perforance of a parallel patafusion ulti-target tracking code for real-tie applications, Software Tech News, Vol. 4, No., High Perforance Coputing. Available: [6] Patrick P.A. Stors, J. Bernard van Veelen, and Erik Boasson, A process distribution approach for ultisensor data fusion systes based on geographical data space partitioning, IEEE Transactions on Parallel and Distributed Systes, Vol. 6, No., January [7] Donald B. Reid, An algorith for tracking ultiple targets, IEEE Transactions on Autoatic Control, Vol. AC-24, No. 6, Dec [8] Chee-Yee Chong, David Garren, and Tiothy P. Grayson, Ground target tracking- a historical perspective, Aerospace Conference Proceedings, 2000 IEEE, March, vol.3. [9] Michael K. Kalandros, Lidija Trailovic, Lucy Y. Pao, and Yaakov Bar- Shalo, Tutorial on ultisensor anageent and fusion algoriths for target tracking, Proceeding of the 2004 Aerican Control Conference, Boston, Massachusetts, June 30 July 2, [0] S. Kirkpatrick, C.D. Gelatt, and M.P. Vecchi, Optiization by siulated annealing, Science, Vol. 220, pp , X/06/$ IEEE 7