Leak Detection using Video Sensors for Shipboard Systems Surveillance

Transcription

1 Steven Latman, Philip Mease, George D. Lecakes, Jr., Michael Russell and Shreekanth Mandayam Patrick Violante and Kimberly J. Drake Leak Detection using Video Sensors for Shipboard Systems Surveillance ABSTRACT Anomalous indications in monitoring equipment onboard U.S. Navy vessels must be handled in a timely manner to prevent catastrophic system failure. The development of sensor data analysis techniques to assist a ship's crew in monitoring machinery and summon required ship-to-shore assistance is of considerable benefit to the Navy. This project addresses the design of video sensor systems (hardware, software and algorithms) for diagnosing the health of shipboard equipment. The specific focus is on detecting leaks in water distribution systems with surveillance video. Automated algorithms were designed and developed to detect for anomalous conditions that indicate the presence and severity of potential leaks. This paper presents a description of the leak-detection algorithm, and results obtained by exercising the technique on a suite of video sensor data obtained under laboratory conditions. Preliminary results indicate that the algorithms provide sufficient accuracy and robustness of detection, demonstrating the potential for field testing. INTRODUCTION The systems on board a U.S. Navy ship are often complex and interdependent. These systems are required to operate under hostile conditions (Zivi 2002), relying on faults and failures to be handled in a narrow time window to prevent catastrophic failure. Currently, engineers are dispatched to areas around the globe to repair these systems. Remote monitoring techniques, specifically those techniques that allow a crew to summon ship-to-shore assistance, would help prevent excess damage and minimize the time, cost, and manpower required for repairs on deployed naval vessels. Utilizing virtual reality technology in combination with fully immersive navigable environments and data fusion for representing data helps an engineer diagnose problems intuitively (Lecakes 2008) without having to travel to the ship. This technique can also be used to asist a ship s crew in monitoring machinery offsite, effectively expanding the resources of the vessel for fault-detection. Software algorithms can be created to automatically detect an anomaly in a system that could possibly precipitate into a catastrophic system failure. This paper reports on an ongoing partnership between Naval Systems Warfare Center (NSWC) NAVSEA and Rowan University to design, develop and test software and hardware systems for video sensing. Video data is obtained by the surveillance of machinery designed to simulate common anomalous conditions aboard a naval vessel. This system can be configured to model various leak types, including the inherent variances of the video data caused by scale, translation, and rotation. Algorithms to automatically recognize and call attention to the presence of leaks from a shipboard system were developed over the course of the project as a proof-of-concept for video processing in the application of monitoring onboard machinery. These algorithms will then be interfaced with the Cave Automatic Virtual Environment (CAVE) to provide an intuitive problem solving environment for engineers to diagnose and help repair fatigued or damaged systems. APPROACH The overall approach for this project can be summarized as follows: Generation of video sensor data for development and validation of algorithms;

2 Laboratory simulation of leaks to model real-world shipboard data; Robust feature extraction from video data; Classification of video data by an artificial neural network (ANN). To evaluate the robustness of the feature extraction routines used, both simple and complex synthesized shapes were created. The shapes used are shown in Figure 1. These shapes were then purposely scaled, translated, and rotated. This transformed data was fed through both feature extraction methods: Invariant Moments (Hu 1962) and Discrete Cosine Transform () (Gonzalez 2008) in order to test the algorithms invariance to these types of transforms. (a) (b) Figure 1. Square (a) and teapot (b) shapes used for feature extraction. Once the feature extraction routines were validated, real video data of actual anomalies was required. Two systems were used for this purpose: the Hampden H-6925 Fluid Circuit Generator shown in Figure 2 and the Modular Leak Simulator (MLS) shown in Figure 3. Figure 3 Modular Leak Simulator (MLS). The Hampden H-6925 system was used in the early stages and only provided basic stream and drip type anomalies. To expand the anomalous data set, the MLS system was constructed to allow several additional types of anomalies (categorized as leaks) to be simulated with high level of control. These conditions include: Varying Intermittency/Size Drips (Figure 4b) Fine to Coarse s (Figure 4c) Fine to Heavy s (Figure 4d) (a) (b) (c) (d) Figure 4. Video data: (a) benign, (b) drip, (c) mist, and (d) stream. Figure 2. Hampden H-6925 Fluid Circuit Generator. Currently, the medium used for simulating leaks is water, however, the MLS system can be easily reconfigured to simulate leaks with other fluids and gases. Video data was collected using a Pelco CC3500 surveillance camera, fed through a Canopus ADVC55 digital video converter, which stored the raw video in.avi format to the host machine. Translated video data was obtained

3 experimentally by moving the camera stage by various displacements. Leak Detection Algorithms The first stage of algorithm development is to segment the video into single image frames. From these images, simple 2-D processing can be performed. Next, motion detection was implemented by averaging the benign video data into a single average frame which is subtracted from the anomalous frames and binarized using a simple threshold. This method removes most, if not all, of the data that is not changing. This step greatly reduces the quantity of data to be processed downstream. A result of the motion detection process is shown below in Figure 5. Final Anomaly Frame Raw Anomaly Frame Figure 5. Motion detection process. Average Frame After the frame backgrounds were eliminated with the motion detection algorithm, they are fed to either an invariant moment or feature extraction algorithm. The outputs are highly compressed representations of the video data and are expected to contain pertinent features of the anomaly. These feature vectors should have the following ideal properties to describe the shapes of the leaks (Clark 1981): Uniqueness The algorithm must be able to distinguish between different shapes, by assigning a set of numbers that are unique to each shape. Parsimony The algorithm should use the smallest possible set of numbers to describe a particular shape, in order to reduce noise susceptibility. Independence No feature vector should require the value of another feature vector for its calculation. Invariance Feature vectors should be resistant/invariant to irrelevant transformations of scaling, translation or rotation. The feature vectors can then be fed as inputs to a classifier. Here, the selected classifier was chosen to be the multilayer perceptron (MLP), a feedfoward-type ANN (Haykin, 2009). The MLP is commonly used for classification purposes. Training and test data are selected from all available feature vectors by taking 50% of frames for training and the remaining 50% reserved for testing. Activation functions used are tan-sigmoid for the hidden layer and purelinear for the output layer. Output classes for the network include benign and anomalous indications. For the results presented here, the classification outputs are: benign, stream, and mist. The output of the network chooses the closest value by a winner-takes-al selection for each classification type. The entire process is outlined in Figure 6. Video Input Training Training Vector Testing New Vector Segment Video to Frames ANN (MLP) ANN MLP Image Processing Feature Extraction Target Vector Classification Outputs Figure 6. Flowchart of entire process from raw video data to classification.

4 RESULTS Comparisons of both the moment and feature extraction methods for benign, translated benign, stream, and translated stream are shown in Figures 7 and 8. Figure 7. feature comparison: untranslated/translated benign and stream. Figure 8. Moment feature comparison: untranslated/translated benign and stream. These comparisons display the degree of invariance of the algorithms for translated data. Feature vectors results for and invariant moments are presented in Table 1. Table 1. Feature vector calculation and classification results Frame Description Image Background Subtraction Coefficients Moment Coefficients Target _0001 _0003 bn_01_0003 bn_01_0003 _0001 _0003 vlsw_01_0001 vlsw_01_0003 _0001

5 _0003 vlmw_01_0001 vlmw_01_0003 An average of the and invariant moment classification results of untransformed and transformed data out of ten classification attempts are shown in Table 2. Table 2. Classification results of various tests with untransformed and transformed data Feature Vector Transformed and Hu Hu Transformed and Videos Used bn_01 vlsw_01 vlmw_01 bn_01 vlsw_01 vlmw_01 CONCLUSIONS Images for Training Percent Correct In this paper, we have presented the description of feature extraction and classification algorithms to analyze video data simulating ship-board conditions for benign and anomalous indications. Currently, the classification results for the test data have been shown to be highly successful. It can be noted that the system still needs to be exercised with larger data sets containing higher variability. We are currently working on obtaining additional video data that will increase the robustness of the network by training it with larger data sets. REFERENCES Clark, M. W., Quantitative shape analysis: a review, Mathematical Geology, vol. 13, pp , Gonzalez, R. C. and R. E. Woods, Digital Image Processing, Third Edition, pp , Upper Saddle River, New Jersey: Prentice Hall, 2008 Haykin, S. S., Neural Networks and Learning Machines, Third Edition, pp , Upper Saddle River, New Jersey: Prentice Hall, 2009 Hu, M. K., Visual patern recognition by moment invariants, IRE Transactions on Information Theory, February Lecakes, G. D., J. A. Morris, J. L. Schmalzel, S. Mandayam, Virtual Reality Platforms for Integrated Systems Health Management in a Portable Rocket Engine Test Stand, IEEE Instrumentation and Measurement Technology Conference Proceedings, May Violante, P., K. Drake, S. Latman, P. Mease, and S. Mandayam, Virtual Environments, Fusion, and Video in Support of Remote Health Management Systems, IEEE International Workshops on Haptic Audio Visual Environments and their Application, October Zivi, E. L. Integrated shipboard power and automation control challenge problem, IEEE Power Engineering Society Summer Meeting, July 2002.

6 ACKNOWLEDGMENTS The authors would also like to acknowledge NAVSEA Philadelphia for their sponsorship and collaborative support of his project. Steven Latman is a graduate student in Electrical & Computer Engineering at Rowan University, in Glassboro, NJ. He graduated with a B.S. in Mechanical Engineering from Rowan University in Philip Mease is a graduate student in Electrical & Computer Engineering at Rowan University, in Glassboro, NJ. He graduated with a B.S. in Electrical & Computer Engineering from Rowan University in George D. Lecakes, Jr. is a graduate student in Electrical & Computer Engineering at Rowan University, in Glassboro, NJ. He graduated with a B.S. in Civil & Environmental Engineering from Rowan University in Michael Russell is a graduate student in Electrical & Computer Engineering at Rowan University, in Glassboro, NJ. He graduated with a B.S. in Electrical & Computer Engineering from Rowan University in Shreekanth Mandayam is the Chair of the Electrical & Computer Engineering department at Rowan University. He directs the Imaging and Virtual Reality Laboratory at the South Jersey Technology Park Patrick Violante is with the Advanced Machinery Systems division at NAVSEA in Philadelphia. Kimberly J. Drake is with the Naval Surface Warfare Center, Carderock Division, in Philadelphia.