AUTOMATIC CLASSIFICATION OF LIDAR POINT CLOUDS IN A RAILWAY ENVIRONMENT

Transcription

1 AUTOMATIC CLASSIFICATION OF LIDAR POINT CLOUDS IN A RAILWAY ENVIRONMENT MOSTAFA ARASTOUNIA March, 2012 SUPERVISORS: Dr. Ir. S.J. Oude Elberink Dr. K. Khoshelham

2 AUTOMATIC CLASSIFICATION OF LIDAR POINT CLOUDS IN A RAILWAY ENVIRONMENT MOSTAFA ARASTOUNIA Enschede, The Netherlands, March, 2012 Thesis submitted to the Faculty of Geo-Information Science and Earth Observation of the University of Twente in partial fulfilment of the requirements for the degree of Master of Science in Geo-information Science and Earth Observation. Specialization: GEOINFORMATICS SUPERVISORS: Dr. Ir. S.J. Oude Elberink Dr. K. Khoshelham THESIS ASSESSMENT BOARD: Prof. Dr. Ir. M.G. Vosselman (Chair) Dr. Ir. D. Lutters (External Examiner, University of Twente)

3 DISCLAIMER This document describes work undertaken as part of a programme of study at the Faculty of Geo-Information Science and Earth Observation of the University of Twente. All views and opinions expressed therein remain the sole responsibility of the author, and do not necessarily represent those of the Faculty.

4 ABSTRACT The purpose of this research is to classify points on rails, contact and catenary wires from 3D lidar point clouds in a railway environment. Two dataset were used for this research i.e. a terrestrial dataset and an airborne dataset. The classification of objects in a railway environment is important as railway companies tend to have a database of their infrastructures due to maintenance and design purposes. In addition, by automating this process companies reduce the cost of maintenance substantially and also by using 3D lidar data they obtain more precise data of their infrastructures. The idea is to do a knowledge based classification which takes advantage of our knowledge of railway environment to classify points on the objects of interest. First characteristics of objects of interest are inspected and based on them constraints are defined. To do the classification, two different methods are developed. First Template Matching (TM) is developed based on the idea of defining a pattern of rails and wires and classifying points by seeking this pattern in data. Next, Region Growing (RG) method is developed based on the idea of finding small parts of rails and wires as seed points and then looking for the rest of them in the neighbourhood of already classified ones based on their characteristics. Results of both methods were evaluated for completeness, correctness and speed by visual inspection and statistical analysis. Both TM and RG methods were applied for terrestrial dataset and they both managed to classify majority of rail and wire points (completeness). Results of RG contained less wrong classifications than that of TM (correctness) and RG did the classification 3 times as fast as TM (speed). As the result of RG in terrestrial dataset was better than that of TM, only RG method was applied for airborne dataset to check its performance to classify points on curved rails. RG method managed to classify majority of rail and wire points with no considerable wrong classifications. To conclude, the idea of knowledge based classification of points on the objects in 3D lidar point clouds is feasible as both methods of TM and RG succeeded to do so. However, RG was more promising and more efficient than TM due to its more accurate results and higher speed algorithm. Furthermore, both methods still can be improved to be more efficient i.e. getting more accurate results in a shorter period of time. Finally as the classification is done just based on characteristics of objects, it can be done for other objects in a railway environment as well as objects in other environments e.g. urban and roads. Keywords Mobile laser scanning (MLS), lidar, Knowledge based classification, 3D point clouds, Region Growing, Template Matching i

5 ACKNOWLEDGEMENTS I owe my deepest gratitude to my supervisors Dr. Ir. S.J. Oude Elberink and Dr. K. Khoshelham for guiding me through my MSc thesis. My special thanks for their patience, knowledge and support. I would like to thank all staff in GFM Domain, ITC Faculty, University of Twente, The Netherlands for their support which made this thesis possible. I offer my sincere gratitude to Prof. Dr. Ir. M.G. Vosselman for his help, advice and time. I also would like to thank my dear friends. It was a great experience to study with people from different countries with different language, culture, interest, attitude, music and food. I do not have a special word to show my thankfulness to my family. This thesis would not have been possible without help, support, guidance and encouragement of my dear family who always supported me in hard circumstances. Finally, I dedicate this thesis to my dear parents Mohammad Ali Arastounia and Sakineh Bahrami Hidaji and my dear brother Mehrdad Arastounia. ii

6 TABLE OF CONTENTS List of figures... iv 1. Introduction Motivation and problem statement Research identification Thesis structure Literature review Principles of laser scanning Road extraction from 3D laser scanned data Building extraction from 3D laser scanned data Power line extraction from 3D laser scanned data Object extraction in an urban area from 3D laser scanned data Object extraction in a railway environment from 3D laser scanned data Materials First dataset: terrestrial laser scanned data Second dataset: airborne laser scanned data Softwares used for programming and visualization Methodology and implementation of 3D lidar data classification General approach Characteristics of objects of interest Applied constraints Template Matching method Region Growing method Results of 3D lidar data classification Results of Template Matching method on terrestrial laser scanned data Results of Region Growing method on terrestrial laser scanned data Results of Region Growing method on airborne laser scanned data Discussion about 3D lidar classification Discussion about common steps of Template Matching and Region Growing methods Discussion about Template Matching Discussion about Region Growing Comparion between Template Matching and Region Growing Conclusions and recommendations for future classification of 3D lidar data Answers to research questions Conclusions Recommendations List of references Appendix 1: Description of wires in a typical railway environment Appendix 2: The railway environment of the terrestrial dataset in Google maps Appendix 3: The railway environment of the airborne dataset in Google maps Appendix 4: A closer view of parts of terrestrial dataset where different methods were applied Appendix 5: Other rail patterns and their results in Template Matching method iii

7 LIST OF FIGURES Figure 3-1. An oblique view of the provided dataset of a railway environment in The Netherlands... 9 Figure 3-2. Two gap areas caused by shadow effect of the outer rails Figure 3-3. Front View of the airborne laser scanned data Figure 4-1. Flowchart of Template Matching method Figure 4-2. Grey arrows point to height jumps caused by rails while the white arrow points to other parts of track bed without any height jumps Figure 4-3. Using 10, 80 and 90 percentiles to remove noises for implementation of height jump detection method Figure 4-4. Big height jump caused by railway station platform V.S. small height jumps caused by rails Figure 4-5. Relative height of wires to rails Figure 4-6. In top view, wires shown by yellow and brown colour lie between rails shown by green colour Figure 4-7. Rail and wire pattern in a railway environment Figure 4-8. Rail and wire pattern as a greyscale image Figure 4-9. Flowchart of Region Growing method Figure 5-1. Result of pre classification of points based on their heights Figure 5-2. Result of first classification of rails including some wrong classifications and some notclassified points Figure 5-3. The extent of railway environment in Google maps Figure 5-4. Result of first classification of rails and wires. Wrong classifications are shown by ovals and arrows point to areas where no rail and wire points are classified Figure 5-5. The greyscale image generated from 3D point clouds Figure 5-6. The Result of Template Matching containing rails and wires Figure 5-7. Result of Template Matching containing only rail pattern Figure 5-8. Filling gap areas by Hough Transform method Figure 5-9. Result of Template Matching with filled gaps. Grey points represent new candidate rail points Figure Result of converting greyscale image back to 3D point clouds Figure Result of Template Matching after verifying candidate rail points Figure Final result of Template Matching method with some points on a tower which are wrongly classified as catenary wires Figure Result of height jump detection method for the first 5 m of rails with some points wrongly classified as rails and some area where no rail points are classified Figure Result of Hough Transform method in which seed points of rail growing are recognized and differentiated from points on track bed Figure Result of classification of the first 5 meters in which rail and wire points are classified separately Figure Result of classification of the first 5 meters in which each pair of rails and their corresponding contact wire are recognized separately Figure Result of classification of the first 5 meters in which all rails and wires are recognized differently Figure Growing pattern of rails in top view Figure Growing pattern of contact wires iv

8 Figure Final result of Region Growing. Rails, contact and catenary wires are classified in different classes Figure Final result of Region Growing in which each pair of rails and their corresponding wires are recognized separately Figure Final result of Region Growing in which all rails and wires are recognized differently Figure Result of height jump detection method for terrestrial dataset Figure Result of height jump detection method for airborne dataset Figure Seed points of different rails and wires in airborne dataset which were extracted manually Figure Growing pattern of rails in airborne dataset Figure Growing pattern of wires in airborne dataset Figure Final result of RG on airborne dataset Figure 6-1. Appearance of rails in high resolution terrestrial point clouds Figure 6-2. Appearance of rails in low resolution airborne point clouds Figure 6-3. Accumulative histogram of geometric distances of classified rail points to 3D rail models for Template Matching (Diaz Benito, To be published in March 2012) Figure 6-4. Accumulative histogram of geometric distances of classified rail points to 3D rail models by Region Growing (Diaz Benito, To be published in March 2012) v

9

10 1. INTRODUCTION 1.1. Motivation and problem statement The purpose of this research is to automatically classify points on objects of interest from 3D lidar point clouds in a railway environment. Two datasets were used in this research. First was a terrestrial laser scanned data and the second was an airborne laser scanned data, both acquired by mobile laser scanning (MLS) in the railway environment but in two different areas in The Netherlands. The main aspects of this research are: 1. The need to have geo information about objects in a railway environment 2. Applying an automatic method 3. Using 3D mobile laser scanned data In the following paragraphs, the above mentioned aspects are discussed. A lot of railway companies want to have 3D data about their infrastructures including railways, wires, towers, signs, stations and other objects in the railway environment. In general, railway companies tend to have a database of their railway infrastructures. There are several motivations for having a database of railway infrastructures like for maintenance and design purposes. For the purpose of maintenance, having data about infrastructures helps companies to monitor them from time to time and check their deformation or if the rails are broken or the distance between them, gauge, has changed. Problematic ecosystem is another instance that can jeopardize the safety of the railway environment. Unchecked vegetation growth (e.g. massive tree roots) along the rail corridor can undermine the stability of the sub-grade material. On the other hand, some vegetation can also provide a needed mechanism to resist the effects of erosion. However, depending on their location, trees along the rail corridor can also interfere with passing trains by obstructing the free passage of rail vehicles (Morgan, 2009). This is absolutely crucial as any deformation of rails, change of gauge or problematic ecosystem will result in deadly catastrophes. Thus it highlights the importance and necessity of the maintenance. In addition to maintenance, future designs can also be improved. In other words, having 3D data about existing railway infrastructure provides the opportunity of finding its problems and defects and solving them for future designs. One of the motivations of this research is to automate the classification of points on the objects of interest in a railway environment. Currently in some countries the task of maintenance like inspection of rails is done manually i.e. by hiring some people to go to the railway environment and make sure that the rails are not broken or deformed. This makes the maintenance very expensive as the railway company has to pay for the hired people. Thus automating this task helps railway companies to decrease the cost and increase the speed of maintenance substantially. As a result, monitoring the railway environment and in general maintenance will be done more frequently which culminates in higher safety of travelling by trains and that is what railway companies are seeking for. A good starting point to develop an automatic algorithm to recognize objects is to inspect how humans do that. In fact, human being recognizes an object by considering its characteristics like size, shape, colour, material and its spatial relation to other objects and tries to compare them with objects he knows. In other 1

11 words, humans have a database of objects they are familiar with and to recognize another object, he tries to find the object in his database which is the most similar with the one he observed. This strategy is the core idea of this research i.e. teaching the computer the characteristics of our objects of interest as well as teaching it how to extract them. The following is the motivation of using MLS systems to collect data of railway environment. Some advantages of using MLS systems are high speed of data acquisition as well as high accuracy and resolution of data. The main benefit of terrestrial laser scanning as the imaging component compared with photogrammetric systems is the direct acquisition of 3D information (Vosselman et al., 2010) with respect to this issue that in mobile laser scanning Data Processing and product supply are principally identical to procedures in the terrestrial laser scanning static mode (Vosselman et al., 2010). Furthermore, MLS is less dependent on weather conditions than aerial photogrammetry and is cheaper than some other methods of data acquisition like ground surveying. As the output of MLS is lidar point clouds consisting of points with 3D coordinates, Antonarakis et al. (2008) stated Light Detection and Ranging (LiDAR) provides high resolution horizontal and vertical spatial point cloud data and is increasingly being used in a number of applications and disciplines, which have concentrated on the exploit and manipulation of the data using mainly its three dimensional nature. LiDAR information potential is made even greater though, with its consideration of intensity Research identification The purpose of this research is to automatically classify points on objects of interest from lidar data in a railway environment. The objects of interest are rails, contact wires and catenary wires and their classification is done by developing an algorithm that is able to recognize objects of interest by inspecting how these objects look like in the MLS data and also by inspecting their spatial relation to each other and to other objects. For a detailed description of wires in a typical railway environment see Appendix 1. The result which is a classified point cloud can be used for further analysis e.g. 3D modelling, making measurements, checking change of gauge and inspecting continuity of rails. For this research particularly, the classified point clouds of the terrestrial laser scanned data is used for 3D modelling of rails by Diaz Benito (To be published in March 2012) Research objectives The objective is to classify points on the objects of interest, each in one class and the remained points are classified in another class as other objects. As a result, there will be four classes and each point in the lidar point cloud will be labelled as one of the classes of rails, contact wires, catenary wires or other objects Research questions The following is the list of research questions: 1. What characteristics of the objects of interest can be used for classification? 2. How to define and implement the constraints for classification? 3. How to assess the result of classification? 2

12 Innovation aimed at The novelty of this research is how to use our knowledge of the railway environment to classify the objects of interest. In other words, the innovation is aimed at the way the constraints are defined and implemented for classification of MLS data in a railway environment Thesis structure This thesis consists of seven chapters. The first chapter is the introduction which explains the motivation, problem statement, research objectives, research questions and innovation. Then literature review is done in Chapter 2 which is followed by Chapter 3 including a description of data and soft wares used for this research. Chapter 4 starts with the description of characteristics of objects of interest and the applied constraints and is pursued by the explanation of the two developed methods i.e. Template Matching and Region Growing as well as their implementations. Chapter 5 presents the results of applying Template Matching and Region Growing methods. In Chapter 6, a discussion about the methodology and results is presented. Finally the last chapter answers to research questions and makes some conclusions and recommendations about potential follow ups and applied methods. 3

13 4 AUTOMATIC EXTRACTION OF OBJECTS FROM LIDAR POINT CLOUDS IN A RAILWAY ENVIRONMENT

14 2. LITERATURE REVIEW A large number of researches were done previously to extract various objects from laser scanned data in different environments like road or urban environment. However, not much research is done to extract objects in a railway environment. The data used for these researches were either airborne or terrestrial and some used the integration of laser scanned data with images. The following are some previous works of automatic object extraction from 3D lidar data in different environments. In cases that the extracted objects are different from objects in a railway environment, there were some points in developed method or strategy which could be used for extraction of objects in a railway environment Principles of laser scanning A reference text book is written by Vosselman et al. (2010) in which data acquisition, characteristics and some applications of lidar data are explained. Vosselman et al. (2004) also review several techniques that can be used for recognizing specific structures in 3D lidar point clouds. Wehr et al. (1999) give an introduction about various topics related to laser scanning and also principles of airborne laser scanning. Then, they explain different scanning mechanisms, integration with GPS/INS and data processing of laser scanned data Road extraction from 3D laser scanned data Road environment was one of areas which were extracted using lidar data. For instance, automatic road extraction from dense urban area was done by Pagounis et al. (2007) for improving roads safety using terrestrial laser scanned data. Tiwari et al. (2009) pursued a semiautomatic approach to do so with satellite images and airborne laser altimetry. Hu et al. (2004) did the extraction of roads from urban area using lidar data integrated with high resolution images. In addition, Pu et al. (2011) developed an automated method for recognizing and extracting structures of road inventories. He started by a rough classification to three classes as ground surface, objects on ground and objects off ground. He considered characteristics of features like position, orientation, shape, etc. as well as their topological relations like intersect and angle. Then he applied a hybrid method of data driven and model driven to recognize objects in a road environment. Finally he evaluated his method by visual performance analysis for completeness and correctness and recommended to use data with high point density, reflectance data and integrating lidar data with images Building extraction from 3D laser scanned data Extraction of building is one of popular topics of previous years. For example, Pu et al. (2009) presented a knowledge based reconstruction of building facade models from terrestrial laser scanned data. The method works on a segmented point cloud which takes advantage of some knowledge based features of segments e.g. size, position, orientation, topology and point density. Disaster management and damage 5

15 detection are other reasons to extract information from buildings which was done by Armesto-González et al. (2010) and Dash et al. (2004) using terrestrial laser scanned data Power line extraction from 3D laser scanned data Jwa et al. (2009) developed an automatic method to reconstruct 3D power lines using airborne lidar data. He applies three constraints: height of transmission line above terrain, direction of power line and the location of towers. In this paper, Hough Transform algorithm is also used to detect linear features. This research was followed by Jwa et al. (2010) to improve the results of previous methods which used airborne lidar data as well Object extraction in an urban area from 3D laser scanned data Charaniya et al. (2004) did a point-based supervised classification by aerial lidar data to classify points to 4 classes of trees, grass, roads and roofs. To do this, he used the integration of aerial lidar data with aerial images and applied constraints of height information, height variation, multiple returns and intensity data of laser data and luminance of images. At the end, he concludes that which constraints are useful to classify which objects e.g. height feature is an important classifier for terrain and height variation is important for classification of high vegetation. Bretar et al. (2004) modelled terrain by classification of airborne laser data. He classified points as two classes of terrain and non-terrain. He followed a multistep method in which points are classified as terrain and non-terrain iteratively until the last step. He used the height feature of points and also he inspected the data locally i.e. inspecting the points in their neighbourhood. At the end, he concludes that neighbourhood size and point cloud density are very dependent on each other. Using terrestrial laser scanning for 3D modelling of tree structures is another example in this field (Rosell et al., 2009; Van der Zande et al., 2006). Nardinocchi et al. (2003) also presented a strategy for classification of lidar data as terrain, building and vegetation. Antonarakis et al. (2008) proposed a supervised object based approach to classify different land covers using airborne lidar data without the need to manipulate multispectral images. Vosselman et al. (2005) discuss the possibility of using airborne laser scanned data for two purposes: change detection and semi-automated 3D mapping of urban environment. Furthermore, Sithole et al. (2006) detected bridges in airborne laser scanned data. To do so, they proposed a geometric algorithm based on the idea that bridges can be detected in a landscape based on their cross sections (profiles). They used several dataset and concluded that the method works well for all datasets Object extraction in a railway environment from 3D laser scanned data Morgan (2009) described the 3D lidar data acquisition in a railway environment. A laser scanner is mounted on a railway speeder and scans the railway environment. Then he shows that the achieved data meets survey grade criteria by providing enough details and accuracy to extract essential spatial information of engineering, maintenance and construction applications. 6

16 Moreover, Jeon (2009) extracted and modelled railroad wires from airborne laser scanned data. He applied two different methods to extract different wires. For contact wires, he applied tracking algorithm and for catenary wires he used RANSAC algorithm. Finally he validated his methods by visual inspection and concluded that his method managed to extract and model wires completely and accurately. Gedrange et al. (2011) did a knowledge based extraction of catenary wire masts in which he used the airborne laser data integrated with images. First by doing object based image analysis, he automatically extracted railroad track centre lines. Then by applying some spatial constraints like distance, area, height and intersection, he extracted support tubes of catenary wire masts and finally catenary wire masts. He assessed his methods by counting the number of true and wrong extraction of catenary wire masts as well as the number of them which were not extracted. Haasnoot (2001) explains the advantages of using airborne lidar data for corridor mapping. He focuses specifically on laser scanning systems mounted on a helicopter and integrated with images. He starts with an introduction of airborne laser scanning systems, aerial cameras, helicopters, etc. Then he describes the different steps of the data acquisition process and finally specifications of derived lidar data. Beger et al. (2011) did a data fusion of extremely high resolution aerial imagery and lidar data to combine the advantages of both datasets for automatic extraction of railroad track centrelines. He developed a three step strategy starting by object based image analysis to derive a railroad track mask from ortho imagery. He followed by classifying rail tracks laser points and then finding the rail tracks centre lines from classified laser points by RANSAC algorithm. Neubert et al. (2008) extracted some objects in a railway environment using very high resolution helicopter borne lidar data and ortho image data. He proposed two methods. RANSAC algorithm for linear object extraction and a knowledge based classification method for the rest of object. 7

17 8 AUTOMATIC EXTRACTION OF OBJECTS FROM LIDAR POINT CLOUDS IN A RAILWAY ENVIRONMENT

18 3. MATERIALS For this research two dataset were used i.e. one terrestrial and one airborne. As mentioned in section 1.2, the result of this research which is classified point clouds was used for 3D modelling by Diaz Benito (To be published in March 2012). The main dataset on which the 3D modelling was done is terrestrial data. On the other hand, the developed algorithm required to be checked whether they can classify points on both straight and curved rails but the terrestrial dataset only include straight rails. So the airborne dataset was used to check the capability of algorithm to classify points on curved rails. Furthermore, by using the airborne dataset, the performance of the algorithm for another dataset with lower resolution and less precise coordinates than that of terrestrial dataset was tested First dataset: terrestrial laser scanned data The data for this research is 3D lidar point clouds of a railway environment which is provided by a Dutch railway company called Movares. The point cloud is obtained by registering several laser strips and each point has 3D coordinates in the Dutch national coordinate system while the precision of the coordinates is in millimetres. The data covers a distance of about 670 meters of the railway between the cities Arnhem and Nijmegen and near the city Elst in The Netherlands. It includes two pairs of straight rails, two contact wires, two catenary wires, ten towers with a distance of about 67 meters from each other as well as points on all other objects in a railway environment e.g. railway stations, trees, people, roads and so forth. Although it is cleaned off noises like air dust and humidity, there are still small amount of noises which needs to be taken care of. Figure 3-1 shows the terrestrial dataset from an oblique view. For its views in Google maps and Google street views see Appendix 2. Figure 3-1. An oblique view of the terrestrial dataset of a railway environment in The Netherlands 9

19 The data was acquired by a mobile terrestrial laser scanner and its resolution, point density, is not homogeneous. For instance, the resolution in the middle of the track bed is much higher than that in the outer parts of track bed. That is so due to scanning pattern of data acquisition. As a result, the resolution close to inner rails is higher than the area close to outer rails. The average point density on track bed is 130 points in 1 square meter area on track bed. In addition, there are some gap areas in the data due to shadow effect of the outer rails. Shadow effect happens once the body of the rail prevents the laser beams to reach the other side of the rail and this leads to a gap of points just next to the outer rails. In Figure 3-2, two white arrows point to two gap areas caused by shadow effect of the outer rails. Figure 3-2. Two gap areas caused by shadow effect of the outer rails 3.2. Second dataset: airborne laser scanned data The airborne dataset like the terrestrial one is 3D lidar point clouds of a railway environment which is provided by Earth Observation Science Department of ITC Faculty, University of Twente, The Netherlands. It also registered i.e. its points have 3D coordinates in Dutch national coordinate system and the precision of coordinates is in centimetres. This dataset represents a more complicated configuration of a railway environment than that in terrestrial dataset due to the existence of more number of rails and wires, curved rails and wires which connect other rails and wires, train wagon. It covers a distance of about 80 meters of a railway environment near the central train station of the city Enschede, The Netherlands. In its point clouds, there are points only on objects in a railway environment i.e. two pairs of straight rails, four pairs of curved rails, seven contact wires, seven catenary wires, five towers and a train carriage and there are no points on objects in the surroundings of railway environment like trees, people and train station. For its views in Google maps see Appendix 3. 10

20 This dataset is acquired by airborne laser scanning and its resolution is much lower than that of terrestrial dataset. Its point density is also not homogeneous and its resolution varies in different part of dataset. The average point density on track bed is 52 points in 1 square meter area on track bed. Due to its low resolution and existence of train carriages on rails, there are a lot of gap areas in which there are no points on rails or wires. Figure 3-3 shows the front view of the airborne dataset in which the big white arrow points to a train carriage standing on rails and two small grey arrows point to two towers. Wires are on top of the image and rails are evident by white lines on the track bed. Figure 3-3. Front View of the airborne laser scanned data 3.3. Softwares used for programming and visualization Python and MATLAB were the two programming languages used for this research. MATLAB was mainly used due to its useful tools for image processing. Moreover, the software PCM was extensively used for the purpose of data visualization. PCM stands for Point Cloud Mapper and is developed by Prof. George Vosselman in Earth Observation and Science Department of ITC Faculty, University of Twente, The Netherlands. 11

21 12 AUTOMATIC EXTRACTION OF OBJECTS FROM LIDAR POINT CLOUDS IN A RAILWAY ENVIRONMENT

22 4. METHODOLOGY AND IMPLEMENTATION OF 3D LIDAR DATA CLASSIFICATION 4.1. General approach The proposed approach is a knowledge based classification which takes advantage of our knowledge of the railway environment to classify points on desirable objects. Also as data contains only elevation information and no reflectance information, the algorithm requires to utilize the spatial relation between objects. To do the classification, first the characteristics of the objects of interest are inspected. Then for each object, based on its deemed characteristics, constraints are defined. The defined constraints should be helpful to differentiate the objects of interest from other objects and from each other. In addition, constraints should be applied efficiently as defining a few weak constraints leads to a poor classification and using too many constraints make the process complicated and slow and it does not necessarily improve the results. The main dataset is the terrestrial one on which 3D modelling was done by Diaz Benito (To be published in March 2012). The terrestrial dataset represents a simple configuration of a railway environment including only two pairs of straight rails. While the airborne dataset represents a complicated configuration of a railway environment including four pairs of curved rails. So the airborne dataset was used to check the ability of the algorithm for classification of curved rails in a low resolution and low precision dataset. For this research, two different methods i.e. Template matching (TM) and Region Growing (RG) were developed. Both methods were applied for terrestrial dataset and based on achieved results, it was concluded that RG was more efficient and faster than TM. Therefore, only RG method was applied for airborne dataset to check its performance for more complicated railway environment settings. In terrestrial dataset, different sections of dataset are almost the same in the sense that they all have same objects. The only difference is the section which includes an intersection of railway and a road. Although both TM and RG methods could be run for the whole dataset, they were run for two different sections of about 100 meters long as two subsets of the whole dataset. For a closer view of the parts of terrestrial dataset in which TM and RG methods were applied see Appendix 4. TM method could be applied in any section of dataset but it was applied in the hardest section for classification due to the existence of an intersection of railway and a road. If it manages to classify points in that section, then it will succeed to do the classification in other sections as well. On the other hand, RG method first finds a short piece of rails and wires and then finds the rest of rails and wires by following them in one direction. So it required to be applied at the beginning of dataset because if it is applied in the middle of dataset, it should follow rails and wires in two directions which is not able to do it. The result of both methods is a point cloud which its points are classified as four classes of: 1. Rails 2. Contact wires 3. Catenary wires 4. Other objects 13

23 4.2. Characteristics of objects of interest Characteristics of rails The characteristics of rails that were used for their classification: 1. Rails lie on the track bed. 2. Rails are considered as linear objects that for a short length like 5 meters direction can be defined. 3. Height of rails is 10 cm so that the points on top of rails are 10 cm higher than points on track bed. 4. The width of rails is 7 cm. 5. Gauge, the distance between two tracks of rails, is about 1.5 meters. 6. There is a limitation on slope of rails. The maximum slope was considered 5% so the height of certain length of a piece of rail does not exceed 5% of its length. 7. There is a limitation on maximum curvature of rails. The maximum displacement of the end of a 1 meter piece of rail is considered 5cm so the direction of a certain length of rail does not exceed a limit Characteristics of contact wires The characteristics of contact wires that were used for their classification: 1. Contact wires are in an elevation of meters above rails. 2. In top view, as if all points on rails and contact wires are projected on a planimetric plane, contact wires lie between two rail tracks. Two characteristics mentioned above are about the relative position of contact wires to rails which is part of railway environment specifications. These specifications are the same for all other parts of Dutch railway environment Characteristics of catenary wires The characteristics of catenary wires that were used for their classification: 1. Catenary wires are in an elevation of more than 5.5 meters above rails. 2. In top view, as if all points on rails and catenary wires are projected on a planimetric plane, catenary wires lie between two rail tracks. Two characteristics mentioned above are about the relative position of catenary wires to rails which is part of railway environment specifications. These specifications are the same for all other parts of Dutch railway environment Applied constraints Based on railway environment specifications and particularly characteristics of rails and wires, the following constraints were applied for classification of rails and wires: 1. Relative position of wires to rails Relative height of wires above rails Contact wires: meters above rails Catenary wires: more than 5 meters above rails Planimetric position of wires with respect to rails In top view, both contact and catenary wires lie between two rail tracks. 14

24 2. Rail height: 10 cm 3. Rail width: 7 cm 4. Gauge: 1.5 m 5. Maximum slope of rails: 5% 6. Maximum curvature of rails: 5 cm curvature in each 10 m of rails Template Matching method The general approach is to classify the objects of interest in several steps i.e. first the algorithm does a rough classification of points on the objects of interest. Then it improves the classification results in next steps by excluding the wrongly classified points and including the desirable points which were not classified in previous steps. Figure 4-1 indicates the flowchart of TM method. Figure 4-1. Flowchart of Template Matching method 15

25 First classification of rails Pre classification First a rough classification is done based on the height of points. This leads to the classification of points as 4 classes: 1. Class 1: contains all points on the track bed and rails. It is assumed that the most common elevation in the dataset represents the elevation of track bed. This assumption was made due to higher point density of track bed in comparison to other objects. Besides, due to slope of the terrain, the elevation of track bed may change. So to include all points on rails and track bed in class 1, all points with elevation 0.5 m more or less than the most common elevation in the dataset are classified as class1. Maximum Track bed height= the most common elevation meter Minimum Track bed height= the most common elevation meter 2. Class 2: contains all points with the maximum elevation 5 meters above track bed. Class 2 does not include any points on the objects of interest. So all points with a maximum elevation of 5 meters above maximum track bed height are classified as class Class 3: contains all points with the elevation meters above track bed including points on contact wires. So all points with an elevation between 5 to 5.5 meters above maximum track bed height are classified as class Class 4: contains all points with the elevation more than 5.5 meters above track bed including the points on catenary wires. So all points with an elevation more than 5.5 meters above maximum track bed height are classified as class 4. As a result, rail points should be detected out of class 1, contact wire points out of class 3 and catenary wire points out of class 4 and all points in class 2 are undesirable for us. Relative heights of contact and catenary wires to rails are constant (5-5.5 m and more than 5.5 m above rails respectively) due to railway environment specification of The Netherlands Grid generation As mentioned in section , class 1 contains all rail points so to classify rail points only points of class 1 are processed. The main idea is to inspect the data locally and this grid facilitates the inspection of data in details and locally. So to implement, a 2D grid with cell size of 50 by 50 cm 2 was generated. In this step, cell size is very influential on results and is highly dependent on data resolution, point density. A smaller cell size would result in a cell with small number of points which decrease the reliability of results and a bigger cell size can lead to cells containing two rails or rails with other objects which makes the analysis more 16

26 complicated. In fact, it is preferred that each cell contains maximum of one rail as it makes the process more efficient. Direction is a characteristic of rails that can be used if there is just one piece of rail in a cell. As the grid is 2D, the location of each cell is indicated by two elements i.e. row and column. So to identify a cell in which a point falls, two elements of row and column should be determined by following steps: 1. Finding minimum of Xs and Ys of all points in class 1: MinX and MinY. 2. Determining that each point lies in which cell in a way described below: a. For a point with coordinates: P(X,Y,Z). b. For a grid cell size of 0.5 m. c. Row = ((X-MinX)/0.5)+1 d. Column = ((Y-MinY)/0.5)+1 e. Row and Column indicate the location of the cell in which the point P falls in. 3. Continue the above steps for all points of class 1. In this way, based on planimetric position of points, their X and Y coordinates, it was determined each point falls in which cell. Then, further analysis was done on all points which lie in one cell. The idea is to have enough number of points in each cell so that they can represent the characteristics of objects in that cell which leads to reliable statistical analysis results. By an inspection of cells containing different number of points, it was found that the ones having less than 30 points do not represent true information about the characteristics of objects inside. Therefore, just cells with more than 30 points inside were considered for further analysis Height jump detection The height jump detection (HJD) algorithm uses the fact that the height difference between the points on top of the rail and the points on the track bed is about 10 cm which is the rail height. The main idea is that the rails cause a height jump on the track bed due to their heights. In Figure 4-2, grey arrows show rails which cause height jumps and the white arrow shows another part of track bed without any height jumps. Figure 4-2. Grey arrows point to height jumps caused by rails while the white arrow points to other parts of track bed without any height jumps 17

27 The HJD method was applied to detect rail points and as only class 1 contains points on rails, only points of class 1 were processed for this stage and the strategy was: 1. To determine if a cell contains rail points or not by inspecting: a. Whether there is a height jump in a cell or not. b. If the size of the height jump is the same as the one caused by rails. 2. To differentiate the points on rail tracks from the points on track bed in a cell containing rails. To determine if there is a height jump in a cell, the height difference between the highest and lowest points in a cell is inspected. If there is a rail in a cell, the height difference between the highest and lowest points should be at least 10 cm which is the rail height. But as the data contains noise, usually the lowest and highest points in each cell do not represent the rail and track bed respectively. This is evident in Figure 4-3 in which rail points are indicated by grey colour and track bed points by white colour and 10, 80 and 90 percentile elevations represent the elevations of track bed, bottom of rail and top of rail respectively. Figure 4-3. Using 10, 80 and 90 percentiles to remove noises for implementation of height jump detection method Moreover, the size of the height jump needs to be checked. Besides rails, there are some other objects that cause height jumps like signs, electrical devices, bumps on track bed and the short walls where the railway stations and track bed intersect (Figure 4-4). So to differentiate between these objects from rails, the algorithm inspects the size of height jump. In other words, the size of height jump determines if it is a piece of rail or not. For instance, the short walls of intersection of track bed and railway stations and some signs make a bigger height jump while electrical devices and bumps on track bed make a smaller height jumps than that of caused by rails. 18

28 Therefore, with respect to the rail height, the algorithm considers only the height jumps in the range of cm as rails and ignores the rest. In Figure 4-4, two arrows on left indicate the height jumps caused by rails and the one on the right point to height jump caused by railway station. Figure 4-4. Big height jump caused by railway station platform V.S. small height jumps caused by rails As a result, the constraint is defined as: If a cell contains a piece of rail, the height difference between the 10 and 90 percentile elevations of its points is more than 10 cm and less than 25 cm: 10 (centimetres) < (90%Elevation) (10% Elevation) < 25 (centimetres) Then for the cells which are considered to have rail points, the points on rails needs to be differentiated from the ones on track bed. To do this, the points higher than 80 percentile elevation of all points in that cell were considered as points on rails (Figure 4-3). One should pay attention that this is the first classification of rail points and some wrong classifications and some classification gaps are expected which will be taken care of in the next steps First classification of wires To classify the points on wires, three characteristics of wires were considered: 1. Contact wires are in elevation of m above rails (Figure 4-5). 2. Catenary wires are in elevation of more than 5.5 m above rails (Figure 4-5). 19

29 3. In top view, as if all rail and wire points are projected on a planimetric plane, both contact and catenary wire points lie between rail points (Figure 4-6). Figure 4-6. In top view, wires shown by yellow and orange colour lie between rails shown by green colour Figure 4-5. Relative height of wires to rails To this stage, some rail points are classified in previous step and all wire points are in class 3 and 4 of pre classification based on height. So if all classified rail points and all points in class 3 and 4 are projected on a planimetric plane, out of all points in class 3 and 4, only points on wires lie in a close neighbourhood of classified rail points. To implement, all projected classified rail points were put in one KD tree data structure and all projected points of class 3 and 4 were put in another KD tree data structure. Then by doing the nearest neighbour analysis, only points of class 3 and 4 which were in a close neighbourhood of rail points were considered as contact and catenary wire points respectively Excluding wrongly classified rail points It should be noticed that the first classification of wire points is dependent on the results of first classification of rail points. That is so because the first classification of wire points is done based on nearest neighbour analysis between points of class 3 and 4 and the classified rail points in previous step. As in the results of first classification of rail points, some wrong classifications and some classification gaps are expected, the results of first classification of wires are expected to have some wrong classifications and some classification gaps as well. 20

30 The idea of TM was to seek the rail pattern in the classified points of previous steps to detect only truly rail points and exclude the wrongly classified ones. This was done by considering the spatial relation of wires to rails. In fact, there are three points that should be noticed: 1. The purpose of this step is to exclude wrong classification of just rail points and not wire points. 2. TM was aimed to exclude wrong classifications and not aimed to include any more rail or wire points which were not classified in previous steps. Classification of the rest of rail and wire points which were not classified in previous steps is done in next steps described in sections and The input of this step is the classified rail and wire points which is obtained in previous steps and the output is only true rail points. So the point cloud is converted to a greyscale image and a kernel based on the defined rail pattern is generated. The rail pattern is defined based on the characteristics of rails and contact wires, mentioned in section 4.2, and contains: 1. Two straight rails: a. 1 meter long b. Distance between the axis of rail tracks:1.4 m c. Width of rail tracks: 20 cm 2. One contact wire: a. In a certain elevation above rail tracks b. In top view, its planimetric position lies between rail tracks c. Width of contact wire: 30 cm 3. A black area in surroundings of rails Then to find rail patterns, the kernel is moved through the greyscale image with the following strategy: 1. Move the kernel through the first 5 meters of rails to get the main direction of rails. 2. Move the kernel through whole greyscale image every 5 rows and in each row every 3 cells to find some of rail patterns as a starting point for the next step. 3. Continue searching for rail pattern from the detected rail patterns in previous step. a. Continue the search for rail pattern just in the neighbourhood of its two ends and follow them to find the rest of rail patterns. b. Update the rails direction every 1 meter after each detection of rail pattern. The strategy explained above is an optimized one as first it commences the rail pattern seeking with 5 rows and 3 columns steps. Then, it continues the search in the neighbourhood of detected patterns and follows them to detect the rest of rails. This algorithm is more efficient and faster than searching for rail pattern in each cell of greyscale image one by one. The result up to this section would be a greyscale image in which most of the wrongly classified rail points are excluded. However, some wrongly recognized rail patterns are still expected which are mostly scattered in different parts of image and not connected to true rails. Therefore, to remove them, connected 21