Image Analysis of Ecotoxicological Experiments - Count and Classification of Collembola. Group 823

Transcription

1 Image Analysis of Ecotoxicological Experiments - Count and Classification of Collembola 1st September 2004

2 AALBORG UNIVERSITY Institute of Electronic Systems Title: Image Analysis of Ecotoxicological Experiments - Count and Classification of Collembola Theme: Interpretation and Presentation of Digital Image Informations Time Period: February 2nd - June 2nd 2004 Term: CVG Group: 823 Abstract: Members: Morten F. Christensen Rasmus Corlin Jakob Kirkegaard Supervisor: Henning Nielsen Number Printed: 6 Report Pages: 108 Appendix Pages: 31 Total Page Count: 139 Ended: June 2nd 2004 This project deals with the development of an image analysis system for automatic identification and counting of insects in ecotoxicological experiments. Methods are analysed for preprocessing and segmentation of images obtained from an ecotoxicological experiment. The preprocessing locates the region of interest based on histogram analysis, while the segmentation separates the insects from the background through the use of optimal thresholding. Afterwards features are extracted from the located objects, after which clusters of insects are separated using morphological methods and Hough transformation. Finally the located objects are classified using unsupervised fuzzy c-means clustering. The system has been implemented in Java by use of the image processing framework ImageJ, which through being open source and extendable forms a good platform for further development. The project succeeded in identifying and grouping the insects from a number of test images obtained in a controlled environment. The overall summary of tests performed on each of the modules showed that the system was capable of solving the given problem. Further investigations and tests are however necessary before the system can be used in a real ecotoxicological laboratory.

3 AALBORG UNIVERSITET Institut for Elektroniske Systemer Titel: Image Analysis of Ecotoxicological Experiments - Count and Classification of Collembola Tema: Fortolkning og præsentation af digital billedinformation Projektperiode: 2. februar - 2. juni 2004 Semester: CVG Gruppe: 823 Synopsis: Gruppemedlemmer: Morten F. Christensen Rasmus Corlin Jakob Kirkegaard Vejleder: Henning Nielsen Oplagstal: 6 Rapportens sideantal: 108 Appendiks sideantal: 31 Total sideantal: 139 Afsluttet: 2. juni 2004 Dette projekt beskæftiger sig med udviklingen af et billedanalyse system til automatisk identifikation og optælling af insekter i økotoksikologiske forsøg. I projektet analyseres metoder til præprocessering og segmentering af billeder opsamlet i forbindelse med et økotoksikologisk forsøg. Præprocesseringen lokaliserer udfra histogram analyse den relevante region i billedet. Den efterfølgende segmentering adskiller insekterne fra baggrunden udfra optimal thresholding. Derefter udtrækkes features for de lokaliserede objekter, hvorefter grupper af insekter bliver adskilt ved brug af morfologiske metoder samt Hough transformation. Endelig klassificeres de fundne objekter i et antal grupper ved hjælp af ikke-superviseret fuzzy c-means mønstergenkendelse. Systemet er blevet implementeret i Java ved brug af billedbehandlings frameworket ImageJ, som pga. åbenhed og udvidbarhed er et godt udgangspunkt for videre udvikling. I projektet et det lykkedes at identificere og gruppere insekter udfra en række test billeder opsamlet under kontrollerede forhold. Overordnet set viste test af de enkelte moduler, at systemet var i stand til håndtere den stillede opgave. Yderligere undersøgelser og test er dog påkrævet, såfremt systemet skal benyttes i et virkeligt økotoksikologisk laboratorie.

4 2

5 Preface This project has been prepared by group 823 at Laboratory of Computer Vision and Media Technology, Department of Health Science and Technology at Aalborg University. The project has been composed during the 8th semester, which extends from February 2nd to June 2nd, The purpose of the project is to apply the obtained technical knowledge from the project courses in the problem oriented and project organised learning form. The target group of the project is students and other interested with a technical insight corresponding to that of the group. The purpose and content of the project is specified in the study regulations for the semester, which are outlined below. to be able to perform synthesis of theories, methods and techniques for advanced processing and handling of digital video images. to be able to understand and apply theories, methods and techniques for generation of 3D computer graphics to be able to apply theories, methods and techniques for digital image analysis and computer graphics in designing information processing systems based on concrete problems In the report all literature references have been composed with author and year in square brackets e.g. [Gonzalez and Woods, 2001]. The position of the reference determines which specific part the reference relates to. If the reference is positioned before a period, it relates to the previous sentence, otherwise if the reference is positioned after a period the reference related to the previous section. Images, source code and other project related files are located on the attached CD-ROM. References to the CD-ROM are written as ( description of the content path/to/content). Aalborg, June 2nd, 2004 Morten F. Christensen Rasmus Corlin Jakob Kirkegaard 3

6 4

7 Contents 1 Introduction Image Analysis in the Life Sciences Ecotoxicology Problem Analysis Analysis of the Experiment Working Procedures Previous Work Folsomia Candida Specification of Demands References User Profile Specific Requirements Requirements to the Course of Development External Interface Requirements Requirements to the Performance of the Application Quality Factors Application Analysis Application Responsibilities Problem Formulation Delimitation Readers Guide Preprocessing Introduction Defining and Constraining the Setup Analysis of Images Colour Space Histogram Analysis Locating the Region of Interest Algorithm and Pseudo Code Region of Interest Fitting Enhancing the Region of Interest Test of Preprocessing Partial Conclusion CONTENTS 5

8 6 CONTENTS 4 Segmentation Introduction Methods Thresholding Techniques Region Based Methods Test of Segmentation Partial Conclusion Representation and Description Introduction Prior Knowledge Methods Labelling Stabilising Features Feature Space Test of Representation and Description Test for Separability Test for Correlation Discussion Partial Conclusion Evaluation of Objects Introduction Identification of Insect Clusters Convex Hull Skeletonising Distance Map Evaluation of Methods Separation of Insect Clusters Hough Transformation Estimating Ellipse Parameters Finding Candidate Ellipses Filtering Candidate Ellipses Finding Best Candidate Test of Object Evaluation Test of Identification of Insect Clusters Test of Counting Insects in Insect Clusters Test of Separation of Insect Clusters

9 CONTENTS Partial Conclusion Object Recognition Introduction Basic Classification Toxicology Methods Clustering Techniques K-Means Clustering Fuzzy C-Means Clustering Validity Index Test of Object Recognition Partial Conclusion Evaluation Conclusion Perspective of the Application Appendix 109 A Minutes 111 A.1 Minute of meeting 23/ A.1.1 Agenda A.1.2 Janeck Scott-Fordsmand Background A.1.3 Test Procedure A.1.4 Previous Applications A.1.5 Collembolas A.1.6 Problem Domain A.2 Visit at DMU in Silkeborg 2nd April A.2.1 Institute A.2.2 Different Procedures A.2.3 Collembola and other insects A.2.4 Other B User Requirement Document 117 B.1 Introduction B.1.1 References B.2 General Description

10 8 CONTENTS B.2.1 System Description B.2.2 Program Functionality B.2.3 User Profile B.2.4 Requirements to the Course of Development B.2.5 Conditions B.3 Specific Requirements B.3.1 Functional Requirements B.4 External Interface Requirements B.4.1 User Interface B.4.2 Hardware Interface B.4.3 Communication Interface B.4.4 Software Interface B.5 Requirements to the Performance of the Program B.6 Quality Factors B.6.1 Reliability B.6.2 Maintenance B.6.3 Ease of Further Development B.6.4 User Friendliness B.6.5 Re-usability B.6.6 Efficiency B.7 Part Delivery C Collection of Test Data 125 D Output 127 D.1 Output fields D.1.1 File header D.1.2 The given features per found insect/object E Principal Component Analysis 129 E.1 General Principle E.2 Calculation of the Covariance Matrix E.3 Calculation of Eigen Values and Eigen Vectors E.4 Selection of Eigen Vectors E.4.1 M-method E.4.2 J-measure E.4.3 SEPCOR E.5 Mapping of Data

11 CONTENTS 9 F Correlation Coefficient Tables 135 G Results of M-method 139

12 10 CONTENTS

13 Chapter 1 Introduction This chapter provides an introduction to the topics discussed throughout the project. After a general survey of the usage of computer vision in various biological and medical applications, a more specific description of the problem is given. Finally, the initiating problem for the project is stated. 1.1 Image Analysis in the Life Sciences. The human visual system is able to collect huge amounts of data in a short period of time, filter these informations and quickly make an interpretation of the scene or object in the field of vision. Although, still being a great leap behind humans, artificial vision systems have undergone a tremendous development in the last three decades. The progress is partly due to the advances in hardware, where computer systems with larger capacity for storing data and more powerful processors for faster data processing have emerged. Together with improvements of existing signal processing algorithms and the appearance of new image processing techniques, the artificial vision systems have found their use in many new areas - among these in biological and medical applications. The general situation when using an artificial vision system in a given application area is to assist or replace an expert performing some task. This can either be because the task at hand is time consuming and tiresome for a human (e.g. inspection of products on a conveyor belt), or because the vision system is able to perform much better and with lower error rates than a human inspector (e.g. identifying blood vessels in an x-ray image). In the following a few examples are given where computer vision systems is utilised on biological or medical related application areas. Microbiology. Analysis of results from biological experiments may require the identification of different cell types or colonies of cells in images. These images can either be obtained from a regular camera or a microscope. Computer vision systems have been applied to this field and many companies specialises in developing equipment for this particular field. [Halcon, 2004] An example of this is shown in figure 1.1(a). Medicine. Computer vision systems are used in many areas of the medicine and health care sector, e.g. in X-ray imaging, where blood vessels and chambers of the heart have to be identified based on some special fluid (contrast medium) that is injected prior to the examination. Image analysis has been applied to these X-ray images with the purpose of identifying coronary disease and strictures of the blood vessels. [Dhawan, 2003] An example of this is shown in figure 1.1(b). 11

14 12 Introduction Ecology Remote monitoring of cities, forests, rivers, and other areas of the earth, is an application field, where image analysis has been widely used. Satellite images of e.g. forests are analysed with an image processor to determine levels of pollution or distribution of different kinds of plants or trees. [Halcon, 2004] An example of this is shown in figure 1.1(c). (a) Identification of particles, e.g. used in identification of malignant cells. (b) Identification of blood vessels from an angiography. (c) Identification of trees and bushes. Used for remote monitoring of pollution. Figure 1.1: Different examples of the usage of computer vision systems in medical and biological application areas. [Halcon, 2004] This project deals with image analysis in ecological toxicology - more specifically automated identification and classification of different insects, which is used in formulating risk assessments of different chemicals. The field of ecological toxicology is described in greater detail in the following, leading to the initiating problem of the project. 1.2 Ecotoxicology The study and classification of toxic substances (toxicology) was systematised in the beginning of the 19th century. Because many substances are known to be poisonous to life (whether plant, animal or microbial), toxicology is a broad field, spread among biochemistry, histology, pharmacology, pathology and many other disciplines. In the ecological sciences, toxicologists play a part in the identification and elimination of environmental contaminants. Ecotoxicology is, more precisely, the science devoted to the study of the production of harmful effects by substances entering the natural environment, especially effects on populations, communities and ecosystems. An essential part of ecotoxicology is the assessment of movement of potentially toxic substances through environmental compartments and through food webs. [Britannica, 2004], [Duffus, 2004] One particular field of ecotoxicology is the specification and performance of bio tests. The basic principle in these tests is exposing some specified organism to a given chemical. Based on the reaction of the organism, the toxicity of the chemical is determined. With the increasing national and international legislation of environment and soil protection, ecotoxicological bio tests with soil organisms have become more important. The ISO standard Soil quality - Inhibition of reproduction of Collembola (Folsomia Candida) by soil pollutants (ISO 1999) sets the international standard for the test of chemicals. [Lemnatec, 2004] The Danish National Environmental Research Institute, Department of Terrestrial Ecology (DMU), functions as a supervisor for Organisation for Economic Cooperation and Development

15 Ecotoxicology 13 (OECD) concerning pesticides. This involves specifying and performing ecotoxicological bio tests as well as formulating risk assessments for different chemicals. The test procedures involve counting a large number of insects located in a petri dish, which done manually is error prone and adds uncertainty to the experiment. This project deals with creating an image analysis system to assist performing bio test on soil organisms. This leads to the initiating problem of the project, which is stated in the following. How is an image analysis system created that is capable of identifying and classifying insect species in a given bio test, thereby favouring more effective work procedures and less uncertain results?

16 14 Introduction

17 Chapter 2 Problem Analysis The previous chapter introduced the problem domain of this project, i.e. the automated identification and classification of insect species in a given bio test. The purpose of this chapter is to analyse the problem more thoroughly concerning both how the experiments are conducted and previous applications for solving the problem. 2.1 Analysis of the Experiment The purpose of this section is to describe the ecotoxicological experiments performed at DMU. The information is based on a meeting with a DMU representative and a visit at DMU in Silkeborg, Denmark (see appendices A.1 and A.2 for minutes of the meetings) Working Procedures The bio test procedure performed at DMU utilises a group of insects called collembola. An example of a collembola is shown in figure 2.1. Collembolas represent the largest order in the group of insects. They are small, primarily wingless insects, characterised by a typical jump fork (furca) on the underside of their belly. Using the furca they are able to jump, covering comparatively large distances. Collembolas live in almost every habitat. The majority of the collembola species live in the top layers of soil, waste-fauna or water. [Lemnatec, 2004] Figure 2.1: Photograph of a collembola. [Meyer, 2004]. The test performed at DMU procedure uses a statistical approach where chemicals in varying concentrations are tested on different samples of collembola both in water and soil to collect information about the toxicity of the chemicals. A schematic showing the principle of the experiment is shown in figure 2.2. The experiment consists of 24 samples, which initially are supplied with the given chemical in varying concentrations (exp 0... exp 6 ). The column of samples with concentration exp 0 is not exposed to the chemical and is used as a control group for the experiment. 15

18 16 Problem Analysis exp 0 exp 1 exp 2 exp 3 exp 4 exp 5 Exposure grp 0 grp 1 grp 2 grp 3 Groups Figure 2.2: The experiment is partitioned into groups, each with varying concentrations of chemical being tested. After the addition of the chemical each sample is supplied with 20 collembola (10 males and 10 females), all of which are hatched out using a specific procedure to obtain an age between 19 and 23 days. This procedure facilitates the separation and classification of the collembola after a period of three weeks. During the experiment the insects breed and about 600 individuals are present after the three weeks. Due to the synchronisation procedure it is possible to identify four groups: two sizes of juveniles, one group of males and one group of females. Heat Figure 2.3: Obtaining the insects. To count the collembola in the soil the container is heated from the surface, which makes the collembola alive leave the soil and drop down in a petri dish. This petri dish is filled with a moist layer of plaster mixed with charcoal. This layer keeps the collembola from hiding in small holes as well as keeping them alive due to the moisture. Figure 2.4 shows an image of the collembola lying around in the petri dish on the layer of plaster, after they have been expelled from the soil. The number of surviving collembola and their distribution in the four generation groups are then used as an indicator of the toxicity of the used chemical.

19 Analysis of the Experiment 17 (a) Image of a petri dish with collembola lying around. (b) Zoom area of the image to the left. Figure 2.4: Image of collembola in a petri dish Previous Work To determine how many collembola are present in the petri dish and how they are distributed in the different generation groups an identification and counting has to be performed. This is done based on images of the petri dish, as shown in figure 2.4. If the count is performed manually by laboratory technicians experience dictates that the deviation will be high - e.g. if three different technicians perform the counting the deviance will be close to 30%. Besides being tiresome and time consuming the high deviance renders the process of manually counting the collembola unacceptable and makes the risk assessments less reliable. Another approach previously performed has been to systematically gather a number of images of different areas of the petri dish. These images would be analysed one by one by an image processing application (named DIP), developed specifically for this purpose. The output of the DIP program is a file containing a list of the located objects in the image, together with some describing features (e.g. area and width). The method is further described in [Krogh et al., 1998]. The features given from the DIP program are then further analysed in a program named SAS, where the four generation groups are identified and separated by visual inspection of the feature space. In order to make this inspection easier a transformation of the features are made by use of principal component analysis in order to maximise the variance in these. A description of the principal component analysis can be found in appendix E. Besides the problems of combining the different images of the petri dish, the image processing application also had problems with stability, performance and most importantly is was patented to the developer, making it impossible to extend the program with new features and knowledge. One of the primary problems was the programs insufficient ability to separate and classify

20 18 Problem Analysis collembola lying in clusters. Additionally the procedure needs the users interaction with respect to the statistical identification and separation of the insect groups Folsomia Candida The collembola used in the risk assessment that this project concerns is named Folsomia Candida. In contact with DMU the following characteristics about this insect have been given: The Folsomia Candida has a 2:1 to 5:1 ratio between the length and width, since these tend to have the form of an ellipse. As mentioned when having performed the experiment four groups representing the age and gender of the collembola exist. For the Folsomia Candida the following characteristics are given for these groups: Adult females and males are approximately the same size. Females tend to be wider than the males, which makes these more circular. Juveniles are present in two size distributions. The reflection of the Folsomia Candida is dependent on the four given groups: The colour of the Folsomia Candida changes over age. When juveniles grow and become adults their physical appearance changes. It changes from bright/transparent/- milkwhite to a darker white. Folsomia Candida eat the carbon in the plaster when in lack of their primary food resources (dry yeast), this can be seen on juveniles and males since these are small and transparent. The females on the other hand are too big for the carbon to make an appearance. 2.2 Specification of Demands The previous section described how the experiments are performed at DMU, together with a description of the previous attempts to solve the problem through use of automated vision systems. Based on the previous experiences at DMU with automated solutions for counting, together with their requests for a new system, a user requirement document was written (see appendix B on page 117). This section lists the important requirements stated in the user requirement document References This specification of demands is based on the following references. Correspondence with Janeck Scott-Fordsmand, Department of Terrestrial Ecology at DMU. Minutes of meetings with Janeck Scott-Fordsmand and of a visit at DMU (included in appendix A). Article Automatic counting of collembolans for laboratory experiments. [Krogh et al., 1998].

21 Specification of Demands User Profile The users of the application are mainly trained laboratory technicians and assistants with majors in biology or chemistry with specific knowledge about the problem domain. The users are expected to be familiar with ordinary computer work, though without any knowledge of image analysis Specific Requirements The specific requirements for the project are formulated from problems enlisted by the user from daily activities in the problem domain, together with user requests for the future application. The objective of the development process is to create an application that based on a single digital image of a given bio test experiment, is able to identify, count and classify the insects in the experiment. The following is a list of things influencing the development process. Problem Domain: There are several problems when it comes to recognising collembola. To achieve precision the following have to be taken into consideration: The collembola tend to form small clusters which changes the result of the count. Other objects such as soil and plant material may disturb the segmentation and classification. Mirrored objects in the petri dish should be sorted out from the count. The toxic substances, which the collembola are exposed to, may have different effects on different groups (e.g. surface dependent effects may harm small collembola more). This may complicate the classification. Movement may occur among the collembola so one image showing all collembola should be taken. It should be possible to identify four groups of insects: females, males and two groups of juvenile. The focus of the project should be on identifying the special collembola specie named Folsomia Candida. Work Procedure: A minimum of interaction with the application and an fully automated process would lighten the workload, which is a requirement from the user. This includes any setup of the application, which often leads to fault. Modifications should though be a possibility to make by the user. Add Species: To expand the program in its use it should be possible to add new species either to be included or excluded in the count/classification. Open Source: By making the application as open source it should ease a standardisation of the testing procedure, which could enhance quality not only nationally but also internationally depending on how this project evolves Requirements to the Course of Development Based on the rough list of user requirements stated in section 2.2.3, requirements to the development course is listed in the following.

22 20 Problem Analysis Documentation The documentation of the project should be made with focus on making it possible for others to continue the development of the application, since it cannot be guaranteed that the project is ended this semester. Besides of making a well documented analysis and design it also requires that the implementation is commented in order to allow others to understand how it has been implemented in order to optimise on it or add new features. Implementation To accommodate with the users requirements for an extendable and open implementation, it has been chosen to implement the application in Java by use of the ImageJ framework [ImageJ, 2004]. ImageJ is a framework which incorporates many standard as well as advanced image processing algorithms. It is a framework that is under development so new algorithms are being added continuously. By making the application in the ImageJ framework also allows the project to contribute to the development of ImageJ, if relevant algorithms that is not already available is implemented. Another reason for choosing to make the application in the ImageJ framework is due to its very simple user interface that among others allows the user to record macros of the image processing they are performing. Naturally the application will not be implemented this way, but it allows the users of the application to try making image processing that eventually can be implemented as part of the application without the user having to be an expert in programming External Interface Requirements In the following the general interfaces defining the environment in which the application is going to be used is defined in order to be able to optimise the solution to these circumstances. Software specific interfaces will not be defined in this context. User interface Below is listed the requirements to the user interface that have been defined in co-operation with the user of the application. It should be possible to process one image at a time. It should be possible to make the application process multiple images so that data from all samples from an experiment are loaded into the application at once. It should be possible to specify which species of insects that should be categorised in the experiment in order to narrow down the possible output of the classification. It should be possible to specify which material the insects are in (e.g. soil, water). It should be possible to specify the type and size of the container used for the experiment (e.g. a circular petri dish). Results should be presented in a scheme. Image data during the analysis of images as well as results should be stored such that these are accessible for later use, e.g. to support a later analysis.

23 Specification of Demands 21 All text including error messages should be in English in order to allow the application to be used internationally. Hardware Interface Since the application is being implemented in Java, there are no specific requirements to the hardware or operating system that it is going to be executed on. It is only required that a Java Virtual Machine is available and running on the given operating system Requirements to the Performance of the Application There are no specific requirements to the performance of the application in terms of processing time. Instead it is of greater importance that the precision of the image recognition is as good as possible and hence more complex algorithms that can give a better recognition results are preferred at the cost of longer processing time Quality Factors This section focuses on how different factors are weighted throughout the development (see table 2.1). These factors will help certain criteria to be fulfilled and thereby functions as a guideline for the development process. These criteria can often be set and described by the user, but can also be formed from contact with the user and weighted on a specific development field instead of the program as a whole. The criteria are also set according to the performance requirements described in section Reliability In order to get stability when counting and identifying the insects the recognition should be reliable. The system should be able to get the same result with the same pictures, and not a deviant result with the same as well as with a slight different picture (e.g. taken a few seconds later). The reliability criteria is weighted as very important. Maintenance The maintenance of the program, e.g. in terms of allowing different species of insects to be added to the program, is rated as very important since it makes the program more flexible for the users. This will also allow the program to be used for other similar experiments. Ease of further development The users should be able to make further development of the program since they are given the code after the project ends. Therefore ease of further development is rated as important. It could have been rated as very important, but since their field of expertise lies in a different area, focus lies on different criteria.

24 22 Problem Analysis User friendliness User friendliness is rated as important since the application is expected to be used by users of various background regarding knowledge of using a computer. However knowledge with the specific problem area of the experiment is expected to be high. Therefore the user interface should be easy to use and terms that are common in the specific field of research would be feasible to use. Re-usability The program is very application dependent and will most probably only be used to count and classify collembola. Re-usability is rated as not important. Efficiency Efficiency in terms of execution time of the application is rated as less important on the basis of reliability and maintenance being prioritised higher. Requirement Not Important Less Important Important Very Important Reliability Maintenance Ease of further development User friendliness Re-usability Efficiency Table 2.1: Table describing guidelines for the development process. 2.3 Application Analysis The purpose of this section is to make a preliminary analysis of the application being developed with focus on defining the responsibility of it as well as defining its general areas of responsibility Application Responsibilities Based on the requirements defined in the previous sections and the description of the experiment being analysed the overall responsibilities of the application will be defined. It has been chosen to do this by partitioning the application into smaller modules with well defined responsibilities. By doing this the development of these will be easier and at the same time lay out the lines for a general framework in which each module can be modified or even replaced without affecting the rest of the system. This will also make further development of the system possible without having changes/extensions of the system affect all modules. It has been chosen to partition the application into six modules that is believed to have suitable responsibilities. The modules are shown in figure 2.5. The order of the modules and the arrows

25 Application Analysis 23 between them indicate that every module is dependent on the module(s) to the left of them in the sense that they will need the output from the preceding module as input. Image Acquisition Preprocessing Segmentation Representation and Description Evaluation of Objects Object Recognition Figure 2.5: Application Modules. In the following there will be a brief description of the responsibility of these modules. Image acquisition The responsibility of the image acquisition module is not that comprehensive but nevertheless relevant. It should take care of loading the images that should be processed in the image analysis, either from e.g. a digital camera or from a hard drive, and make them available for the preceding modules. Pre-processing The purpose of the pre-processing module is to get the image into a state where it is suitable for segmentation. This means both defining the region of the image that is of interest as well as enhancing the image with respect to the objects that should be classified in the end (the insects). The purpose of defining a region of interest is to discard areas in the image that for sure do not contain any useful information, e.g. the area around the petri-dish (the table). Actually the region of interest should be specified even more precise so that it do not include the sides of the petri-dish, because these contain reflections of the insects inside the petri-dish. This is considered as noise and could possibly lead to mis-classification. Regarding enhancing the image with respect to the objects of interest this e.g. could be in terms of smoothing the image in order to remove small disturbances such as soil which can have almost the same colour as the insect, but is considerably smaller. The pre-processing module will be highly dependent on the given experiment setup so if another setup is used, e.g. another background or a petri-dish that is not circular, the module will have to be replaced because it rely on these parameters. Segmentation By doing the segmentation it should be possible to remove almost everything from the picture that is not of interest. The output of the segmentation therefore defines the areas in the picture that contains the insects that should be classified. Focus in this project will be on finding the insect specie Folsomia Candida. However it should be possible to extend to application to find other insects with other characteristics. Instead of being able to setup the segmentation module to find various types of insects the philosophy will be to develop segmentation modules for the different insects. It will mainly be if the insects vary considerably in colour that it will be necessary to use different segmentation modules

26 24 Problem Analysis because this affects the method of segmentation very much. From the main program it then will be possible to run the pre-processed image through more segmentation modules and use all the segmented objects in the following Representation and Description. Representation and Description With the knowledge of which parts of the image contains insects features of every object expected to be an insect can be extracted. Examples of possible features are: size, length, width, colour, circularity, etc. Evaluation of Objects As described in the analysis of the experiment (section 2.1) a problem often encountered is that the insects find together in clusters. When doing a segmentation it therefore cannot be guaranteed that the objects found each actually represent a single insect. Therefore the responsibility of this module is both evaluate on the features that have been extracted from the objects in order to evaluate whether the given object only represent a single insect or whether there can be any doubt about this. If this is the case a more thoroughly of the given objects need to be performed in which the individual insects in such clusters are partitioned. After doing this features from these new objects naturally need to be extracted before being passed on to the object recognition. Object Recognition As the purpose of the experiment being performed is to count and classify the insects in a sample as being either females, males, large juveniles or small juveniles the purpose of the object recognition module is to perform this classification based on the information extracted in the Representation and Description. 2.4 Problem Formulation From the outlined test procedure (section 2.1), the need for an systematised and automated counting seems obvious. Although previous attempts have been made to create a system for automating the procedure of identification and segmentation, a number of problems with these approaches was identified. The overall goal of the project is to design and implement an application that is able to count and classify insects in a series of images obtained from a given ecotoxicological experiment with the Collembola specie Folsomia Candida. The problems addressed in this project are more specifically: To create a preprocessing module that is able to locate the region of interest in the images such that no reflections of insects are present, without removing any insects in the process. Furthermore it is to enhance the located region of interest, thereby emphasising the difference between insects and the background. To create a segmentation module, which based on the region of interest, is able to automatically partition the image into a region of background pixels and a number of object

27 Readers Guide 25 regions. This should be done without any false negatives (insects treated as background), together with a minimum of false positives (non-insect objects treated as insects). To create a feature extraction module that is able to extract relevant and correct features from the located insect objects in the image. To create a module that is able to identify clusters of insects, the number of insects in these clusters and the borders separating the individual insects. To create a classification module that is able to group the found insects into the four groups (male, female and two groups of juveniles), based on the extracted information Delimitation The primary purpose of project will be investigating, describing and testing different image analysis methods on the problem at hand. The analysis, design and implementation of the program parts apart from the image analysis modules (e.g. user interface), will not be addressed further, since it is beyond the scope of this project. The user requirements have been noted and will be used wherever they have influence on the image analysis part of the program. 2.5 Readers Guide In the preceding problem analysis a partitioning of the responsibilities that an application solving the problem at hand should be able to handle was defined. Independent modules covering these responsibilities was setup and with the problem formulation it was defined which of these problem areas should be addressed. The remainder of this report concerns documenting the efforts put into these areas. Each module will be analysed individually and tests performed on the implementation made of each module will be described in connection with this. The modules being described in the following chapters therefore are: Preprocessing Segmentation Representation and Description Evaluation of Objects Object Recognition It has been chosen to address the individual modules independently of each other because of the responsibilities of these being well defined. Based on this conclusions on the results obtained within the individual modules are made for each of these. After the documentation of all modules an evaluation of the work done throughout the project will be given concluding on the results obtained and the experiences gained regarding the given problem domain. Also a perspective of the application implemented will be given considering the further development of this.

28 26 Problem Analysis

29 Chapter 3 Preprocessing The purpose of this chapter is to describe the preprocessing module. The responsibility of this module is to prepare the acquired pictures for further processing. Initially the image acquisition setup is described, after which the process of locating and enhancing the region of interest is considered. This is done to facilitate the operation of the following image processing modules. 3.1 Introduction The purpose of the preprocessing is to narrow the area in which there will be searched for insects (reduce the search space). It can be described as a rough segmentation in the sense that only parts of the image, in which the insects can be present, are included. A reduction of the search space avoids any misclassifications outside the region of interest e.g. because of mirrored insects. In figure 3.1 some mirrored insects can be seen, figure 3.1(a) shows the actual image and figure 3.1(b) shows candidates for recognisable insects, which all in some way has changed features due to mirroring. There are three different types of mirroring in the petri dish: The object is mirrored, but with a slight distortion and a change of colour. The object is mirrored, but only a part of the object reflects, since these consist of illuminated areas which are mirrored more easily - e.g. only the reflections of the brown insects is mirrored. The object is mirrored, but due to dirt the object tends to blur. (a) Insects are mirrored in the petri dish. (b) Insects and mirrored objects marked. Figure 3.1: Example of how insects are mirrored in the petri dish and change features due to this mirroring. It is a requirement to the preprocessing that mirrored objects are removed since these hold very similar features which heightens the risk of being classified as a specific insect. 27

30 28 Preprocessing The location of the region of interest is a way of reducing background noise, which is important even though the image samples are collected in a controlled environment - e.g. the environment may be considered differently by different laboratory technicians. Background noise is a considerable factor and it would effect the rest of the image processing without proper handling. Reducing the search space has the advantage that no noise from the background have to be taken into account in the further processing, which makes the method more robust and improves the performance - both in efficiency and reliability. Everything outside the region of interest can be ignored and the region itself can be enhanced to accentuate certain details. In case the container of the experiment is exchanged the preprocessing is also exchangeable or even reusable to some extend (i.e. if the new container holds some of the same properties, it can be exchanged without affecting the rest of the image processing). 3.2 Defining and Constraining the Setup In the previous section the advantages of proper location the region of interest was stated. Before investigating the problem further the given setup is described. The setup is a mixture of the existing equipment utilised in previous applications at DMU (described in section 2.1.2) together with an added number of constraints defined in the following. The pictures are obtained using a Nikon D70 camera with an attached ring flash. The ring flash is attached to supply uniform illumination and thereby produce shadowless images. The obtained images have a resolution of 6 [M P ixels] yielding image sizes. The camera is mounted on a tripod to obtain increased stability and uniformity between individual image acquisitions. The setup is shown in figure 3.2. Figure 3.2: Figure showing the image acquisition setup at DMU, consisting of a Nikon D70 camera with an attached ring flash mounted on a tripod. Besides the described camera setup, the following constraints are put on the setup.

31 Analysis of Images 29 It is assumed that the background of the obtained images is similarly coloured. Green has been chosen, which is discussed in greater detail in the next section. It is assumed that the same lighting is used every time. Is is assumed that the same material of the petri dish is used every time and that no noise on the sides (e.g. writings) is present. It is assumed that the petri dish always is centred in the obtained images. By use of the given setup a number of test images have been obtained ( 50 images of Folsomia Candida images/040428/). These form the basis for the further investigation of methods for locating and enhancing the region of interest. 3.3 Analysis of Images It has been chosen to investigate details of the given test images to acquire knowledge of the problem area and different regions in the image, especially the region of interest, which is the main focus throughout the rest of the image processing. Figure 3.3: Image sample of the problem area Colour Space Histogram Analysis The images are analysed by looking at their histogram of different spectral components for both the colour spaces RGB and HSV. The colour space of the input images is initially RGB, because of the image generation process in the camera. The RGB colour space consists of the spectral components red, green and blue, to which the human eye are all strongly perceptive. The colour space HSV is a transformation of the RGB space and it consists of the spectral components hue, saturation and value. Generally speaking the hue component describes the purity of the colour, while saturation describes a degree to which the colour is diluted by white light. The value is the brightness and describes the intensity of the colours. The HSV is often used in image processing algorithms, because the spectral components H, S and V, describe