Elsevier Editorial System(tm) for Computers in Biology and Medicine Manuscript Draft

Transcription

1 Esevier Editoria System(tm) for Computers in Bioogy and Medicine Manuscript Draft Manuscript Number: CBM-D--00R1 Tite: CLASSIFICATION AND RETRIEVAL ON MACROINVERTABRATE IMAGE DATABASES Artice Type: Fu Length Artice Keywords: Biomonitoring, cassification, radia basis function networs, mutiayer perceptrons, Bayesian Networs, Support Vector Machines, Benthic macroinvertebrate Corresponding Author: Dr. seran iranyaz, Corresponding Author's Institution: Tampere University of Technoogy First Author: seran iranyaz Order of Authors: seran iranyaz; Turer Ince; Jenni Puinen; Moncef Gabbouj; Johanna Arje; Same Karainen; Vie Tirronen ; Martti Juhoa; Tuomas Turpeinen ; Kristian Meissner Abstract: Aquatic ecosystems are continuousy threatened by a growing number of human induced changes. Macroinvertebrate biomonitoring is particuary efficient in pinpointing the cause-effect structure between sow and subte changes and their detrimenta consequences in aquatic ecosystems. The greatest obstace to impementing efficient biomonitoring is currenty the cost-intensive human expert taxonomic identification of sampes. Whie there is evidence that automated recognition techniques can match human taxa identification accuracy at greaty reduced costs, so far the deveopment of automated identification techniques for aquatic organisms has been minima. In this paper, we focus on advancing cassification and data retrieva that are instrumenta when processing arge macroinvertebrate image datasets. To accompish this for routine biomonitoring, in this paper we sha investigate the feasibiity of automated river macroinvertebrate cassification and retrieva with high precision. Besides the state-of-the-art cassifiers such as Support Vector Machines (SVMs) and Bayesian Cassifiers (BCs), the focus is particuary drawn on feed-forward artificia neura networs (ANNs), namey mutiayer perceptrons (MLPs) and radia basis function networs (RBFNs). Since both ANN types have been procaimed superior by different investigations even for the same benchmar probems, we sha first show that the main reason for this ambiguity ies in the static and rather poor comparison methodoogies appied in most earier wors. Especiay the most common drawbac occurs due to the imited evauation of the ANN performances over just one or few networ architecture(s). Therefore, in this study, an extensive evauation of each cassifier performance over an ANN architecture space is performed. The best cassifier among a, which is trained over a dataset of river macroinvertebrate specimens, is then used in the MUVIS framewor for the efficient search and retrieva of particuar macroinvertebrate pecuiars. Cassification and retrieva resuts present such a high accuracy that can match experts' abiity for taxonomic identification.

2 *Manuscript CLASSIFICATION AND RETRIEVAL ON MACROINVERTABRATE IMAGE DATABASES 1 Seran.iranyaz@tut.fi, turer.ince@ieu.edu.tr, jenni.puinen@tut.fi, moncef.gabbouj@tut.fi, johanna.arje@jyu.fi, same.arainen@jyu.fi, vie.e.t.tirronen@jyu.fi, Martti.Juhoa@cs.uta.fi, tuomas.turpeinen@jyu.fi, Kristian.Meissner@ymparisto.fi Abstract-- Aquatic ecosystems are continuousy threatened by a growing number of human induced changes. Macroinvertebrate biomonitoring is particuary efficient in pinpointing the cause-effect structure between sow and subte changes and their detrimenta consequences in aquatic ecosystems. The greatest obstace to impementing efficient biomonitoring is currenty the cost-intensive human expert taxonomic identification of sampes. Whie there is evidence that automated recognition techniques can match human taxa identification accuracy at greaty reduced costs, so far the deveopment of automated identification techniques for aquatic organisms has been minima. In this paper, we focus on advancing cassification and data retrieva that are instrumenta when processing arge macroinvertebrate image datasets. To accompish this for routine biomonitoring, in this paper we sha investigate the feasibiity of automated river macroinvertebrate cassification and retrieva with high precision. Besides the state-of-the-art cassifiers such as Support Vector Machines (SVMs) and Bayesian Cassifiers (BCs), the focus is particuary drawn on feed-forward artificia neura networs (ANNs), namey mutiayer perceptrons (MLPs) and radia basis function networs (RBFNs). Since both ANN types have been procaimed superior by different investigations even for the same benchmar probems, we sha first show that the main reason for this ambiguity ies in the static and rather poor comparison methodoogies appied in most earier wors. Especiay the most common drawbac occurs due to the imited evauation of the ANN performances over just one or few networ architecture(s). Therefore, in this study, an extensive evauation of each cassifier performance over an ANN architecture space is performed. The best cassifier among a, which is trained over a dataset of river macroinvertebrate specimens, is then used in the MUVIS framewor for the efficient search and retrieva of particuar macroinvertebrate pecuiars. Cassification and retrieva resuts present such a high accuracy that can match experts abiity for taxonomic identification. Index Terms-- Biomonitoring, cassification, radia basis function networs, mutiayer perceptrons, Bayesian Networs, Support Vector Machines, Benthic macroinvertebrate. I CLASSIFICATION AND RETRIEVAL ON MACROINVERTABRATE IMAGE DATABASES Seran Kiranyaz, Turer Ince, Jenni Puinen and Moncef Gabbouj 1, Johanna Arje and Same Karainen, Vie Tirronen and Martti Juhoa, Tuomas Turpeinen and Kristian Meissner I. INTRODUCTION T is an unfortunate fact that aquatic ecosystems are facing a growing number of anthropogenic pressures operating at severa tempora and spatia scaes (e.g. eutrophication, goba warming).

3 CLASSIFICATION AND RETRIEVAL ON MACROINVERTABRATE IMAGE DATABASES We panned biomonitoring is essentia to detect the cause-effect structure between the often subte pressures and their ecosystem consequences. The resuting growing goba need to impement more biomonitoring is apparent but due to the cost-intensive human expert taxonomic identification of sampes, this need cannot currenty be met. Automatic and semi-automatic signa and image processing techniques have been successfuy appied in simiar fieds of appication to sove such chaenges. For instance, automatic image recognition techniques of aquatic phytopanton have been shown to match human taxa identification accuracy at a greaty reduced cost [1]. Despite their obvious potentia, the deveopment of automated taxa identification techniques has ong been hampered by the reuctance of taxonomic experts to embrace aternative methods of taxa identification. A detaied review on advances in automated taxa identification [] deemed misconceptions, the ac of vision and the ac of enterprise more imiting to the deveopment of automated taxa identification than actua practica constraints. Research on automated recognition of aquatic organisms has mainy concentrated on panton [], whie automated cassification and particuary retrieva of freshwater macroinvertebrates has received itte attention []. In a recent wor based on SVMs [] on a set of river macroinvertebrates, mean correct cassification rates of.% and.% have been achieved in training and test sets, which match the eves of human accuracy for other aquatic taxonomic groups []. In this paper, the primary goa is to deveop a ow-cost, automated and accurate benthic macroinvertebrate cassification and retrieva system for routine biomonitoring. In order to accompish this we sha investigate the state-of-the-art cassifiers such as Support Vector Machines (SVMs), Bayesian cassifiers (BCs) and two most common feed-forward artificia 1 This wor was supported by the Academy of Finand, project No. 1 (Finnish Centre of Exceence Program (00-0)

4 CLASSIFICATION AND RETRIEVAL ON MACROINVERTABRATE IMAGE DATABASES neura networs (ANNs): mutiayer perceptrons (MLPs) and radia basis function networs (RBFNs). SVMs are nown to be efficient cassifiers, i.e. they can cassify test cases with a high accuracy. BCs are simpe to impement, and they are often efficient cassifiers, i.e. they cassify test cases amost as accuratey as more sophisticated and compicated cassifiers, see e.g. []. Particuar focus wi be drawn on feed-forward ANNs since earier wor on cassification of aquatic organisms has shown that neura networs usuay outperform decision trees [] and other cassica statistica techniques [1]. ANNs have proven to perform compex cassification tass, provided that a proper structure for the networ is seected and a suitabe training technique is appied to a sufficienty representative set of data. In order to achieve the highest retrieva performance possibe, among a cassifiers investigated, the best cassifier, which achieves the minimum cassification error (CE) in the test (vaidation) set, thus showing the utmost generaization abiity, wi then be used for retrieva within a Benthic macroinvertebrate image database. Finay, in order to achieve a ow-cost appication and avoid computationay extensive features, we use ony a simpe and basic feature extraction technique composed of mainy geometrica and statistica features. In this way the efficiency of the cassifiers wi aso be ceary investigated and tested against the imited discrimination power of those simpe features. A certain ambiguity arises when comparing the performance of ANNs or to search for the best ANN type or configuration among many aternatives. For instance most of the biomedica appications of ANNs use the we-nown MLPs []. Some promising resuts have aso been gained with another basic type of feed-forward ANN, the RBFNs []. Compared to MLPs, the interna operation of RBFNs is easier to comprehend and faster training agorithms are avaiabe for them []. However, the use of RBFNs for practica purposes is sti quite imited. One of the

5 CLASSIFICATION AND RETRIEVAL ON MACROINVERTABRATE IMAGE DATABASES reasons is certainy that the comparison resuts between the two ANN types are varying and sometimes even contradictory. Severa artices report better resuts with RBFNs than MLPs with appications in function approximation [1], image segmentation [1], speaer recognition [1], modeing of rating curves in hydroogy [], etc. Yet some artices have found MLPs to perform better than RBFNs when appied to, for exampe, remote sensing cassification [1] and automatic speech recognition [1]. Severa comparisons of MLPs and RBFNs have aso been conducted within the biomedica domain, but the resuts are equay varying. To address this probem, we sha first show that the major cause for such inconsistencies is due to the fact that the evauation methodoogies used are imited or rather deficient and often biased. For instance, in most comparative evauations of MLPs and RBFNs, ony one or few networ architecture(s) are considered. We sha demonstrate that there are significant variations between performances of different architectures especiay with MLPs. If this simpe fact is omitted, it can ead to either of the ANN types to be better on neary any probem. Moreover, the training method for both networ types, the we-nown bac-propagation (BP) yieds significanty varying networ parameters (weights and biases) after each training session and thus a imited number of training runs cannot yied statisticay significant performance measures and accurate evauations. In order to carify such varying or even contradictory resuts of the previous studies, we first propose an accurate and in-depth performance assessment approach for comparing these two common networ types. In the current wor, we evauate the cassification error (CE) over the training and the test sets and as performance criteria we use the mean and the minimum errors that an ANN cassifier of a particuar type can achieve. We aso anayze the variations of these errors among different architectures for each networ type in order to investigate the dependence of the cassification performance on the variations of the networ

6 CLASSIFICATION AND RETRIEVAL ON MACROINVERTABRATE IMAGE DATABASES configurations. In the proposed assessment scheme, to avoid the bias of the networ architecture used, rather than focusing on ony one or few architectures, we propose that the evauations are performed over an architecture space containing a arge variety of ANNs, e.g. from the simpest singe-ayer perceptrons (SLPs) to the MLPs with severa hidden ayers and severa neurons; and simiary, from the simpest RBFNs with few hidden neurons to networs with many hidden neurons. Foowing performance evauation of ANN cassifiers using the proposed assessment methodoogy and comparison with the other two common cassifier types (SVMs and BCs), the best cassifier, which yieds the best generaization capabiity (the owest CE in the test/vaidation set) is then integrated into the search engine of the MUVIS [] and used for the (dis-) simiarity distance computation within the simiarity-based queries. In this way we sha demonstrate that the retrieva performance can be significanty improved, particuary as compared to the traditiona query methods. The rest of the paper is organized as foows. Section II surveys the reated wor on a cassifiers used in the present wor and discusses briefy their training methods. The proposed approach for feature extraction, cassification and retrieva in macroinvertebrate databases with the detaied discussion over the proposed assessment system for evauating ANN cassifiers are presented in Section III. Section IV provides the cassification and retrieva experiments conducted over Benthic macroinvertebrate image databases and discusses the resuts. Finay, Section V concudes the paper and proposes topics for future research.

7 CLASSIFICATION AND RETRIEVAL ON MACROINVERTABRATE IMAGE DATABASES II. RELATED WORK A. Support Vector Machines SVMs are a modern computationa data anaysis method to cassify data mainy to two casses [] - [1]. The idea is to find a hyperpane that optimay separates a given dataset, i.e, has the maxima margin between the data points and the hyperpane. In the inear case this is iustrated in Figure 1. To hande non-ineary separabe cases the so caed erne tric is used. Instead of finding the hyperpane directy in the given feature space, a noninear mapping (transformation) is appied to the data. In this usuay higher dimensiona space it is possibe to buid a inear hyperpane to separate two casses optimay instead of possiby poorer cassification opportunities in the origina dimension (e.g. see conceptua iustration in Figure ). This resuts in a non-inear cassifier in the origina feature space. Figure 1: Formation of the inear optimay separating hyperpane. Figure :Non-separabe sets and transformation to a non-inear cassifier.

8 CLASSIFICATION AND RETRIEVAL ON MACROINVERTABRATE IMAGE DATABASES Casses refer to different macroinvertebrate species in the current context. Since this is a muticass probem, (i.e., there are more than two casses) SVMs have to be extended to hande such probems. Up to date muticass cassification with SVMs is an ongoing topic in the research community. There are two main approaches for muticass extensions. One approach is based on training severa binary cassifiers and combining their resuts. It is possibe to divide the training data of c casses to c sub-probems so that the training data of each cass is separated from the union of the other c-1 casses and this dichotomy is aternatey repeated for a c divisions. A separate SVM is then trained for each of c divisions. The other approach considers a the data at once in a singe optimization tas (e.g. see [] and []). In [] the former approach was found sighty more practica. In this wor we appy SVM impementation as described in [] that uses the foowing oneagainst-one methodoogy (aso described in []) for muticass probems. The training probem is divided into c(c-1)/ sub-probems so that each pair of casses is used to train one cassifier that separates those two casses. In cassification a voting strategy is used. Each of the c(c-1)/ cassifiers is appied to the data (feature) point x which is then assigned to the cass with the majority of the resuting predictions. B. Bayesian Cassifiers Bayesian cassifier (BC) is a traditiona method for cassifying data of severa casses using features of cass individuas and prior nowedge of proportions of each cass. The rues on which cassification decisions are based are expressed in terms of probabiities []. Mathematicay speaing, we consider the cass w of an individua to be a random variabe having vaues w 1,..., w and corresponding probabiities P ( w 1 ),..., P( w ), which are caed prior probabiities. Furthermore, features of each cass, x x,..., x }, are assumed to be generated by { 1 P

9 CLASSIFICATION AND RETRIEVAL ON MACROINVERTABRATE IMAGE DATABASES a statistica mode, whose distribution, in the case of continuous features, is expressed by the probabiity density function (or ieihood) f x w ). This mode can, for exampe, be a mutinorma distribution. Our interest is focused on the posterior probabiity, that is the probabiity of each cass given the features x of an individua. According to Bayes rue, this posterior is formuated as, P( w x) ( i f ( x w ) P( w ) i, i f ( x) where f (x) is the probabiity density function of features x. An integra part of the posterior is the product of the ieihood and the prior whie f (x) can be interpreted as a scaing factor. When maing the decision of the cass of an individua with its features x, the natura choice is to choose the cass for which the posterior P( w i x) is maximum. This is the so-caed Bayesian decision rue and it can be shown that it minimizes the average error probabiity. When training the Bayesian cassifier, the statistica mode of features needs to be estimated from the training sampes. For instance in the case of Gaussian features the Bayesian decision rue of cassification is simpy reached by estimating the mean and covariance matrix for each cass that are extracted from the training sampes. C. Artificia Neura Networs An ANN consists of a set of connected processing units, usuay caed neurons or nodes. ANNs can be described as directed graphs, where each node performs some activation function to its inputs and then gives the resut forward to be the input of some other neurons unti the output neurons are reached. ANNs can be divided into feed-forward and recurrent networs according to their connectivity. In a recurrent ANN there can be bacward oops in the networ structure, whie in feed-forward ANNs such oops are not aowed. Furthermore, feed-forward i (1)

10 CLASSIFICATION AND RETRIEVAL ON MACROINVERTABRATE IMAGE DATABASES ANNs are usuay organized into ayers of parae neurons and ony connections between adjacent ayers are possibe. A ayers besides the input and output ayers are caed hidden ayers. The input ayer is just a passive ayer, where no computations are carried out and it is not counted to the tota number of ayers. The active neurons perform an activation function f of the form, where y p, 1 N p, p, 1 y f wj y j, j1 is the output of neuron of ayer, when pattern p is fed to the ANN, number of neurons in ayer -1, () 1 N is the tota wj is the connection weight between neuron j in ayer -1 and neuron in ayer, is the bias of the neuron in ayer. For the first processing ayer (the ayer right after the input ayer) p p p p pattern x x x j xn neurons y y is naturay the j th dimension of the input p, 1 p,0 j j,...,,..., 1. The number of input neurons N i i and the number of output No for ANNs are defined by the probem, whie the number of hidden ayers and the number of neurons in each hidden ayer is somehow decided usuay by an expert and with respect to the probem. A sampe feed-forward ANN is iustrated in Figure. It has three ayers (two hidden ayers and the output ayer). Figure aso shows the connection weights bias for the first neuron in ayer. 1 w j1 and the

11 CLASSIFICATION AND RETRIEVAL ON MACROINVERTABRATE IMAGE DATABASES Figure : An exampe of fuy-connected feed-forward ANN. The most common ANN type is the mutiayer perceptron (MLP) []. It is a feed-forward networ, which contains one or more ayers of hidden neurons. The degree of neuron connectivity is usuay high and the neurons have noninear activation functions, but the noninearity is smooth (differentiabe everywhere). The use of noninear activation functions is essentia because otherwise the MLP coud aways be reduced to a singe-ayer perceptron (SLP) without changing its capabiities. A commony used activation function is the tangent hyperboic (tanh) function which is defined as foows: ( v p, ) ( v p, ) 1 N p, p, e e p, p, 1 y tanh( v ) p, p, where v ( v ) ( v ) wj y j () e e j1 Another popuar type of feed-forward ANN is the radia basis function (RBF) networ [], which has aways two ayers in addition to the passive input ayer: a hidden ayer of RBF units and a inear output ayer. Ony the output ayer has connection weights and biases. The activation function of the th RBF unit is defined as y, () where is a radia basis function or, in other words, a stricty positive radiay symmetric function, which has a unique maximum at N-dimensiona center and whose vaue drops

12 CLASSIFICATION AND RETRIEVAL ON MACROINVERTABRATE IMAGE DATABASES rapidy cose to zero away from the center. is the width of the pea around the center. The activation function gets noteworthy vaues ony when the distance between the N-dimensiona input and the center,, is smaer than the width. The most commony used activation function in RBFNs is the Gaussian basis function defined as, where and () y exp, are the mean and standard deviation, respectivey, and denotes the Eucidean norm. More detaied information about MLPs and RBFNs can be obtained from []. D. The Bac-propagation Agorithm Bac-propagation (BP) [] is the most commony used training technique for feed-forward ANNs. It is a powerfu supervised training technique which has been used in pattern recognition and cassification probems in many appication areas. BP has the advantage of appying directed search and has strong oca search abiity. However, BP is just a gradient descent agorithm in the error space, which can be compex and may contain many deceiving oca minima (muti-moda). Therefore, BP gets most iey trapped into a oca minimum, maing it entirey dependent on the initia (weight) settings. There are many BP variants and extensions trying to address this probem, [], [], [], yet the performance and computationa cost of each agorithm varies with respect to the probem at hand; and the question of which ANN architecture (number of ayers and interconnections, number of nodes, etc.) shoud be used for a particuar probem sti remains unanswered. The BP agorithm can be summarized as foows: 1. Initiaize the weights centers and. w j and biases randomy. For RBF-networs initiaize aso the pea. Feed pattern p to the networ and compute the output y p, of each neuron.

13 CLASSIFICATION AND RETRIEVAL ON MACROINVERTABRATE IMAGE DATABASES 1. Cacuate the error between the computed output output p t as e t y. p, o p p, o. For each neuron, cacuate the partia derivatives defined as E p o y po, E h of each output neuron and the desired p, where p E is the tota error energy 1 p, o ( e ) and h is a uniform symbo for each parameter w j,, and The name of the bac-propagation agorithm comes from the fact that one starts to cacuate the oca gradients from the output ayer and then iterativey proceed bacwards toward the input ayer. The formuas for cacuating the oca gradients for MLPs and RBFNs can be found in [] and [0], respectivey.. Update the parameters as foows: p E h( t 1) h( t) () h where is the earning rate parameter.. Repeat steps - unti some stopping criteria is reached. One compete presentation of the training set is caed an epoch. Usuay many epochs are required to obtain the best training resuts, but, on the other hand, too many training epochs can ead to over-fitting. In the above reaization of the BP agorithm the networ parameters are updated after every training sampe. This is caed the onine or sequentia mode. The other possibiity is the batch mode, where a the training sampes are first presented to the networ and then the parameters are adjusted so that the tota training error is the number of training sampes, is minimized instead of p1 o. P 1 po, E ( e ), where P p E. The sequentia mode is often favored over the batch mode as it requires ess storage space. Moreover, the sequentia mode is ess iey to get trapped in a oca minimum as updates at every training sampe mae the search

14 CLASSIFICATION AND RETRIEVAL ON MACROINVERTABRATE IMAGE DATABASES 1 stochastic in nature. However, the stochastic aspects aso mae it difficut to derive theoretica conditions for the convergence of the sequentia BP mode, whie for the batch mode the convergence to a (oca) minimum can be guaranteed under simpe conditions. Aso most of the existing improvement strategies are based on the batch mode. In this study we use sequentia BP mode for MLP training and SuperSAB enhancement [1] of the BP agorithm when training RBFNs. The main difference in SuperSAB is that it modifies the update-vaues for each parameter according to the sequence of signs of the partia derivatives. This ony eads to a faster convergence, whie the probems of a hi-cimbing agorithm are not soved. Further detais about BP and SuperSAB can be found in [1] and [], respectivey. III. CLASSIFICATION AND RETRIEVAL IN MACROINVERTEBRATE DATABASES A. Dataset Creation and Feature Extraction The Benthic macroinvertebrate image dataset used in this wor consists of 1 images representing different taxonomica groups: Baetis rhodani, Diura nanseni, Heptagenia suphurea, Hydropsyche peucidua, Hydropsyche sitaai, Isopera sp., Rhyacophia nubia and Taeniopteryx nebuosa. Members from the same taxonomica group were imaged by a fatbed scanner, digitized, normaized and eventuay each macroinvertebrate in each scan was saved as an individua image. Three individuas from four taxonomica casses are shown in Figure and demonstrate some crucia properties of the data: specimens are semi-rigid so that the actua shape may vary from one sampe to another. Furthermore, there can be overapping, repetitions, rotations, scaing and variations in the intensity eves, a of which mae the cassification probem even more chaenging and the need for a powerfu cassifier is imminent to match the accuracy of expert-eve human cassification.

15 CLASSIFICATION AND RETRIEVAL ON MACROINVERTABRATE IMAGE DATABASES 1 (A) (B) (C) (D) (E) (F) Figure : Three sampes from Baetis rhodani (top: A, B, C), Diura nanseni (top: D, E, F), Hydropsyche peucidua (bottom: A, B, C) and Heptagenia suphurea (bottom: D, E, F) casses. Feature extraction and seection depend on the data and the cassifier used. As such, features can be seen as a part of the cassifier system itsef. However, such feature seection in case of images has remained empirica science as there are many possibe features and an enormous amount of their combinations. Exampes of cassica features are various edge, curve, ridge, bob, and corner based features, shape descriptors [] such as various moments and Fourier descriptors, simpe textura features such as histograms of intensity, gradient [] and gray-scae co-occurrences []. Current state-of-the-art feature extraction methods incude Loca Binary Patterns [], Gabor pacet based methods [], Co-occurrence matrices [], scae invariant features of SIFT agorithm [] and various other orientation based features such as in []. Among a these possibiities, in order to achieve a ow-cost soution and to demonstrate the efficiency of the proposed cassifier for simiarity-based retrieva, we have appied a simpe and basic feature extraction technique composed of mainy geometrica and statistica features. -D features of each macroinvertebrate image are extracted by using ImageJ [], which is a pubic domain, Java-based image processing program. The foowing set of features are seected by using ImageJ s buit in measurement and anaysis functions: pixe vaue (grayscae) statistics {,, Mode, Median, IntDen, Kurtosis, Sewness} and geometric features {Area, Perimeter,

16 CLASSIFICATION AND RETRIEVAL ON MACROINVERTABRATE IMAGE DATABASES Width, Height, Ferret, Major, Minor, Circuarity}, for which ImageJ performs a simpe threshoding operation to extract the binary mas of each macroinvertebrate sampe. The detaied description of these features can be found in []. In a pre-processing step, each feature vector is then normaized to have a zero mean and ineary scaed into [-1, 1] interva before being presented to the cassifier. B. The Assessment Methodoogy for Feed-forward ANNs In order to objectivey assess the performance of MLPs and RBFNs, we appy exhaustive BP training over a wide variety of networ architectures whist eeping in mind that too compicated configurations may not be appicabe in practice. In this way we can avoid the bias or possibe effect of a particuar networ on the performance, whie many of the aforementioned studies faied to do so as they were mosty performed using ony one or few fixed networ architecture(s). For training, the tangent hyperboic activation function given in Eq. () was used with MLPs whie the Gaussian basis function given in Eq. () was used with RBFNs. The architecture space for MLPs may be defined over a wide range of configurations, i.e. say from a SLP to compex MLPs with many hidden ayers and neurons. Suppose that a range is defined for the number of ayers, L, L } and another for the number of neurons for each { min max ayer, N, N }. Consequenty, the architecture space can now be defined with ony two { min max 1 Lmax1 1 Lmax1 arrays, R { N, N,..., N, N } and R { N, N,..., N, N }, one for minimum min I min min O max and the other for maximum number of neurons aowed for each ayer of a MLP. The size of both arrays is naturay L max 1, where the corresponding entries define the range of neurons possibe on the th hidden ayer for a those MLP configurations, which have an th hidden ayer. L and L max L min can be set to any vaue meaningfu for the probem at hand. The size of min 1 I max max O

17 CLASSIFICATION AND RETRIEVAL ON MACROINVERTABRATE IMAGE DATABASES 1 the input and output ayers, N, N }, are defined by the probem (e.g. { N, } {,} for the { I O I N O current wor) and remains the same for a configurations in the architecture space, whie the geometric pyramid rue (GPR) [] may for instance be used as a rough guideine for setting the range for the number of neurons in each hidden ayer. A networ configurations in the architecture space are then enumerated into a hash tabe with a proper hash function, which basicay rans the configurations with respect to their compexity, i.e. it associates higher hash indices to architectures with higher compexity. The hash indices start from the simpest SLP and proceed to the most compex networ configuration with L max 1 hidden ayers, each of which has the maximum number of neurons given in R. Tabe I. Sampe MLP architecture space containing MLP configurations. Ind. Configuration Ind. Configuration Ind. Configuration 0 x 1 x x x x x x 1 x x 1 x x x x x x x x x x x x x x x x x x x x x x x x 0 x x x x x x x 1 x 1 x 1 x x x 1 x x x 1 x x 1 x x x 1 x x x 1 x x 1 x x x 1 x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x 1 x x x x x x x x x 1 x 1 x x x 1 x x x 1 x x 1 x 1 x x 0 x 1 x x x 1 x x x 1 x x 1 x 1 x x x 1 x x x x x max Tae, for instance, the foowing range arrays, R { N i,,, N } and R { N i,,, N }, which indicate that L max. If L 1 then the hash function enumerates a MLP min configurations in the architecture space as shown in Tabe I for { N, } {,}. The hash min o I N O max o

18 CLASSIFICATION AND RETRIEVAL ON MACROINVERTABRATE IMAGE DATABASES 1 function associates index 0 with the simpest networ configuration, which is the SLP with no hidden ayers. From indices 1 to, a configurations beong to -ayer MLPs with to neurons in the singe hidden ayer (as specified in the nd entries of arrays Rmin and R max ). Simiary for indices and up, -ayer MLPs are enumerated in which the number of neurons in the 1 st and nd hidden ayers is varied according to the corresponding entries in Rmin and R max. Finay, the most compex MLP with the argest possibe number of ayers and the highest number of neurons in each ayer is associated with the highest index, d=. Therefore, a entries in the hash tabe span the architecture space with respect to the configuration compexity. Since there is ony one hidden ayer for RBFNs, the number of hidden neurons can be directy used for the architecture space indexing. In this wor, we used an architecture space consisting of RBFNs containing - hidden neurons. C. Training of the Cassifiers Reca that in the Benthic macroinvertebrate image dataset used in this wor, -D features are extracted per image and there are casses, which mae the input and output ayer sizes as { N, } {,}. In order to evauate the effect of the data partitioning, we created different I N O training and test partitions over the entire dataset, each with randomy chosen 0 and 00 sampes, respectivey. Therefore, in each partition the training and test sets contain a random shuffe of images whist the test set has the majority (00) of the dataset. Figure presents an overview of cassifier training for a particuar training set. In order to accompish an exhaustive evauation of ANN cassifiers, independent runs are performed with random parameter initiaization to compute the error statistics for each dataset partition and furthermore, the training process is performed for each configuration in the architecture spaces so as to accompish the proposed assessment methodoogy as expained earier. We consider the training

19 CLASSIFICATION AND RETRIEVAL ON MACROINVERTABRATE IMAGE DATABASES CE and particuary the test CE, which is the primary objective of the cassifier as it shows the cassification accuracy eve achieved as we as the generaization capabiity of each networ type. Both training and test CEs sha then be evauated by considering the mean and the minimum errors achieved by each cassifier. Figure : Overview of a cassifier training. For BP training, we have appied 000 epochs for the training of both types of ANNs. The increase and decrease factors of BP (SuperSAB) operations over RBFNs are set to 1.0 and 0., respectivey. The earning rate for MLP training is set to Such parameters for BP (high iteration number and quite ow earning rate / increase factor) are purposefuy set to prevent osciations and to ensure convergence to a oca optimum. The support vector machines are appied with RBF erne and automatic parameter seection. The critica parameters for this erne, C and, are seected using cross vaidation and parae grid search. In cross vaidation the data is spit into severa fods. Then each fod is in turn considered as a vaidation set for the rest of the data. The parameter seection is done by testing the parameter space Cx using a grid of, C,,..., and, 1,...,. Parameters resuting in highest cross-vaidation accuracy are used for the training. This approach is simiar to

20 CLASSIFICATION AND RETRIEVAL ON MACROINVERTABRATE IMAGE DATABASES what is described in []. Finay, BCs are trained in such a way that the priors are set to be equa for each cass. As the statistica mode of features the mutinorma distribution is used when the cassification rue is reached by estimating the mean and covariance matrix of each cass from the training sampe. A. Cassification Resuts IV. EPERIMENTAL RESULTS The cassification performance evauation of the SVMs and BCs are reativey straightforward since both techniques have deterministic training methods, which do not depend on (random) parameter initiaization. So the training and test CEs per partition, as enisted in Tabe, can directy be used for comparative evauations. It is cear that both cassifiers show comparabe performance over the test sets. Yet minimum training CEs are achieved ony by SVMs performing consistenty better cassification resuts than BCs within the training sets. Tabe : Train and test cassification error statistics for SVMs and BCs per dataset partition. The best (minimum) error statistics are highighted. Partitions Training CE Test CE SVM BC SVM BC Par Par Par Par Par Par Par Par Par Par In order to perform a comprehensive and systematic assessment of the performance of ANN cassifiers, we pot the error statistics (both training and test CEs) versus hash indices for each

21 CLASSIFICATION AND RETRIEVAL ON MACROINVERTABRATE IMAGE DATABASES 0 configuration in the architecture space(s). Note that a hash index simpy corresponds to the number of hidden (Gaussian) neurons for RBFNs and see Tabe I for the architecture space used for MLPs. Due to space imitations we present resuts ony on two partitions (1 and ). Best Test CE Figure : MLP Train (top) and test (bottom) CE statistics vs. hash index pots for partitions 1 and. Figure presents the training and test CE statistic pots obtained from MLP training over the dataset partitions 1 and. Both pots ceary demonstrate that the MLP cassification performance significanty depends on the networ configuration used since particuary the mean training CEs vary among different MLPs more than %. Note that severa MLP configurations achieved 0% CE in the training set and ess than % CE in the test set. Among a partitions, the best MLP configuration, which achieves the minimum test CE (~.% corresponding to the train CE ~0.%) is obtained from the training of the partition, as indicated in the bottom pot of Figure (with index 1, equivaent to -ayer MLP configuration x1xx).

22 CLASSIFICATION AND RETRIEVAL ON MACROINVERTABRATE IMAGE DATABASES 1 RBFs with high CE Best Test CE Figure : RBF train (top) and test (bottom) CE statistics vs. hash index pots for partitions 1 and. Figure presents the training and test CE statistics pots obtained from RBF training over the same partitions (1 and ). Note that the effect of different dataset partitioning is quite visibe here, i.e. different RBFN configurations yied the poorest cassification performance (i.e. the two peas in the CE pot), as shown in the top pot of Figure. It is aso worth mentioning that compared to MLPs, RBFNs usuay exhibit a monotonous and stabe performance eve, that is somewhat independent from the networ configuration even though (both training and test) CEs get sighty ower as the networ compexity (number of Gaussian neurons) rises. Yet RBFNs exhibit the poorest cassification performance eve among a cassifiers. This is in fact an expected outcome since BP is just a gradient descent agorithm on the error space, and besides weights and biases (as these are the ony parameters computed for MLPs), it furthermore computes -D centroids and variance per Gaussian neuron for RBFNs. In this case, the error surface obviousy becomes extremey compex and contains massive amount of deceiving oca minima. Therefore, BP in RBFN training most iey gets trapped into a oca minimum earier,

23 CLASSIFICATION AND RETRIEVAL ON MACROINVERTABRATE IMAGE DATABASES maing its cassification performance entirey dependent on the initia settings. The overa cassification resuts, first of a, show that training and test performances of both ANN types depend on the networ architecture used and the dataset partitioning appied, each of which has varying effects on the two performance criteria empoyed. This justifies the use of the proposed assessment technique for both ANN types, MLPs and RBFNs, over Benthic macroinvertebrate image dataset. Among a cassifiers, the overa best cassification performance is obtained with the MLP configuration mentioned earier and it is therefore, used to improve the retrieva resuts, as detaied next. B. Retrieva Resuts The retrieva process in MUVIS [] is based on the traditiona query by exampe (QBE) operation. The features of the query item are used for (dis-) simiarity measurement among a the features of the visua items in the database. Raning the database items according to their simiarity distances yieds the retrieva resut. The traditiona (dis-) simiarity measurement in MUVIS is computed by appying a distance metric such as L (Eucidean) between the feature vectors of the query and the (next) database item. So in Benthic macroinvertebrate image database, this corresponds to computing the Eucidean distance between two -D feature vectors. In order to obtain the highest retrieva performance, we have chosen the MLP cassifier with the best generaization abiity (i.e. the -ayer MLP, which achieved the overa minimum test CE for partition-, with the foowing configuration: x1xx). When the cassifier is used, the same (L) distance metric is now appied to the cass vectors at the output ayer to compute the (dis-) simiarity distance between the query and the (next) database image. Since the cassifier has an eegant discrimination capabiity among different cass members, in this way the retrieva performance can further be improved.

24 CLASSIFICATION AND RETRIEVAL ON MACROINVERTABRATE IMAGE DATABASES The god standard (or ground truth) data is avaiabe for the entire dataset we used. Basicay a retrieved image is reevant if it has the same cass type (macroinvertebrate type) as with the query image. For each query we used a search window, which is equa to the number of images beonging to the query s cass. In order to measure the retrieva performance, we used an unbiased and imited formuation of Normaized Modified Retrieva Ran (NMRR(q)), which is defined in MPEG- as the retrieva performance criteria per query (q). It combines both the traditiona hit-miss counters; Precision and Reca, and further taes the raning information into account as given in the foowing expression: AVR( q) N ( q) R( ) 1 and W N( q) N( q) AVR( q) N( q) 1 NMRR( q) 1 W N( q) 1 () ANMRR Q q1 NMRR( q) 1 Q where N(q) is the minimum number of reevant (via ground-truth obtained by an expert in the fied) images in a set of Q retrieva experiments, R() is the ran of the th reevant retrieva within a window of W retrievas, which are taen into consideration during each query, q. If there are ess than N(q) reevant retrievas among W then a ran of W+1 is assigned for the remaining (missing) ones. AVR(q) is the average ran obtained from the query, q. Note that if the first N(q) retrievas are a reevant, then NMRR(q)=0, and the best retrieva performance is thus achieved. On the other hand, if no reevant item is retrieved among W then NMRR(q)=1, as the worst case. Therefore, the ower NMRR(q) is, the better (more reevant) the retrieva for the query, q. ANMRR is just the average of NMRR scores for a queries. Aong with ANMRR, we aso used average precision (AP) measure and both measures are computed querying a (1) images in the database within a retrieva window equa to the number of ground truth images, N(q) for each

25 CLASSIFICATION AND RETRIEVAL ON MACROINVERTABRATE IMAGE DATABASES query q. This henceforth maes the AP identica to average Reca and average F1 measures, too. With the traditiona approach (without cassifier), we obtain ANMRR = 0. and AP = 0.1, indicating in fact a quite poor retrieva performance due to the imited discrimination power of the simpe features used. With the use of the cassifier, the retrieva performance has been improved to the eve of ANMRR = 0.0 and AP = 0.. This eventuay presents such a deicate soution that matches and even may surpass the accuracy of human experts when identifying macroinvertebrate specimens. Traditiona qa With Cassifier Traditiona qb With Cassifier Traditiona qc With Cassifier Traditiona qd With Cassifier Figure : Four sampe queries of Baetis rhodani (qa), Hydropsyche peucidua (qb), Heptagenia suphurea (qc) and Rhyacophia nubia (qd) with (right) and without (eft) using cassifier. Top-eft is the query image as we as the first retrieved image. Each irreevant retrieva is mared with a red For visua evauation, Figure presents four typica retrieva resuts (from the snapshot of the first page of MUVIS-MBrowser appication) with and without using the proposed cassifier. Note that the retrieva resuts with the traditiona approach are highy erroneous since many members of irreevant casses are retrieved even within the first 1 rans. With the use of cassifier, a

26 CLASSIFICATION AND RETRIEVAL ON MACROINVERTABRATE IMAGE DATABASES retrievas in the first page (as we as in the second and third pages that are not shown therein due to space imitations) are reevant. V. CONCLUSIONS In this paper, we addressed the probem of cost-intensive manua taxonomic cassification and retrieva of macroinvertebrate specimens by investigating the best cassifier among powerfu state-of-the-art automatic cassifiers. Among many aternatives, in order to demonstrate the efficiency of the cassifier and to propose a ow cost soution, we have intentionay extracted basic and simpe features from the Benthic macroinvertebrate images. With the proper normaization, the deterministic cassifiers, SVMs and BCs achieved training and test CEs much ower than %. Particuary SVMs performed quite we in the training sets, i.e. ~1% CE. The performance of both types of ANN cassifiers was evauated by using a nove assessment methodoogy, which can evauate training (error minimization) and test (generaization) performances of both ANN types, MLPs and RBFNs, with respect to different networ configurations whist considering two statistica criteria, the mean and the minimum errors within a certain number of training runs. The experimenta resuts ceary demonstrate that both training and generaization performance depend on the networ configuration and the data partitioning between training and test sets. The fact that most of the earier comparative studies evauate networ performances over ony one or few networ configurations, may ead to the erroneous seection of either of the ANN types as the winner for any given probem. The proposed method for performance assessment of ANN cassifiers can aso be used for other types of ANNs with different training techniques and in other appication areas. This is subject to our future wor, which wi focus on comparison not ony the ANN types but aso different training methods such

27 CLASSIFICATION AND RETRIEVAL ON MACROINVERTABRATE IMAGE DATABASES as BP versus evoutionary agorithms, particuary Genetic Agorithms and Partice Swarm Optimization []. With the proposed in-depth assessment, the best cassifier in terms of test (vaidation) cassification performance is obtained among MLPs and then used for the purpose of improving simiarity-based retrievas within the MUVIS framewor. The retrieva resuts from the extensive query experiments show that a high performance in retrieva accuracy is achieved, particuary when compared to the traditiona (without cassifier) retrieva methodoogy. REFERENCES [1] P. F. Cuverhouse, R. G. Simpson, R. Eis, J. A. Lindey, R. Wiiams, T. Parisini, B. Reguera, I. Bravo, R. Zoppoi, G. Earnshaw, H. McCa and G. Smith, Automatic cassification of fied-coected dinofageates by artificia neura networ, Marine Ecoogy Progress Series, vo, pp.1-,. [] K.J. Gaston, and M.A. O'Nei, Automated species identification: why not?, Phiosophica Transactions of the Roya Society of London Series B, vo., pp. -, 00. [] M. Benfied, (and 1 others) RAPID: Research on Automated Panton Identification, Oceanography, vo. 0, no., pp. 1-, 00. [] N. Larios, H. Deng and W. Zhang, Automated insect identification through concatenated histograms of oca appearance features, Machine Vision and Appications, vo., pp. -1, 00. [] V. Tirronen, A. Caponio, T. Haanpää and K. Meissner, Mutipe order gradient feature for macroinvertebrate identification using support vector machines, Lecture Notes in Computer Science, in Press. [] P.F. Cuverhouse, R. Wiiams, B. Reguera, V. Herry and S. Gonzaes-Gi, Do experts mae mistaes? A comparison of human and machine identification of dinofageates, Marine Ecoogy progress Series, vo., pp. 1-, 00. [] L. Bertei, R. Cucchiara, G. Paternostro and A. Prati, A semi-automatic system for segmentation of cardiac M-mode images, Pattern Anaysis and Appications, vo., pp. -0, 00. [] Y. P. Ginoris, A. L. Amara, A. Nicoau, M. A. Z. Coeho and E. C. Ferreira Recognition of protozoa and metazoa using image anaysis toos, discriminant anaysis, neura networs and decision trees, Anaytica Chimica Acta, vo., pp. 1-1, 00. [] S. Hayin, Neura Networs: a Comprehensive Foundation, Prentice ha, USA, June. [] T. Poggio and F. Girosi, A theory of networs for approximation and earning, A.I. Memo No., M.I.T. A.I Lab,. [] Lu Yingwei, N. Sundararajan and P. Saratchandran, Performance evauation of a sequentia minima radia basis function (RBF) neura networ earning agorithm, IEEE Transactions on Neura Networs, vo., no., pp. 0, March. [1] L. Özyimaz, T. Yidirim and K. Kou, Comparison of Neura Networs for function Approximation, Paistan Journa of Appied Sciences, vo., no., pp. -, 00. [1] D. Kovacevic and S. Loncaric, Radia Basis Function-based Image Segmentation using a Receptive Fied, in Proc. of the Tenth IEEE Symposium on Computer-Based Medica Systems, pp. 1-, Sovenia, June. [1] R.A. Finan, A.T. Sapeu and R.I. Damper, Comparison of Mutiayer and Radia Basis Function Neura Networs for Text-Dependent Speaer Recognition, in Proc. of IEEE Int. Conf. on Neura Networs, vo., pp. -, USA, June. [] K.P. Sudheer and S. K. Jain, Radia Basis Function Neura Networ for Modeing Rating Curves, Journa of Hydroogic Engineering, Vo., No., pp. 1-1 May/June 00. [1] S. Gopa and M. Fischer, A Comparison of Three Neura Networ Cassifiers for Remote Sensing Cassification, in Proc. of Internationa Geoscience and Remote Sensing Symposium IGARSS, vo. 1, pp. -, USA, May. [1] B. A. Hawichorst, S. A. Zahorian, and R. Rajagopa, A Comparison of Three Neura Networ Architectures for Automatic Speech Recognition, in Proc. of the Artificia Neura Networs in Engineering Conference, Inteigent Engineering Systems Through Artificia Neura Networs, vo., pp. 1-, USA, November.