How To Predict Chlorophyll A

Transcription

1 Ecological Modelling 146 (2001) Predicting chlorophyll-a in freshwater lakes by hybridising process-based models and genetic algorithms P.A. Whigham a, *, Friedrich Recknagel b a Department of Information Science, Uni ersity of Otago, PO Box 56, Dunedin, New Zealand b Department of Soil and Water, Adelaide Uni ersity, Waite Campus, Glen Osmond, Adelaide 5064, South Australia, Australia Abstract This paper describes the application of several machine learning techniques to modify a process-based difference equation. The original process equation was developed to model phytoplankton abundance based on measured limnological and climate variables. A genetic algorithm is shown to be capable of calibrating the constants of the process model, based on the data describing a lake environment. The resulting process model has a significantly improved performance based on unseen test data. A symbolic genetic algorithm is then applied to the process model to evolve new expressions for the grazing term of the equation. The results indicate that this approach can be used to explore new process formulations and to improve the generalisation and predictive response of process models Elsevier Science B.V. All rights reserved. Keywords: Chlorophyll-a prediction; Process-based model; Genetic algorithm; Time series model 1. Introduction This paper describes the application of a process-based model to predict the timing and magnitudes of algal blooms for Lake Kasumigaura, in the South-Eastern part of Japan. The process model is calibrated using a genetic algorithm, and subsequently portions of the model are modified using a symbolic learning system. This data has previously been studied using an artificial neural network (Recknagel, 1997; Recknagel et al., 1997, 1998; Liu and Yao, 1999) that demonstrated the * Corresponding author. Tel.: ; fax: address: pwhigham@infoscience.otago.ac.nz (P.A. Whigham). potential for these tools to predict highly non-linear phenomena such as blue-green algal blooms in freshwater lakes. Although neural networks have produced accurate predictive models for this dataset it is difficult to use them for extracting process knowledge of the system. The purpose of this paper is to explore the combination of machine learning techniques and process-based descriptions to develop better predictive models and explore extensions to process understanding in freshwater systems Lake Kasumigaura dataset Lake Kasumigaura is situated in the South- Eastern part of Japan. It is a large, shallow water body where no thermal stratification occurs. Wa /01/$ - see front matter 2001 Elsevier Science B.V. All rights reserved. PII: S (01)

2 244 P.A. Whigham, F. Recknagel / Ecological Modelling 146 (2001) ter temperatures vary widely, from 4 C in the winter to 30 C in summer. The lake has high external and internal nutrient loadings and therefore primary productivity is high. A number of climatic and limnological variables have been collected over a 10 year period ( ) for Kasumigaura (Takamura et al., 1992), as shown in Table 1. A simple linear interpolation has been used to fill missing values to produce a complete daily time series for this period. For this study the last 5 years of data ( ) have been selected, since they have some regularities in terms of chlorophyll-a, and include the largest peak concentration measured over the 10 year period The ecology of freshwater phytoplankton Phytoplankton includes representatives of several groups of algae and cyanobacteria. They are usually distinguished by being freely floating and dependent on water movement for maintenance and transport (Reynolds, 1984). Many factors affect their population dynamics and they vary depending on the type of phytoplankton under consideration. However, all algae species rely on light as a basic input for photosynthesis and require nutrients such as nitrogen and phosphorus for growth and reproduction. Factors such as water temperature, turbidity, mixing, competition and grazing are also relevant to the population dynamics of algae. Even though much work has been done on phytoplankton, there are still difficulties with developing reliable predictive models for algal growth. There are several reasons for this, including the highly non-linear behaviour of the population, the inherent errors associated with data collection, the time scales for data collection Table 1 Factors measured with the daily time series data Measured factor Average S.D. Units Ortho-phosphate (p) mg/l Solar radiation (l) MJ/m Water temperature (t) C Copepoda (cop) ind/l Cladocera (clad) ind/l chlorophyll-a (chla) g/l versus the possible time scales of the system, and the complex, dynamic behaviour of aquatic ecosystems. 2. Genetic algorithms and genetic programming Originally formalised by Holland (Holland, 1975), genetic algorithms (GA s) are general optimisation techniques based on using a population to search for good solutions. Standard GA s use a bit string to represent each member of the population, which are then decoded to give the values of the variables being optimised for the problem. These population-based techniques have been successfully applied to many different problems (Grefenstette, 1985; Cleveland and Smith, 1989; Goldberg, 1989; Bennet et al., 1991; De Jong et al., 1993), and are generally applicable to problems where there is little knowledge available about the form of the best solution. The main search operators used with GA s are crossover, where two parent bit-strings are combined to produce two children, and mutation, where random elements in a bit-string are flipped from a zero to one or vice-versa. The driving force behind GA s (and all evolutionary systems) is the use of a fitness measure to bias selection of individuals. Population members that are fitter are more likely to be selected for breeding and therefore good partial solutions are propagated throughout the population. Genetic Programming (GP) extended the concepts of GA s by allowing a more flexible representation for the population members (Koza, 1992). Each member of the population in GP is a functional program, represented as a tree. The search operators of crossover and mutation act on these trees, allowing the possible solutions to expand and grow during the evolution. One particular extension to GP uses a context-free grammar (CFG) to represent the language that can be expressed by the functional programs (Whigham, 1995, 1996; Whigham and Crapper, 1999; Whigham, 2000) and has been successfully applied to several different spatial and temporal problems. This system, entitled CFG GP, allows the form of any program to be controlled by a grammar, and therefore the system allows the user to search within a con-

3 P.A. Whigham, F. Recknagel / Ecological Modelling 146 (2001) Fig. 1. Prediction using difference equation (training RMSE=87.3, test RMSE=82.7). strained representation of the problem. CFG GP will be used in this paper to explore several forms of a partial extension to a process-based difference equation. 3. Training and test data setup Table 1 shows the measured variables used for developing the models. For all experiments, 2 years of daily data ( ) were used for training and 3 years of daily data ( ) used for testing the generalisation behaviour of the resulting equations. The root mean square error (RMSE) was used as the fitness function for the training data and as a measure of accuracy for the test data. A lower RMSE was taken to indicate a better prediction of the test data. When comparing two different learning techniques a lower RMSE for the unseen (test) data implied that the learning system had better generalised the patterns found in the training data. 4. Process-based modelling A difference equation for algal growth Eq. (1) was originally developed as part of a module of the lake ecosystem model SALMO (Recknagel and Benndorf, 1982) based on the theoretical behaviour of freshwater systems and validated using laboratory experiments and field data. Using this original equation the prediction for the training and test periods is shown in Fig. 1. The RMSE for the period was 87.3 and for was Chla t+1 =Chla t +Chla t *(Phot Re sp) Chla t *(Cop+Clad)*0.0001, (1) where Phot=(0.068*T)* 0.025*L *L * P 1.7 Chla t X + P X P Chla t Chla t X=5.76*Chla t 0.41,

4 246 P.A. Whigham, F. Recknagel / Ecological Modelling 146 (2001) and Resp=( *T) +0.3*Phot Using genetic algorithms to calibrate the process model In an attempt to improve the performance of Eq. (1) the constants were tuned based on the training data. Each constant could vary within a range of 20% based on the 2 years of training data. The optimal settings for the constants were searched using the Genetic Algorithm Optimisation Toolbox for Matlab 5. The parameters for this GA used a population of 100, with a simple binary crossover (90%) and binary mutation (5%). The selection scheme used was roulette wheel, and the GA was run for 150 generations. This setup was run 20 times, with the final solution selected from the lowest training result. The RMSE was used as the fitness measure, however only values of chlorophyll-a that were above 75 g/l contributed to the error measure in an attempt to force the equation to better model the peak events. The prediction for the test and training period is shown in Fig. 2 and the modified equation is shown as Eq. (2). This equation had a dramatically improved RMSE of 27.6 for the training period and 31.9 for the 3 years of test data. Chla t+1 =Chla t +Chla t *(Phot Resp) Chla t *(Cop+Clad)* , (2) where Phot=( *T)* *L *L * P 1.36 Chla t X + P X P Chla t Chla t Resp=( *T) *Phot, and X=4.608*Chla t Fig. 2. Prediction of chlorophyll-a using a difference equation with constants calibrated using the training data (training RMSE=27.6, test RMSE=31.9).

5 P.A. Whigham, F. Recknagel / Ecological Modelling 146 (2001) The success of this equation demonstrates that the constants of process-based models may be successfully calibrated to conditions in specific freshwater lakes using simple machine-learning techniques. The ability to also search for new constant values within some range (in this case 20%) is a useful method for constraining the search within limits that are physically meaningful Learning modifications to the process-based model This section will consider using CFG GP to extend the process-based model by searching for new representations of one component of the process model, a single component at a time. For example, the photosynthesis, respiration or grazing terms could be evolved within the original process-based model framework, while keeping the other factors in the model constant. To demonstrate this concept, the grazing term from Eq. (2) will be evolved within the original model. The purpose is to search for a better representation of the grazing term in relation to the theoretically and experimentally determined process model. The original grazing term including parameter calibration was ( chla t *(cop+ clad)* ). Two approaches will be described; the first will evolve a grazing term of the form (chla t * (cop,clad)); the second will allow the chla t variable to be used more then once in the grazing term, in other words, the grazing term is a function of all three variables: (chla t,cop,clad). Note that both equations do not allow the use of the minus operator, to ensure that the grazing term is always positive and therefore, when substracted, always lowers the predicted chla t+1 concentration. The following grammars represent the two grazing equations that will be evolved.

6 248 P.A. Whigham, F. Recknagel / Ecological Modelling 146 (2001) Fig. 3. Chlorophyll-a prediction using G grazing1. Both equations were evolved using a population size of 1000 and evolved for 50 generations. Crossover was set to 90% and mutation 5% over the non-terminal GT. Each setup was run 20 times, and the best result based on the training data, over the 20 runs, selected as the solution. The fitness measure used was RMSE, and the evaluation of the grazing term was achieved by incorporating Eq. (2), with the grazing term removed, into the fitness evaluation. Although it was possible to define Eq. (2) directly in the grammar, and let the system just evolve new terms for grazing, incorporating Eq. (2) into the CFG GP program lowered the evaluation time for the program Resulting grazing equations The best solution for G grazing1 had a training error of 28.5 and a test error of Note that these results are comparable with the original calibrated difference equation. Since it was not possible to significantly improve on the original equation, when constrained to only use Copepoda and Cladocera as variables, it is possible to conclude that this form of equation is an appropriate formalisation of the model. The grazing term produced using G grazing1 was: chla t * *[2cop+clad X] (3) X=( clad cop)cosh clad cop. Eq. (3) is basically the same form as the original grazing term and can be simplified to chla t * *(2cop+clad), since the cosh terms are multiplied by a factor of and therefore can be ignored. The resulting prediction is shown in Fig. 3. The main difference compared with the original equation is the increased dominance of the Copepoda term. Note that the constant appears a number of times in Eq. (3). This is a result of the crossover operator propagating useful sub-expressions, which contain this constant, through a number

7 P.A. Whigham, F. Recknagel / Ecological Modelling 146 (2001) of generations to construct the solution. Because the constant was associated with good partial solutions it has spread throughout the population. In terms of producing process equations this is a useful side effect of the technique, since it allows general constants in an equation to be discovered, which may have some fundamental meaning for the system. The best solution for G grazing2 had a training error of and a test error of 29.63, which is an improvement over the original, calibrated process model. The resulting predicted curve is shown in Fig. 4. The grazing term produced using G grazing2 is shown in Eq. (4). exp(chla t /18.03) cop. (4) (18.03/cla t)+(clad/cop)+clad+cop (28.86/cla t )+chla t Note that the constant is repeated in this equation, and an interesting question arises as to whether should also replace This would indicate a general constant in the grazing term that could be given some meaning related to how the system is functioning. These considerations, however, are beyond the scope of this preliminary work. Eq. (4) has also used the chla t term several times. Since this formalisation was not part of Eq. (2) it has extended the possible interpretation of the grazing term. The form of Eq. (4) is reminiscent of hyperbolic/inverse hyperbolic relationships that are quite common in multiple resources dynamics. This is a promising outcome of the work and shows that it may be possible to use this datadriven approach to reconstruct and extend theories regarding freshwater ecosystem dynamics. 5. Discussion The previous studies have demonstrated that models can be developed for the non-linear dynamics of phytoplankton. The use of a genetic algorithm to calibrate the constants of the process-based equation has been shown to significantly improve the predictive performance of the model. Additionally, the use of an extended genetic programming system has been used to evolve new components of the process equation, improving the overall performance on unseen data. This is a promising area of work that should allow the exploration of new formulations of photosynthesis, respiration and grazing terms for process-based equations. Although this approach is not likely to produce the best predictor (Liu and Yao, 1999) based on the training and test data, it shows the possibility of allowing better process models to be produced and therefore to extend our knowledge of the underlying dynamics of ecological systems. In that respect, it is a complimentary approach to other machine learning and hybrid systems, since the combination of different techniques can help support new descriptions and developments in a fundamental manner. Several more general issues have been highlighted by this work. The root mean square error was used as the fitness function, however this is essentially a non-temporal measure for time series since it does not take into account the shape of the curves that are being compared (Whigham and Aldridge, 2000). Other error metrics for time series, which take into account the shape of the series, may improve the developed models by changing the shape of the fitness landscape and driving the system towards better partial solutions. In particular, previous work in time series similarity and data mining have shown that RMSE is not always the most appropriate measure for comparing a predicted and actual time series (Keogh and Pazzini, 1999). Further work is required to determine whether better similarity measures would improve the development of these process-based equations, for both the constant calibration and the CFG GP approaches. This work has not attempted to place any physical meaning to the evolved expression for grazing. Rather, the fact that the new grazing term improves the prediction on the test data has been used to demonstrate that improvements for some portion of the process model are possible. Future work is required to explore

8 250 P.A. Whigham, F. Recknagel / Ecological Modelling 146 (2001) Fig. 4. Chlorophyll-a prediction using G grazing2. whether using this approach to gradually evolve small modifications to the current process model can extend process understanding, however the initial results presented here are very promising. The general formulation of Eq. (1), with some extensions, can also be applied to individual algal species. Future work will investigate the use of GA s to calibrate the constants of this equation for predicting various algal species dynamics in different freshwater lakes. By comparing the evolved constants within each equation the driving factors for the behaviour of each species should be able to be inferred and perhaps generalised by means of data from different lakes. Comparing the inferred behaviour with the known dynamics of each species will help to understand how a particular population of algal species behaves under the physical constraints of a variety of freshwater lakes. 6. Conclusion The calibration of a deterministic process-based model using genetic algorithms has been demonstrated. The dramatically improved performance of the calibrated model indicates that GA s are an appropriate method for tuning the constants of a process-based model. The application of a symbolic machine learning system has also been shown for exploring modifications to the process model, by evolving a new term for one particular component of the system. The success of this approach indicates that it is possible to explore process descriptions in an incremental fashion. Areas for future research have also been highlighted. References Bennet, K., Ferris, M.C., Ioannidis, Y.E., A genetic algorithm for database query optimization. In: Belew, R.K., Booker, L.B. (Eds.), Proceedings of the Fourth International Conference on Genetic Algorithms Cleveland, G.A., Smith, S.F., Using genetic algorithms to schedule flow shop releases. In: Schaffer, J.D. (Ed.), Proceedings of the Third International Conference on Genetic Algorithms De Jong, K., Spears, W., Gordon, D., Using Genetic Algorithms for Concept Learning. Machine Learn. 13, 5 29.

9 P.A. Whigham, F. Recknagel / Ecological Modelling 146 (2001) Goldberg, D.E., Genetic Algorithms in Search, Optimization, and Machine Learning. Addison-Wesley. Grefenstette, J.J., Optimization of control parameters for genetic algorithms. IEEE Trans. Syst., Man Cybern. 16 (1), Holland, J.H., Adaptation in Natural and Artificial Systems. The University of Michigan Press, Ann Arbor, MI. Keogh, E.J., Pazzini M.J., Relevance Feedback Retrieval of Time Series Data. The 22nd Annual international ACM-SIGIR conference on research and development of information retrieval. Koza, J.R., Genetic Programming: On the Programming of Computers by means of Natural Selection. MIT Press, Cambridge, MA. Liu, Y., Yao, X., Time Series Prediction by Using Negatively Correlated Neural Networks. Lecture Notes in Artificial Intelligence, vol Springer-Verlag, Berlin, pp Recknagel, F., ANNA Artificial Neural Network model for predicting species abundance and succession of blue-green algae. Hydrobiologia 394, Recknagel, F., Benndorf, J., Validation of the ecological simulation model SALMO. Int. Revue ges. Hydrobiol. 67 (1), Recknagel, F., French, M., Harkonen, P., Yabunaka, K., Aritificial neural network approach for modelling and prediction of algal blooms. Ecol. Model. 96 (1 3), Recknagel, F., Fukushima, T., Hanazato, T., Takamura, N., Wilson, H., Modelling and prediction of phyto- and zooplankton dynamics in Lake Kasumigaura by artificial neural networks. Lakes and Reservoirs: Research and Management 3, Reynolds, C.S., The Ecology of Freshwater Phytoplankton. Press Syndicate of the University of Cambridge, New York. Takamura, N., Otsuki, A., Aizaki, M., Nojiri, Y., Phytoplankton species shift accompanied by transition from nitrogen dependence to phosphorus dependence of primary production in Lake Kasumigaura. Arc. Hydrobiol. 124, Whigham, P.A., Grammatically-based Genetic Programming. In: Rosca, J. (Ed.), Proceedings of the Workshop on Genetic Programming: From Theory to Real-World Applications Whigham, P.A., Search bias, language bias and genetic programming. Genetic Programming 1996: In: Koza, J.R., Goldberg, J.R., Fogel, D.E., Riolo, R.L. (Eds.), Proceedings of the First Annual Conference, MIT Press, Stanford University, Cambridge, MA. Whigham, P.A., Induction of a marsupial density model using genetic programming and spatial relationships. Ecol. Model. 131 (2 3), Whigham, P.A., Aldridge C., A shape metric for evolving time series models, Information Science Department Discussion Series, University of Otago. Whigham, P.A., Crapper, P.F., Time series modelling using genetic programming: an application to rainfallrunoff models. In: Spector, L., Langdon, W.B., O Reilly, U., Angeline, P.J. (Eds.), Advances in Genetic Programming 3, vol. 5. MIT Press, Cambridge, MA, USA, pp