Software for the analysis of extreme events: The current state and future directions

Extremes (2006) 8: 87 109 DOI 10.1007/s10687-006-7962-0 Software for the analysis of extreme events: The current state and future directions Alec Stephenson & Eric Gilleland Received: 29 October 2005 / Revised: 29 October 2005 / Accepted: 22 November 2005 # Springer Science + Business Media, LLC 2006 Abstract The last few years have seen a significant increase in publicly available software specifically targeted to the analysis of extreme values. This reflects the increase in the use of extreme value methodology by the general statistical community. The software that is available for the analysis of extremes has evolved in essentially independent units, with most forming extensions of larger software environments. An inevitable consequence is that these units are spread about the statistical landscape. Scientists seeking to apply extreme value methods must spend considerable time and effort in determining whether the currently available software can be usefully applied to a given problem. We attempt to simplify this process by reviewing the current state, and suggest future approaches for software development. These suggestions aim to provide a basis for an initiative leading to the successful creation and distribution of a flexible and extensible set of tools for extreme value practitioners and researchers alike. In particular, we propose a collaborative framework for which cooperation between developers is of fundamental importance. Keywords Extreme value theory. Software development. Statistical computing AMS 2000 Subject Classification Primary 62P99 A. Stephenson (*) Department of Statistics and Applied Probability, Faculty of Science, National University of Singapore, Singapore 117546, Singapore e-mail: stasag@nus.edu.sg E. Gilleland Research Applications Laboratory, National Center for Atmospheric Research, 3450 Mitchell Lane, Boulder( CO 80301, USA e-mail: ericg@ucar.edu

88 A. Stephenson, E. Gilleland 1. Introduction In many diverse areas of application the extreme values of a process are of primary importance. Internet traffic, rainfall, material strength, pollutant concentrations and insurance claim sizes are all examples of processes of this type. A number of different techniques can be justifiably employed to analyse the extremes of one or more processes, and the need for software for both presentation and evaluation is common to all. This typically takes place on an individual level, where code is written to satisfy the objectives for the specific task at hand. The code may only be applicable to a particular dataset, or may be targeted at a specialised methodology. It would not therefore be suitable for similar problems, and would be of limited use if distributed to a wider community. In some cases though, code is developed or altered to be more general in scope, so that it implements a broader methodology, or can be applied to datasets of a certain form or class. Such code could be supplemented by some form of documentation and be made publicly available as a packaged unit. It is on this basis that the majority of the software introduced in Section 2 has been developed. The development and distribution of software involves important decisions with regard to elements such as; the programming language or languages and the related environment, the incorporation of external routines or algorithms, the license on release, methods of documentation, the programming interface, and more generally the overall design strategy. The consideration of these issues should be made in the context of the objectives of the project or packaged unit, and preferably prior to development, thus encouraging well designed software. Such issues are discussed further in Section 3. Because much of the software of Section 2 has been initiated through code written for the purpose of an individual, the decisions outlined in the previous paragraph have largely been based on personal preference. Furthermore, the packaged units of publicly available software are spread about the statistical landscape, requiring scientists seeking to apply extreme value methods to spend considerable time and effort in determining whether the currently available software may be usefully applied to a given problem. Moreover, certain statistical techniques are independently implemented in more than one packaged unit, and there often exists slight differences between implementations with respect to both reliability and methodology. We believe that the creation of a software initiative to develop a reliable set of extreme value routines will be of considerable benefit to the extreme value community. It will also help to promote the methodology to practitioners from a wide range of scientific disciplines. Software developers can play a key role in the success of such an initiative, so the issue of how to attract and encourage developers is critical. In Section 2 we provide an overview of existing publicly available software for the analysis of extreme values, including a summary of the statistical techniques that are available in each distributed unit. In Section 3 we discuss possible future directions for the development of software for the analysis of extreme values. We propose that a collaborative effort be made, and in order to develop flexible and extensible software, that an open source computing environment be the basis for this effort. In Section 4 we discuss incentives for software developers and comment on funding issues. A concluding discussion is given in Section 5.

Software for the analysis of extreme events: The current state and future directions 89 2. The current state In this section we survey the current software available for the analysis of extreme values. We make no guarantee that all publicly available software will be covered, but all that is known to the authors will be addressed, and we make the presumption that this accounts for all software in common use at the time of writing. We begin by briefly reviewing standard extreme value techniques, in order to provide more clarity when discussing different packaged software units. The categorisation of different methodologies essentially follows Beirlant et al. (2004), but here we give more emphasis to those methods which are implemented more commonly in available software. Alternative texts covering the techniques outlined here include Coles (2001), Reiss and Thomas (2001) and Embrechts et al. (1997). We then summarise the different packages available and discuss the techniques that they implement. We begin this discussion with those units that are global in their aims, and go on to discuss those units that focus on specific aspects of extreme value methodology. We underline that the summary of the utility of software units in terms of the statistical techniques they perform is merely a summary, and that detailed discussions of specific implementations of methodology are sacrificed in order to obtain a simplified overarching view of what is available to the practitioner. 2.1. Extreme value techniques 2.1.1. Classical methodology Let X 1 ; ; X n be a random sample from a distribution F, and let M n ¼ maxfx 1 ; ; X n g. Extreme value theory specifies the class of non-degenerate limiting distributions for the maximum M n under linear normalisation. This class is given by the parametric form h i GðxÞ ¼exp f1 þ ðx Þ=g ; ð2:1þ where, > 0 and are the location, scale and shape parameters, respectively, and z þ ¼ maxðz; 0Þ. G is known as the generalised extreme value distribution; the Gumbel distribution obtained in the limit as! 0. Larger values for yield heavier tailed distributions. The domain-of-attraction problem considers the type of distributions F that lead to particular forms of G. It transpires that the sign of is the dominating factor; for >0 convergence is obtained if and only if 1 FðxÞ ¼ x 1= ðxþ, where ðþ satisfies the slowly varying property that the limit as x!1of the ratio ðxtþ= ðxþ is one for all t > 0. Similar conditions can be obtained in the cases ¼ 0 and <0. The above result forms the basis for a number of different statistical methods. For large N, the generalised extreme value distribution approximates the distribution of M N. Given a dataset of maxima, this approximation can be treated as exact, and hence the distribution can be fitted to the data by estimating the parameters ð; ; Þ. This is commonly known as the block maxima approach since it can be employed after dividing the sample X 1 ;...; X n into blocks and considering the maxima derived from each block. The estimation can be performed by maximum likelihood, and likelihood inferential methods can subsequently be used. The likelihood problem is non-regular since the end points of the generalised þ

90 A. Stephenson, E. Gilleland extreme value distribution depend on the parameters, but it can be shown (Smith, 1985) that the usual properties exist when > 0:5, which is typically the case for statistical applications. Alternative estimation methods are possible, and of these, the probability weighted moments method of Hosking et al. (1985) is perhaps the most popular. This method has desirable properties, but it does not easily extend to regression type models which we discuss subsequently. Estimation and inference under the Bayesian paradigm can also be performed (e.g., Stephenson and Tawn, 2004). Given any set of parameter estimates, extreme quantiles of the maximum distribution can be estimated by inverting the estimated generalised extreme value distribution function. Extreme quantiles of the distribution F can be estimated similarly on the basis that F n approximates G. Although the above result focuses on maxima, it leads to an approximation to the conditional exceedance probability PrðX > xjx > uþ for all x greater than some predetermined large threshold u. Specifically, the generalised Pareto distribution (Pickands, 1975) can be obtained as an approximation of the conditional distribution of exceedances. For statistical inference this distribution is often treated as exact, and hence the data that exceed u can be used to derive estimates of the generalised Pareto parameters ð u ;Þ. Maximum likelihood or probability weighted moments estimates can again be used. The conditioning probability PrðX > uþ can be estimated using the empirical proportion of exceedances of u, and extreme quantiles can then be estimated by simply inverting the corresponding model for the unconditional tail probability PrðX > xþ. This technique is known as the peaks over threshold (POT) method. It should be observed that the generalised Pareto scale parameter u depends on the threshold u. An alternative point process representation typically uses a different parameterisation for which this is not the case, and hence it is preferable in statistical applications to use this representation whenever regression type models, to be discussed subsequently, are employed. For the moment we assume that >0, and we return to the domain of attraction condition 1 FðxÞ ¼x 1= ðxþ. There are a number of statistical methods, and in particular estimates of, based on such conditions. Let X 1;n... X n;n denote the order statistics of X 1 ;...; X n, so that M n ¼ X n;n. Using the domain of attraction condition the conditional tail probability PrðX=u > yjx > uþ for large u and y > 1is approximately equal to y 1=. The maximum likelihood estimate for based on the exceedances of u has a closed form, and for u equal to the kth largest order statistic X n k;n it is given by the Hill estimator (Hill, 1975) ^ H ¼ 1 k X k j¼1 log X n jþ1;n log X n k;n : ð2:2þ The corresponding tail model is a POT model with >0 and u ¼ u, and therefore the Hill estimator can be obtained as a maximum likelihood estimator in a constrained POT analysis with threshold u ¼ X n k;n. The Hill estimator is a popular estimator with a simple form. However, it is not location invariant and it can be severely biased if the distributional approximation is poor. These weaknesses have led to proposals of a large number of estimators which can be seen as variants of ^ H ; see Chapter 4 of Beirlant et al. (2004). We have thus far assumed that >0. More general domain of attraction conditions for the case of 2 R can also be used to construct estimators. One such simple estimator was proposed by Pickands (1975), and many variants and alternatives have been suggested. A review of such

Software for the analysis of extreme events: The current state and future directions 91 estimators and associated estimators of extreme quantiles is given in Chapter 5 of Beirlant et al. (2004). Common to all methods based on the full sample X 1 ;...; X n is the choice of a threshold u, or equivalently an integer k such that u ¼ X n k;n. Generally estimates are derived for all reasonable values of u or k, and then a plot of these estimates against u or k can be constructed. One common method is to then choose the threshold as the lowest value above which the estimates are roughly constant. In a POT analysis using the generalised Pareto representation, the parameter u is expected to depend on the threshold, so it is typical to instead plot the threshold independent quantity u u. In all cases the variance of each estimate must be taken into consideration. At higher thresholds the variance becomes much larger because fewer data points are used. This often makes such plots difficult to interpret. 2.1.2. Extended methodology Suppose we have a stationary sequence X 1 ; X 2 ;... of random variables with common marginal distribution function F. The block maxima approach can be applied under the weaker stationary assumption provided that a long range dependence constraint (e.g., Leadbetter et al., 1983) is assumed. The dependence affects the parameter estimates, and the extent of this effect can be summarised by the extremal index 0 1, with ¼ 1 for both independent processes and dependent processes satisfying a short range dependence restriction. In the stationary case it is F n, and not F n, that is approximated by G. Extending methods based on threshold exceedances to stationary processes is more difficult. One approach is to construct a formal model for the process, or more specifically for the exceedances of the process. The data can then be used to estimate the model parameters. Markov chain model structures are commonly assumed. Alternative methods typically involve identifying clusters of exceedances. One common method is to take the maximum of each cluster, and perform a POT analysis on the cluster maxima. Estimates of quantiles and other derived quantities must be adapted to incorporate and hence account for the dependence. Estimates of can be based on its interpretation as the reciprocal of the mean cluster size in the limit as u!1. The runs method and the blocks method are two popular methods of estimation. We now suppose we have covariate information, and we wish to include such information in our model. A typical approach to the inclusion of covariate information is to impose regression models directly on the parameters ð; ; Þ of the generalised extreme value distribution or of the point process representation of the POT method. For example, in the block maxima case, it may be prudent to model the ith annual maximum as generalised extreme value with parameters ð i ;;Þ with i ¼ 0 þ 1 z i, where z ¼ðz 1 ;...; z n Þ are n observations on a single covariate, such as the year of observation, and i ¼ 1;...; n. The parameter set is therefore extended from ð; ; Þ to ð 0 ; 1 ;;Þ, and the four parameters can be easily estimated using maximum likelihood. The regression models for ð; ; Þ can naturally be extended to incorporate multiple covariates. The form of the models can also be extended to generalised linear types, or to include non-parametric or semi-parametric terms. The multivariate extension of the limiting distribution (2.1) focuses on componentwise maxima. Briefly, extreme value theory specifies the class of d-dimensional limiting distributions for the componentwise maxima under linear normalisations of

92 A. Stephenson, E. Gilleland each component. This class is known as the class of multivariate extreme value distributions. The form of the dependence structure has no finite parameterisation, see e.g., Chapter 8 of Beirlant et al. (2004) for details and domain of attraction conditions. The asymptotic result leads to a multivariate extension of the block maxima approach whereby the multivariate extreme value distribution G replaces the generalised extreme value distribution. Since there is no finite parameterisation, a parametric model for the dependence structure is typically assumed. Alternatively, non-parametric estimates can be used. The multivariate extension to POT methods is not so obvious. There are two common approaches, one based on a point process approach (Coles and Tawn, 1991) and one based on a censored likelihood (Ledford and Tawn, 1996). One difficulty with using extreme value distributions as the basis of an inferential framework is that asymptotic independence cannot be obtained if dependence exists at finite levels (Coles et al., 1999). Model summaries and extensions that attempt to account for this are given in Chapter 9 of Beirlant et al. (2004). Available software does not currently allow for models that account for asymptotic independence, and so we do not discuss this issue further. 2.2. Software summary We now summarise the different packaged units available, in no particular order of preference. In order to maintain the flow of the discussion, a glossary is provided as an appendix, which contains explanations of acronyms and technical computing terms, details of how software may be obtained, and software authorship. Please note that any web addresses given in the glossary may be liable to change. At first mention the terms appearing in the glossary are marked with the y symbol. We begin with units which form extensions of the S-Plus y environment. The S- Plus module S þ FinMetrics y provides tools for econometric and financial analysis. Loosely speaking, an S-Plus module is an add-on which must be purchased, whereas an S-Plus library is available at no additional charge. S þ FinMetrics includes functions for analysing extreme values, and in particular for copula modelling. The extreme value functions in S þ FinMetrics are divided into the two categories: Classical Extreme Value Theory and Extreme Value Analysis with Statistical Copula Estimation. The categories are, respectively, based on version four of the library EVIS y, and on the library EVANESCE y. There appears to be little or no difference between the categories and their respective libraries in terms of the methods that they implement. We therefore restrict our attention to the two categories of S þ FinMetrics, which we denote as categories ðaþ and ðbþ, respectively. The library EVIS is openly available, but the library EVANESCE and the related library safd y are only available through an obscure web link and furthermore do not appear on the list of libraries given on the S-Plus web site. There are a number of units which form extensions of the R y open source statistical software environment, and all those given here can be downloaded from CRAN y. The R package ismev y contains code that performs analyses of Coles (2001). The original S-Plus code on which it is based is available from the web address given in Coles (2001). The package extremes y is primarily a GUI interface to the package ismev, with some additional functionality for clustering and estimation of the extremal index. We will refer to extremes only when discussing the additional functionality, and we will refer to ismev when discussing functionality common to both packages. The package evir y is an R port, or

Software for the analysis of extreme events: The current state and future directions 93 conversion, of the S-Plus library EVIS. Since evir implements the same methods as EVIS, we do not consider it further. The packages evd y and evdbayes y implement a number of different methods using classical and Bayesian inference, respectively. Rmetrics y is a project which aims to provide an open source solution for financial and econometric analysis. It includes seven packages, one of which is fextremes y, which in turn requires the package fbasics. Many functions within fextremes are largely based on those within the three packages ismev, evir and evd. The packages VaR y and RandomFields y also contain functions related to extreme value analysis, although neither is primarily focused on extreme value methodology. The former implements the POT method in order to estimate a financial tail quantity known as Value-at-Risk, while the latter contains functions to simulate max-stable processes. A max-stable process has the property that the joint distribution of the process observed at finitely many points is multivariate extreme value. The MATLAB y package EVIM y implements a wide range of extreme value methods, and is similar in functionality to EVIS. The package EXTREMES y contains a suite of functions written in C++ with a MATLAB GUI for the Windows operating system, providing tools for extreme quantile estimation and tail modelling. WAFO y is a toolbox of MATLAB routines for the statistical analysis and simulation of random waves and random loads, and it contains a number of extreme value routines. The environments Xtremes y and HYFRAN y are stand-alone statistical software systems; the former is associated with the book Reiss and Fig. 1 Screenshot of extremes GUI interface. The R console window containing the command line interface is shown in the background

94 A. Stephenson, E. Gilleland Thomas (2001). A new edition of the book and a new version of the software is due for release in late 2005. Both environments employ GUI interfaces for implementing extreme value methods. HYFRAN also implements estimation for a large number of non-standard statistical distributions. Xtremes allows user enhancement through the StatPascal programming language. Many of the algorithms used by Xtremes are also available in the finance library of XploRe y. Finally, there are a number of publicly available routines, written in S and FORTRAN code, associated with the book Beirlant et al. (2004); the overall suite of associated software is in development and therefore we do not consider these routines further. The above description of packaged software units essentially groups the units in terms of the associated software environment, but it is worth considering other categorisations. The majority of packages seek to implement a wide range of standard statistical extreme value methods that are of interest to the developer. The exceptions are evdbayes, VaR, RandomFields and WAFO, each of which has more specialised aims, and where for the latter three, extreme value methods are not the primary focus. The majority of packaged units use a command line interface. The exceptions are extremes, EXTREMES, Xtremes and HYFRAN which employ GUI interfaces, and hence they are very easy to use. The associated penalty is the lack of flexibility, though for sufficiently able users this is countered by the availability of a command line interface for more complicated tasks. Figs. 1, 2, 3 give typical screenshots for extremes, EXTREMES and Xtremes, respectively. Fig. 2 Screenshot of EXTREMES GUI interface

Software for the analysis of extreme events: The current state and future directions 95 2.3. Software implementations 2.3.1. General-purpose units The packaged units described here are general-purpose because they seek to give the user a suite of functions that enable standard extreme value methods to be easily implemented. The specialised units evdbayes, VaR, RandomFields and WAFO are described subsequently. We discuss the implementations of each general-purpose unit by considering different methodologies, approximately ordered by increasing complexity. Table 1 provides a brief outline of the methods implemented by these units, and could be regarded as a crude summary of the discussion to follow. We emphasise that although a given unit may enable the user to implement a large range of statistical techniques, this does not imply that the unit is recommended above all others since there is no guarantee that the included routines are robust or reliable. Considerable experience with a given software unit is required in order for this to be determined. Graphical tools. As with any statistical analysis, the analysis of extremes typically begins with initial exploratory investigations incorporating graphical tools. These tools are largely standard, and are therefore incorporated into standard statistical environments. This may initially appear to be a disadvantage to the user of the stand-alone Xtremes or HYFRAN environments, but the environments themselves each contain an extensive range of graphical tools, incorporating all those which Fig. 3 Screenshot of Xtremes GUI interface

96 A. Stephenson, E. Gilleland Table 1 A crude summary of the methods implemented in a number of distributed software units BM POT DOA PDG CST REG BBM BPOT S þ FinMetricsðAÞ Y Y Y Y Y L S þ FinMetricsðBÞ L L L ismev Y Y Y Y extremes Y Y Y Y evd Y Y Y Y L Y L fextremes Y Y Y Y Y EVIM Y Y Y Y Y Xtremes Y Y Y Y Y Y L HYFRAN Y Y Y EXTREMES L Y Y Y The letters Y and L denote implemented methods; the latter denotes implementations to a lesser degree. The column headings summarise different methodologies; block maxima (BM), peaks over threshold (POT), other domain-of-attraction based estimators (DOA), plotting diagnostics (PDG), clustering (CST), non-stationary regression models (REG), bivariate block maxima (BBM), and bivariate peaks over threshold (BPOT). S þ FinMetricsðAÞ has the same functionality as both EVIS and evir. S þ FinMetricsðBÞ has the same functionality as EVANESCE. The software units VaR, RandomFields, WAFO and evdbayes do not appear in the table because they implement specialised methodology. would be required for any regular analysis. Of the remaining software units fextremes, S þ FinMetricsðAÞ and EVIM have functions for quantile plotting, with the former being a little more versatile. Empirical distribution plotting functions are additionally included in these three units as simple wrappers to underlying code. The units also have plots for record development, and fextremes has plots based on ratios of maxima and sums. The commonly used mean excess (or mean residual life) plot is available in all packaged units. We note that since quantile and distribution plotting are fairly standard tools, most users will be able to produce such plots without the need for specific packaged functions, though perhaps in a more rudimentary form. Classical univariate analysis. The most important distributions in extreme value analyses are the generalised extreme value and generalised Pareto distributions. Consequently, all units contain routines that simulate from these distributions and calculate distribution and quantile functions. The majority of units contain routines that calculate the associated density functions. Other (univariate) distributions may be of interest when considering domain-of-attraction issues. Statistical environments typically provide routines for standard distributions. Non-standard distributions of interest from a domain-of-attraction perspective, such as Burr distributions, are not specifically included in any packaged unit, but with minimal programming experience it is easy to construct such routines within any environment. On a related theme, stable distributions are of interest when considering sums of random variables with infinite variance, and routines related to stable distributions are included in Xtremes and the R package fbasics. The HYFRAN environment and the EXTREMES package also implement statistical distribution fitting for a wide range of distributions that are not necessarily associated with extreme value theory.

Software for the analysis of extreme events: The current state and future directions 97 All packaged units allow the user to perform maximum likelihood parameter estimation for the block maxima and POT methods in the iid case, though for the former the implementation of EXTREMES appears somewhat restrictive. Some units also allow for moment based estimation methods. Specifically, S þ FinMetricsðAÞ and EXTREMES have moment based procedures for POT models, and Sþ FinMetricsðBÞ, fextremes, HYFRAN and Xtremes contain moment based procedures for both models, though standard errors of moment based estimates are generally not given. Xtremes and EXTREMES also contain a number of other estimation methods. S þ FinMetricsðBÞ contains estimation procedures which combine moment based and maximum likelihood methods, but unfortunately all estimation procedures in this unit, including maximum likelihood, are somewhat limited because standard errors of estimates are not returned and there are no diagnostics available subsequent to model fitting. One consideration for POT modelling is the choice of threshold, and there are two common graphical tools to aid the user in this choice. The first is the mean excess plot discussed above. The second is to plot parameter estimates and confidence intervals at different thresholds for parameters that are expected to remain constant above thresholds at which the asymptotic approximation is valid. The packages ismev and evd implement these plots, while S þ FinMetricsðAÞ, EVIM and fextremes implement this plot for the shape parameter and a related plot which shows quantile estimates at each threshold choice. S þ FinMetricsðBÞ also implements this plot for the shape parameter but without pointwise confidence interval bands, so the function in S þ FinMetricsðAÞ is preferred. EXTREMES implements this plot for the parameters of the generalised Pareto distribution, also without pointwise confidence interval bands. Excluding S þ FinMetricsðBÞ, all units have the capacity to produce a number of diagnostic plots subsequent to fitting routines. The uniformly high quality of each unit with regard to diagnostic plotting perhaps reflects the collective belief of the authors that such diagnostics form an invaluable component of statistical inference. Figure 4 shows a screenshot of diagnostic plots for the block maxima method produced using ismev. The plots shown mirror those given in Figure 3.7 of Coles (2001). Figure 5 shows a screenshot of the diagnostic plotting menu and an associated plot for the POT method produced using EVIM. The details of the diagnostic plots illustrated in Figs. 4 and 5 can be found in the documentation for the respective units. The next stage of the analysis typically involves more formal testing procedures. EXTREMES contains functions for performing some goodness-of-fit tests; highlighting two from the doctoral thesis of Myriam Garrido. All units that produce standard errors can be used to construct Wald tests and confidence intervals for individual parameters. Most units produce the value of the optimised likelihood function or its logarithm, and allow models to be fitted under the restriction ¼ 0, and hence likelihood ratio tests can be performed for this hypothesis. The evd package is a little more general in that it allows models to be fitted under any specified restrictions, so more general tests can be performed directly. A related issue is one of profile likelihood plotting. The ismev and fextremes package can plot profile likelihoods for the shape parameter and a given quantile, yielding profile confidence intervals for both, and allowing related likelihood ratio tests to be performed indirectly. The evd package can also plot profile likelihoods, again under a little more generality. S þ FinMetricsðAÞ and EVIM can calculate profile confidence intervals for given quantiles. Excluding S þ FinMetricsðBÞ, all units

98 A. Stephenson, E. Gilleland Fig. 4 Screenshot of ismev. Four diagnostic plots are shown. The plots mirror those given in Figure 3.7 of Coles (2001) can produce maximum likelihood estimates and standard errors for extreme quantiles, reflecting the importance of such quantities in an extreme value analysis. A number of packaged units implement estimators based on domain-of-attraction conditions. Such estimators require a threshold or equivalently a given number of upper order statistics and are typically depicted using a plot of the threshold versus the estimator. If a particular estimate is required, the choice of threshold can be made on the basis of the shape of the plot or on more formal quantities such as the asymptotic mean squared error. S þ FinMetricsðAÞ and EVIM implement this procedure for the Hill estimator (2.2). fextremes additionally implements this procedure for the estimator of Pickands (1975) and for an estimator due to Dekkers et al. (1989). Xtremes and EXTREMES also implement Eq. 2.2 and a number of other estimators. Temporal dependence. The effect of dependence between observations was outlined in Section 2.1, where the importance of the extremal index was discussed. S þ FinMetricsðAÞ and EVIM allow estimation of the extremal index by the blocks method, whereas evd and extremes allow estimation by the runs method and calculation of the associated clusters. Additionally, extremes implements the estimator of Ferro and Segers (2003). fextremes implements blocks and runs estimates. Xtremes also has a number of clustering functions. Functionality for estimation of the extremal index and the associated identification

Software for the analysis of extreme events: The current state and future directions 99 Fig. 5 Screenshot of EVIM. The writing on the left is the diagnostic plot menu. The plot depicted on the right is a scatterplot of residuals of clusters does not imply functionality that allows the index to be taken into account in inference for the POT model. S þ FinMetricsðAÞ and EVIM can account for the index under the point process characterisation, and evd can additionally account for the index under the generalised Pareto characterisation. Non-stationarity. The R packages evd and ismev allow non-stationary models to be fitted. While the former only permits linear models for the location parameter, the latter permits generalised linear models for all three parameters. Coupled with R functions that calculate covariate basis matrices, ismev can be used to fit models of very general form, such as those incorporating regression splines. Estimation is by maximum likelihood, because moment-based estimators do not easily generalise to non-stationary models. Diagnostic plots which account for the non-stationarity are implemented in each package. Another publicly available R package VGAM y allows additional flexibility through the general framework of vector generalised additive models (Yee and Wild, 1996). This package (at the time of writing) is in the development stage, though it will be officially archived on CRAN in the near future (personal communication). Multivariate analysis. Multivariate extensions of the block maxima approach use multivariate extreme value distributions in the same manner as generalised extreme value distributions are used in the univariate case. Xtremes, evd and

100 A. Stephenson, E. Gilleland S þ FinMetricsðBÞ contain functions associated with bivariate componentwise maxima. The latter two contain functions for a large number of parametric models. The main difference between them is that S þ FinMetricsðBÞ focuses on copula models which have uniform margins, whereas in evd generalised extreme value marginals are used directly. The implication is that for an extreme value analysis in S þ FinMetricsðBÞ the user must have the ability to use the programming environment to transform the margins appropriately. More importantly, for inference it implies that the marginal and dependence features must be estimated separately. Unfortunately, S þ FinMetricsðBÞ does not return standard errors or optimised likelihood functions, and diagnostic plots are limited. The Xtremes environment focuses on three particular models, and can calculate moment estimates of the dependence parameter in each case. Bivariate extensions of the POT method are currently only implemented on a fairly superficial level. S þ FinMetricsðAÞ implements the point process approach, but only one particular parametric model is currently available. evd implements the censored likelihood approach for a number of different parametric models, but plotting diagnostics are not currently available. The Xtremes environment calculates estimates for three particular models using the method outlined in Reiss and Thomas (2001). 2.3.2. Specialised units In financial applications, the Value-at-Risk of a financial time series of returns is essentially the level below which a future observation will drop with only a small probability. The small R package VaR implements commonly used estimation methods for this extreme quantile, including the extreme value POT approach. The generalised Pareto tail model is fitted to the negative returns and by subsequent inversion estimates of Value-at-Risk are obtained. VaR also gives confidence intervals and diagnostic plots, though it does not allow for dependence or nonstationarity. The larger package RandomFields includes a function which simulates from different types of max-stable processes. The simulations are based on Gaussian event profile functions (de Haan, 1984). The MATLAB toolbox WAFO contains an extensive set of routines for wave analysis. This includes routines for extreme value techniques such as POT modelling. The R package evdbayes concentrates only on extreme value methodology, but does so within a Bayesian inferential framework. Under this framework, the parameters ð; ; Þ are treated as random variables for which there is an associated trivariate distribution based on beliefs prior to the data being observed (i.e., the prior distribution). After the data are observed, the model is used to update this belief on the basis of the observations, resulting in a posterior distribution. The package computes Markov chain Monte Carlo (MCMC) simulations from posterior distributions based on the block maxima approach and the POT approach using the point process representation. A related order statistics approach is also implemented. It is necessary to simulate from the posterior distributions because, in all but the simplest cases, their evaluation involves an intractable multidimensional integral. Simple linear models of the form i ¼ 0 þ 1 z i can be implemented. A limited number of model diagnostics are available, and MCMC diagnostics are available through the R package coda y.

Software for the analysis of extreme events: The current state and future directions 101 3. Future directions In this Section we discuss the future of the development of software for the analysis of extreme values. We propose the collaborative creation of a software initiative to develop a reliable and coherent set of tools. We feel that this will further increase the use of extreme value methods and will aid the transition of the methodology from a specialised branch of statistics to a conventional constituent of statistical theory. By providing a set of routines regarded as universally dependable it will also help to sharpen the focus of the practitioner, thereby promoting straightforward application. There are many issues to be considered when undertaking such a project, and all must be considered in terms of its objectives. In general terms, we suggest that the primary objective should focus on users who have little expertise in extreme value analysis, but whose applications dictate that such methods should be applied. The aim to facilitate appropriate, user-friendly analyses should lead to the development of a consistent collection of basic tools. However, this need not be the only objective focus. It is also possible to account for researchers who wish to construct their own methods. Such research-based objectives are far more ambitious because they require a system of tools that are both transparent and extensible, but we believe it is prudent to consider both application- and research-based users because the added complications that arise through considering both are outweighed by the benefit of increasing the potential user base. In the remainder of this section we discuss some of the main issues that arise when considering a project of this type. In Section 3.1 we discuss the programming language, the statistical environment and other related issues such as licensing. In Section 3.2 we discuss documentation and feedback, and in Sections 3.3 and 3.4 we, respectively, discuss design and development issues. 3.1. Language and environment No single statistical computing environment stands out as being uniquely popular among members of the extreme-value community. The environment used for the project must therefore be user friendly and undemanding with respect to standard tasks, given suitable documentation. On a more practical level, it must have good numerical and graphical capabilities, as these are of general fundamental importance. The availability of reliable mathematical and statistical algorithms is also a consideration. For example, likelihood inferential procedures require good optimisation routines, and reliable uniform random number generators are needed to create simulation routines for distributions related to extreme value theory. The user interface is another consideration; a GUI interface provides ease of use, but may lack the flexibility required for more sophisticated procedures or extensions desired by advanced users. One problem that arises in attracting a user community is that many users will inevitably not be familiar with the environment, and may not want to spend the required learning time simply to use the distributed software. This problem is difficult to resolve. Perhaps the best solution is to emphasise that learning a new environment has advantages beyond that of simply having the ability to use the distributed routines to perform an extreme value analysis. For example, experience of a GUI-based environment yields generic abilities that can be applied to envi-

102 A. Stephenson, E. Gilleland ronments of any analogous type. Similarly, experience of an environment based on a command-line interface often leads to the development of programming skills, and the ability to use statistical routines related to a number of diverse methodologies. Another solution is to target potential users directly. This can be done in a number of ways, such as promoting use of the software in conference sessions or through courses, or by providing a comprehensive tutorial. This can give potential users insight into the potential benefits, and motivate them to undertake the initial learning required to master the fundamental principals. The consideration of programming language will be partially dependent on the preferences of the main developers, but it will also involve a trade-off between speed of development and speed of application. The coding of methods in a highlevel interpreted language (e.g., R) can be done with speed and relative ease, but such methods may not run as quickly as when implemented in a lower level language (e.g., C). A hybrid approach can be taken whereby the speed-critical sections of high-level implementations can be reimplemented in a lower level language. We believe such an approach yields a good compromise, and allows researchers to build on the tools developed using the high-level framework. If the aim of the project is not geared towards extensibility, and is only focused on building a largely static collection of basic tools, then the avoidance of a high-level language becomes a more attractive option. There are many other elements to the choice of the statistical environment which comprise important components of the overall initiative. The documentation included in the packaged distribution, as discussed in Section 3.2, is among the most important. The statistical environment is also linked closely with the overall design strategy, the latter of which we discuss in more detail in Section 3.3. For example, the exact form of construction of the packaged software units, the details of their construction, and the interdependence between units may all depend to some extent on the environment. The nature of code construction, including concerns such as data structures, object classes, inheritance and the extent to which object oriented methodology is applied, is also influenced by the language and environment. The license which is given to the software upon public release is another important consideration. In particular, decisions must be made with regard to whether users can read, redistribute and modify the associated source code. These decisions impact the choice of environment because some environments may not allow for such practices. On the other hand, an open source license dictates that such practices must be allowed. The form of license will again depend on the aims of the project. We believe that if the project aims to create a flexible and extensible set of tools for both practitioners and researchers, then an open source environment will give the greatest opportunity for success. We support an open source environment in the context described above for a number of reasons, the primary reason being transparency. It is important that all users have the opportunity to read the code so that the accuracy and validity of the output can be readily and openly verified. Moreover, experienced users who are unsatisfied by the documentation descriptions can read the code to determine what precise methods are being used. Similarly, researchers may easily alter and extend the routines allowing the software to be used as a basis for implementing their own methods. This will allow researchers to promote their methods by making available routines that reproduce published results. Finally, the open source paradigm typically leads to beneficial software evolution as users, through interaction with

Software for the analysis of extreme events: The current state and future directions 103 core developers, adapt and improve the software and fix existing errors or bugs. The pace of software development is then seen to be rapid. We should emphasise that the majority of the advantages that we present in favour of the use of an open source environment primarily benefit researchers, and are of considerably less help for users who have little specialised knowledge of either programming languages or extreme-value methodology. If the aims of the project primarily concern such users, and a largely static compendium of standard tools is the objective focus, then a different model may prove more beneficial. 3.2. Documentation and feedback Comprehensive and accurate documentation is vital, and is necessary to make the experience of the user as trouble-free as possible. Documentation can be produced in many forms, the more traditional of which include help files incorporated into software units and manuals which can be read by the user in order to get an overview of the capabilities of the unit in use. Both the content and the construction of the documentation will depend on the environment. For example, the documentation for a primarily GUI based system will typically be less prescriptive than that for a command-line interface. Furthermore, many current environments specify certain design constructions on documentation files to achieve standardisation. For example, the developer may be required to write documentation files in a simple markup language in order that it can be converted into various formats. A related issue to documentation is that of automated testing procedures. More advanced use of documentation allows the developer to run specified segments of executable code in order to test that an error is not created, or to test that a particular result is produced for a given set of inputs or arguments. The ability of an environment to perform such tests and the extent to which this ability is suited to the goals of the project should be considered. More generally, other utilities may exist to aid development and make the process less time consuming, or to guard against other problems that users may face. Version control, installation considerations, documentation errors and platform dependencies are all examples of such issues. Comprehensive documentation provides guidance to users, and this is often reciprocated by user feedback. Feedback from users is useful in a number of ways, and must be actively encouraged. Most obviously, feedback may highlight problems in the software which can then be resolved by developers. In particular, the local conditions of every user cannot be addressed during development, so that problems occurring under particular system conditions can only be detected by users themselves. Creating software that is useful to a large number of users is important, and user feedback gives developers some guide as to the popularity of the software and to the extent and composition of the user community. Effective documentation is essential here, and users should be referred to the documentation whenever possible. On the other hand, user feedback may elicit cases where the documentation is insufficient, assisting the developer in making improvements there as well. 3.3. Design issues The code-level design may dictate that certain rules be followed in order to achieve consistency, to the advantage of both users and developers. At the package level, the form of each packaged unit and the relationships between each unit must be

104 A. Stephenson, E. Gilleland carefully considered. The way in which the developers collaborate so that they can work on the project simultaneously, and the manner in which the software is distributed, must be specified. For all such issues, the environment used will influence the decisions taken. The creation of coding guidelines should obviously be made with reference to the goals of the project. If the software is designed to be extensible, careful thought must be given to construct components that can be expanded or altered to suit the needs of the researcher; and the developer should be open minded to the possibility of such alteration and extension. Otherwise, the resulting code will not be useful as a building block for derivations that may arise in future research. The software introduced in Section 2 contains a number of useful routines, most written in high-level interpreted languages. It is therefore prudent to consider the extent to which these routines and associated code should be incorporated into any future initiative. Clearly there may be licensing issues to be addressed, so we refer here only to code that can legally be used. Reimplementation of different methods is to be avoided when there exists a universally accepted, reliable routine. We suggest then that existing routines be examined by the core developers and utilised whenever the developers regard those routines as reliable and robust. There is little point in reinventing the wheel, and reliable existing routines should aid development whenever possible. The construction of the packaged units will depend largely on the environment. Current environments typically have packaging protocols whereby certain file types and structures must be used. This is to ensure consistency and allow the automated identification of version numbers, dependencies, interpreted and compiled code, datasets, documentation and other such components. It may also allow for certain validation tests to be performed. The dependencies between packaged units must be taken into consideration. It may be preferable that certain units perform as individuals, and depend on no other unit. On the other hand, it may seem natural for one unit to require another unit if the former can be seen as an extension of the latter. Decisions of this type can only be made with reference to the routines contained within each unit, and may consist of a compromise between the complexity of the packaged units and the complexity of the interdependencies. 3.4. Development strategy Collaboration between the core developers is of primary importance, and a system must be in place whereby all developers can work simultaneously on the same project. For a small group of developers, communication is key, and good communication and design is required to ensure that the changes made by one developer do not adversely affect the code of another. The responsibility of a developer is not only to make routines that are reliable and robust, but also to ensure that they fit within the context of the overall project. At any given time, two versions of the software should be maintained: (a) a current version that can allow minor changes and be updated and distributed at frequent intervals; and (b), a development version for more substantial changes that can be accessed by the developers using some kind of management system. It is typically beneficial for the core developers to initially collaborate and derive, to some specified level of formality, a number of basic rules to ensure consistency with respect to the code and the programming interface, allowing greater ease of

Software for the analysis of extreme events: The current state and future directions 105 programming and ultimately usability; particularly for groups of similar functions, such as those based on likelihood inference. For example, in the object oriented methodology, data structures, object classes and related methods for those classes can be, at least for standard methods, designed in advance. The absence of such rules may lead to confusion for the user, who may feel lost among a number of unrelated software components; and additional difficulties for the developer, who may be forced to work with code written in vastly different styles. It may also lead to the creation of software units that cannot successfully interact, even when the developer considers such interaction to be beneficial. The core developers also have responsibility for introducing code written by other contributors, and to encourage the submission of such contributions. A system must be in place to manage such contributions and to assess their suitability. This may consist of checking that the submission satisfies certain advertised criteria, or that the claims made in supporting documentation are fulfilled. The core developers must decide on the level of quality control, and to what extent automated procedures are used. Collaboration will be the key to the success of any initiative. Common, welldefined aims must not give way to the needs and interests of an individual. Those involved must carefully decide upon the methods to be implemented, and must not focus on a particular methodology preferred by any given research group, else the resulting software will not be employed by the general extreme-value community. Proficiency with regard to both extreme-value methods and computer programming is not necessary for each individual in a collaborative effort. The developers must be willing to work together towards a common goal and to communicate effectively irrespective of geographical separation. Only under a collaborative framework such as this can a new software initiative achieve its objectives. 4. Development dilemmas Developing well-designed software is a difficult and time-consuming task where the benefits to the academician often are not sufficient enough to justify the sacrifice that can restrict progress in research and other areas of responsibility. On the other hand, there are compelling arguments that favour undertaking such a project. In this section we address some of these arguments, and then give some ideas of how relevant journals can facilitate development. Finally, we discuss issues related to funding, including the inevitable problem of assessing the popularity and usefulness of the final product. New statistical techniques and theory published in journals are not likely to be employed by scientists and other professionals if no tools are available to implement them. When good software does exist, however, the techniques can get wide use leading to greater numbers of citations for corresponding publications. See, for example, Donoho (2002) who states that one reason for his being a BHighly Cited Author (as defined by The Institute for Scientific Information) is that his research on wavelets was complemented by the development of free companion software which was promoted in his papers. Related to this, obtaining appreciative users throughout the academic community often leads to numerous new contacts both in statistics and other areas of research that can lead to new collaborations. Additionally, the developer will gain further knowledge of programming and other

106 A. Stephenson, E. Gilleland computational issues. A thorough understanding of the methodology involved is obtained, and in many cases the implementation of methods leads to additional insights that often inspire new ideas and observations. For core developers, a global view of the available software and methodology can be acquired, and the appreciation of the extreme value community can be anticipated. Academic recognition for software has not historically been comparable to that based on scientific publication, though this may be changing. It is important for journals to recognise that they can have a positive role in creating and supporting software initiatives such as that proposed in this paper. They can do this by publishing papers demonstrating or reviewing software when that software has a potentially significant role in furthering science. Furthermore, they can encourage authors to cite software used in their papers when appropriate. Funding is an essential factor when considering an initiative of this magnitude. It would likely be difficult to obtain funding for such a project based solely on developing a software package. Funding is more likely to be acquired if such a project is included as part of a grant proposal where the focus is not only on software development, but where the immediate benefit of the software can be justified. The software package extremes, for example, was funded primarily through the broader Weather and Climate Impacts Assessment Science (WCIAS) Initiative (now the WCIAS Program) at The National Center for Atmospheric Research (NCAR) in Colorado. Another possibility is to obtain numerous smaller grants from varying sources, where again successful requests would more likely emphasise the practical applications of the software. For example, it might be possible to obtain some funding from a finance agency, some from an agency interested in hydrology, and so on. The diverse areas of scientific interest to which extreme value theory can be applied are indicative of the potential for incorporating software development in cross-disciplinary funding. Whether a software package is included in a larger grant or is funded as the primary product through one or more funding sources, there will inevitably be a need to demonstrate that the package is useful to the intended community. This is a difficult task, and eliciting feedback from users is not always easy. Often the developer will only hear from users who have problems installing or running the software, and seldom from those who use it successfully. Obtaining statistics on the numbers of visits to the software_s website is one method that has been employed to argue for the software_s popularity. Such statistics can be misleading, however, because a visit does not mean that the visitor found the software useful; many visitors might not even be users. Another approach is to have users register their use of the package. This can also be difficult because many potential users will not want to register. Requiring registration can therefore deter potential users from using the software at all. 5. Discussion The software currently available for the analysis of extreme values goes some way towards allowing practitioners to perform complicated tasks relatively easily. On the other hand, there are different methods and techniques implemented in a number of different software units, and this can lead to confusion. The initiation of a project which aims to develop reliable software within a clear and consistent methodo-