VERIFICATION AND VALIDATION OF CFD SIMULATIONS

Transcription

1 ERFATON AND ALDATON OF FD SLATONS by Fred Stern, Robert. Wilson, Hugh W. oleman*, and Eric. Paterson of owa nstitute of Hydraulic Research and * Proulsion Research enter echanical and Aerosace Engineering Deartment niversity of Alabama in Huntsville Huntsville AL Sonsored by Office of Naval Research rant N rant N * rant N HR Reort No. 407 owa nstitute of Hydraulic Research ollege of Engineering The niversity of owa owa ity A 54 Setember 999

2 Table Of ontents age Abstract... iii Acnowledgements... iii Nomenclature... iv ntroduction... erification and alidation Procedures... 3 erification and alidation ethodology oncets and Definitions erification onvergence Studies terative onvergence onotonic onvergence: eneralized Richardson Extraolation Oscillatory onvergence Divergence alidation ethodology Single FD ode omarison of ultile odes and/or odels Prediction of Trends orrected vs. ncorrected Simulations Examle for RANS FD ode eometry, onditions, and Benchmar Data rids erification and alidation of ntegral ariable: Resistance erification and alidation of Point ariable: Wave Profile... 5 onclusions... 3 References... 4 Tables and Figures... 6 Aendix A. Derivation of Simulation Error Equation Aendix B. eneralized Richardson Extraolation Aendix. Analytical Benchmars ii

3 Abstract erification and validation methodology is resented for FD simulation results from an already develoed RANS FD code alied for secified obectives, geometry, conditions, and available benchmar information. oncets and definitions are rovided for errors and uncertainties and verification and validation. The simulation error and uncertainty equations are derived with modeling and numerical errors being additive and modeling and numerical uncertainties combining by root-sum-square. The concets and definitions rovide the mathematical framewor for the verification and validation methodology. erification is defined as a rocess for assessing numerical uncertainty and, when conditions ermit, estimating the sign and magnitude of the numerical error itself and the uncertainty in that error estimate. terative and arameter convergence studies are conducted using multile solutions with systematic arameter refinement to estimate numerical errors and uncertainties. Three convergence conditions are ossible: (i) monotonic convergence; (ii) oscillatory convergence; and (iii) divergence. For condition (i), generalized Richardson extraolation for J inut arameters and use of correction factors to account for the effects of higher-order terms and defining and estimating errors and uncertainties is used. For condition (ii), the uer and lower bounds of the solution oscillation are used to estimate uncertainties. For condition (iii), errors and uncertainties can not be estimated. alidation is defined as a rocess for assessing modeling uncertainty by using benchmar exerimental data and, when conditions ermit, estimating the sign and magnitude of the modeling error itself. The comarison error (difference between data and simulation values) and validation uncertainty (combination of uncertainties in data and ortion of simulation uncertainties that can be estimated) are used in this rocess. An examle is rovided for a RANS FD code and results for steady flow for a cargo/container shi. Acnowledgements This research was sonsored by the Office of Naval Research under rants N , N , and N under the administration of Dr. E.P. Rood. The authors gratefully acnowledge Dr. Rood and other colleagues, esecially Prof. W.. Steele and Dr. H. Raven, who made significant contributions through insightful discussions and comments on early drafts. The recent asters Theses of essrs. B. hen and. Dolhin and Ph.D. Thesis of Dr. S.H. Rhee all at The niversity of owa, Deartment of echanical Engineering were helful both in the develoment and testing of the resent verification and validation rocedures and methodology. iii

4 D correction factor benchmar data E, E comarison error, corrected R order of accuracy arameter refinement ratio S, S simulation result, corrected T D, E E P, P reqd S, S S SA SPD S N truth uncertainty estimate data uncertainty Nomenclature comarison error uncertainty, corrected iteration uncertainty arameter uncertainty (e.g., grid size and time ste T), corrected rogrammatic validation requirement simulation uncertainty, corrected simulation modeling uncertainty simulation modeling assumtion uncertainty simulation uncertainty due to use of revious data, simulation numerical uncertainty, corrected, validation uncertainty, corrected x increment in th inut arameter (e.g., grid size and time ste T),, P P S, S SA error error estimate with sign and magnitude iteration error, estimate arameter error, estimate simulation error, corrected simulation numerical error simulation modeling assumtion error solution change error in iv

5 . ntroduction Discussion and methodology for estimating errors and uncertainties in comutational fluid dynamics (FD) simulations has reached a certain level of maturity with increased attention and recent rogress on common concets and terminology (AAA, 998), advocacy and detailed methodology (Roache, 998), and numerous case studies (e.g., ehta, 998). Progress has been accelerated in resonse to the urgent need for achieving consensus on concets and terminology and useful methodology, as FD is alied to increasingly comlex geometry and hysics and integrated into the engineering design rocess. Such consensus is required to realize the goals of simulation-based design and other uses of FD such as simulating flows for which exeriments are difficult (e.g., full-scale Reynolds numbers, hyersonic flows, off-design conditions). n site of the rogress and urgency, the various viewoints have not converged and current methodology falls short of roviding ractical rocedures and methodology for estimating errors and uncertainties in FD simulations. The resent wor rovides a ragmatic aroach for estimating errors and uncertainties in FD simulations. Previous wor on verification (Stern et al., 996) is extended and ut on a more rigorous foundation and combined with subsequent wor on validation (oleman and Stern, 997) [hereafter referred to as &S] thereby roviding the framewor for overall rocedures and methodology. The hilosohy is strongly influenced by exerimental fluid dynamics (EFD) uncertainty analysis (oleman and Steele, 999), which has been standardized. Hoefully, FD verification and validation rocedures and methodology can reach a similar level of maturity and user variability can reach similar low levels, as for EFD. The wor is art of a larger rogram (Rood, 996) for develoing and imlementing a strategy for verification and validation of Reynolds-averaged Navier- Stoes (RANS) shi hydrodynamics FD codes. The rogram includes comlementary FD and EFD towing-tan investigations and considers errors and uncertainties in both the simulations and the data in assessing the success of the verification and validation efforts. The wor also benefited from collaboration with the st and nd nternational Towing Tan Resistance ommittees (TT, 996 and 999). The focus is on verification and validation rocedures and methodology for FD simulation results from an already develoed FD code alied for secified obectives, geometry, conditions, and available benchmar information. The rocedures and methodology were develoed considering RANS FD codes, but should be alicable to a fairly broad range of codes such as boundary-element methods and certain asects of large-eddy and direct numerical simulations. The resent wor differs in many resects from recent literature. The resentation is relatively succinct with intention for use for ractical alications (i.e., industrial FD) for which numerical errors and uncertainties can not be considered negligible or overlooed. The definitions of errors and uncertainties and verification and validation that are used in any aroach need to be clearly stated. Table summarizes the resent definitions along with those given by the AAA (998) and Roache (998) for comarison. The resent and Roache (998) definitions for errors and uncertainties are consistent with those used for EFD. The AAA (998) definitions are from an information

6 theory ersective and differ from those used in EFD, but are not contradictory to the resent definitions. The resent definitions for verification and validation are closely tied to the resent definitions of errors and uncertainties and equations derived for simulation errors and uncertainties. The Roache (998) and AAA (998) definitions are broader, but not contradictory to the resent definitions. The resent aroach includes both the situations () of estimating errors and the uncertainty of those estimates and () of estimating uncertainties only. Richardson extraolation (RE) is used for verification, which is not new; however, the resent generalizations for J inut arameters and use of correction factors to account for the effects of higher-order terms and in defining and estimating errors and uncertainties constitute a new aroach. The use of quantitative estimates for errors and the use of uncertainties for those estimates also constitute a new aroach in verification and validation.. erification and alidation Procedures The overall FD verification and validation rocedures can be conveniently groued in four consecutive stes: () rearation; () verification; (3) validation; and (4) documentation. Prearation. The st ste is rearation, which involves selection of the FD code and secification of obectives, geometry, conditions, and available benchmar information. The obectives might be rediction of certain variables at certain levels of validation (e.g., rogrammatic validation requirements ). The variables can either be integral (e.g., resistance) or oint (e.g., mean velocities and turbulent Reynolds stresses) values and the rogrammatic validation requirements may be different for each variable. erification. The nd ste is verification, which is defined as a rocess for assessing simulation numerical uncertainty and, when conditions ermit, estimating the sign and magnitude of the simulation numerical error itself and the uncertainty in that error estimate (referred to as the corrected simulation numerical uncertainty S N ). terative and inut arameter convergence studies are conducted using multile solutions with systematic arameter, as described in Section 3.. alidation. The 3 rd ste is validation, which is defined as a rocess for assessing simulation modeling uncertainty by using benchmar exerimental data and, when S conditions ermit, estimating the sign and magnitude of the simulation modeling error S itself. The comarison error E (difference between data D and simulation S values) and validation uncertainty (combination of uncertainties in data and ortion of simulation uncertainties that can be estimated) are used, as described in Section 3.3. Documentation. The 4 th ste is documentation, which is detailed resentation of the FD code (equations, initial and boundary conditions, modeling, and numerical methods), obectives, geometry, conditions, verification, validation, and analysis. reqd

7 3. erification and alidation ethodology erification (Section 3.) and validation (Section 3.3) methodology is resented for FD simulation results from an already develoed FD code alied for secified obectives, geometry, conditions, and available benchmar information. Section 3. discusses concets and definitions for errors and uncertainties and verification and validation, which rovide the mathematical framewor for the verification and validation methodology. Analytical benchmars can be defined as the truth and are useful in develoment and confirmation of verification rocedures and methodology and in code develoment, but can not be used for validation and are restricted to simle equations. Results from the use of analytical benchmars are rovided in Aendix. 3. oncets and Definitions Accuracy indicates the closeness of agreement between a simulation/exerimental value of a quantity and its true value. Error is the difference between a simulation value or an exerimental value and the truth. Accuracy increases as error aroaches zero. The true values of simulation/exerimental quantities are rarely nown. Thus, errors must be estimated. An uncertainty is an estimate of an error such that the interval ± contains the true value of 95 times out of 00. An uncertainty interval thus indicates the range of liely magnitudes of but no information about its sign. For simulations, under certain conditions, errors can be estimated including both sign and magnitude (referred to as an error estimate *). Then, the uncertainty considered is that corresonding to the error in *. When is estimated, it can be used to obtain a corrected value of the variable of interest. Sources of errors and uncertainties in results from simulations can be divided into two distinct sources: modeling and numerical. odeling errors and uncertainties are due to assumtions and aroximations in the mathematical reresentation of the hysical roblem (such as geometry, mathematical equation, coordinate transformation, boundary conditions, turbulence models, etc.) and incororation of revious data (such as fluid roerties) into the model. Numerical errors and uncertainties are due to numerical solution of the mathematical equations (such as discretization, artificial dissiation, incomlete iterative and grid convergence, lac of conservation of mass, momentum, and energy, internal and external boundary non-continuity, comuter round-off, etc.). The resent wor assumes that all correlations among errors are zero, which is doubtless not true in all cases, but the effects are assumed negligible for the resent analyses. The simulation error S is defined as the difference between a simulation result S and the truth T. n considering the develoment and execution of a FD code, it can be ostulated that S is comrised of the addition of modeling and numerical errors = S T = + () S A derivation of the simulation error equation () is rovided in Aendix A. The uncertainty equation corresonding to error equation () is S S S = + () 3

8 where S is the uncertainty in the simulation and modeling and numerical uncertainties. where For certain conditions, the numerical error can be considered as * is an estimate of the sign and magnitude of S and are the simulation = * + (3) and is the error in that estimate (and is estimated as an uncertainty since only a range bounding its magnitude and not its sign can be estimated). The corrected simulation value S is defined by with error equation S S = S (4) * = S T = + (5) The uncertainty equation corresonding to error equation (5) is where S S S S N S = + (6) is the uncertainty in the corrected simulation and N is the uncertainty estimate for. Debate on verification and validation has included discussion on whether errors such as are deterministic vs. stochastic and thus how they should be treated in uncertainty analysis was unclear. n the aroach given by equations. (3)-(6), a deterministic estimate * of and consideration of the error in that estimate are used. The aroach is analogous to that in EFD when an asymmetric systematic uncertainty is zero-centered by inclusion of a model for the systematic error in the data reduction equation and then the uncertainty considered is that associated with the model (oleman and Steele, 999). n the uncorrected aroach given by equations ()-(), any articular is considered as a single realization from some arent oulation of 's and the uncertainty is interreted accordingly in analogy to the estimation of uncertainties in EFD (with a similar argument for and ). N S S 3. erification For many FD codes, the most imortant numerical errors and uncertainties are due to use of iterative solution methods and secification of various inut arameters such as satial and time ste sizes and other arameters (e.g., artificial dissiation). The errors and uncertainties are highly deendent on the secific alication (geometry and conditions). The errors due to secification of inut arameters are decomosed into error contributions from iteration number, grid size, time ste T, and other arameters 4

9 P, which gives the following exressions for the simulation numerical error and uncertainty J = = + (7) T P = = + + T + P = + J = (8) Similarly, error estimates * can be decomosed as J = + (9) which gives the following exressions for the corrected simulation and corrected simulation numerical uncertainty = S J = S ( + = ) = T + + S (0) S N = erification is based on equation (0), which is ut in the form + J = () S = S J + ( + ) () Equation () exresses S as the corrected simulation value S lus numerical errors. S is also referred to as a numerical benchmar since it is equal, as shown by equation (0), to the truth lus simulation modeling error and resumable small error in the estimate of the numerical error solutions are used to obtain estimates for the be useful, =. Power-series exansions for each inut arameter and multile s in equation (). For this aroach to must be accurately estimated or be negligible for each solution. 3.. onvergence Studies terative and arameter convergence studies are conducted using multile (m) solutions and systematic arameter refinement by varying the th inut arameter x while holding all other arameters constant. The resent wor assumes inut arameters can be exressed such that the finest resolution corresonds to the limit of infinitely small arameter values. any common inut arameters are of this form, e.g., grid sacing, time ste, and artificial dissiation. Additionally, a uniform arameter refinement ratio r = x x = = x x 3 x x m between solutions is assumed. m The use of uniform arameter refinement ratio is not required (Roache, 998); however, it 5

10 simlifies the analysis and in the authors exerience use of non-uniform arameter refinement ratio is not needed. areful consideration should be given to selection of uniform arameter refinement ratio. The most aroriate values for industrial FD are not yet fully established. Small values (i.e., very close to one) are undesirable since solution changes will be small and sensitivity to inut arameter may be difficult to identify comared to iterative errors. Large values alleviate this roblem; however, they also may be undesirable since the finest ste size may be rohibitively large if the coarsest ste size is designed for sufficient resolution such that similar hysics are resolved for all m solutions. Also, similarly as for small values, solution changes for the finest ste size may be difficult to identify comared to iterative errors since iterative convergence is more difficult for small ste size. Another issue is that for arameter refinement ratio other than r =, interolation to a common location is required to comute solution changes, which introduces interolation errors. Roache (998) discusses methods for evaluating interolation errors. However, for industrial FD, r = may often be too large. A good alternative may be r =, as it rovides fairly large arameter refinement ratio and at least enables rolongation of the coarse-arameter solution as an initial guess for the fine-arameter solution. Equation () is written for the th arameter and mth solution as S m = S m m J (3) =, m terative convergence must be assessed and S m corrected for iterative errors rior to evaluation of arameter convergence since the level of iterative convergence may not be the same for all m solutions used in the arameter convergence studies. ethods for estimating or and are described in Section 3... With evaluated, S is corrected for iterative errors as m m Sˆ m = S m m m J = S + + (4) =, m Equation (3) shows that iterative errors m must be accurately estimated or negligible in comarison to m for accurate convergence studies and that they should be considered within the context of convergence studies for each inut arameter. Ŝ m can be calculated for both integral (e.g., resistance coefficients) and oint (e.g., surface ressure, wall-shear stress, and velocity) variables. Ŝ m can be resented as an absolute quantity (i.e., non-normalized) or normalized with the solution as a ercentage change; however, if the solution value is small, a more aroriate normalization may be the range of the solution. onvergence studies require a minimum of m=3 solutions to evaluate convergence with resect to inut arameter. Note that m= is inadequate, as it only indicates sensitivity and not convergence, and that m>3 may be required. onsider the situation for 6

11 3 solutions corresonding to fine Ŝ, medium Ŝ, and coarse Ŝ values for the th inut 3 arameter. Solution changes for medium-fine and coarse-medium solutions and their ratio R are defined by Three convergence conditions are ossible: = Sˆ ˆ S = Sˆ ˆ S (5) 3 3 R = 3 (i) onotonic convergence: 0 < R < (ii) Oscillatory convergence: R < 0 i (6) (iii) Divergence: R > For monotonic convergence (i), generalized RE is used to estimate or and. ethods for estimating errors and uncertainties for condition (i) are described in Section For oscillatory convergence (ii), the solutions exhibit oscillations, which may be erroneously identified as condition (i) or (iii). This is aarent if one considers evaluating convergence condition from three oints on a sinusoidal curve (oleman et al., 999). Deending on where the three oints fall on the curve, the condition could be incorrectly diagnosed as either monotonic convergence or divergence. ethods discussed here for estimating uncertainties for condition (ii) require more than m=3 solutions and are described in Section For divergence (iii), the solutions diverge and errors and uncertainties can not be estimated. Additional remars are given in Section Determination of the convergence ratio R for oint variables can be roblematic since solution changes and 3 can both go to zero (e.g., in regions where the solution contains an inflection oint). n this case, the ratio becomes ill conditioned. However, the convergence ratio can be used in regions where the solution changes are both non-zero (e.g., local solution maximums or minimums). Another aroach is to use a global convergence ratio R, which overcomes ill conditioning, based on the L norm of the solution changes, i.e., / R =. < > is used to denote an averaged value and / 3 N = i denotes the L norm of solution change over the N oints in the region i= of interest. aution should be exercised when defining the convergence ratio from the ratio of the L norm of solution changes because the oscillatory condition (R < ) cannot condition. i As discussed in the text that follows, 0 < R < and R > may also occur for the oscillatory 7

12 be diagnosed since R will always be greater than zero. Local values of R at solution maximums or minimums should also be examined to confirm the convergence condition based on an L norm definition. 3.. terative onvergence terative convergence must be assessed and simulation results S m corrected for iterative errors rior to evaluation of arameter convergence since the level of iterative convergence may not be the same for all m solutions used in the arameter convergence studies. ethods for estimating or and are described in this section. The methods are alicable to both integral and oint variables. For oint variables, an L norm over all grid oints is often used as a global metric. There are many integral and oint variables that can be monitored to establish iterative stoing criteria; however, resent discussion is secifically within the context of evaluating use in the arameter convergence study for S m or and for. Further wor is needed on assessing iterative errors and their role in arameter convergence studies and for assessing iterative errors and uncertainties for unsteady flows. Tyical FD solution techniques for obtaining steady state solutions involve beginning with an initial guess and erforming time marching or iteration until a steady state solution is achieved. For time-accurate calculations using imlicit methods, convergence of the solution is required at each time ste. are must be exercised in evaluating iterative convergence based solely on solution residuals, i.e., change in solution from iteration to iteration. Small time stes and/or relaxation arameters can result in small solution residuals while iterative error can be large (Ferziger and Peric, 997). f S is a rimary deendent variable, an alternative aroach that removes this roblem is to use the residual imbalance of the discretized equations (i.e., the difference in the left- and righthand sides) as a measure of convergence; since, the iterative error satisfies the same equation as this residual imbalance. The number of order magnitude dro and final level of solution residual (or residual imbalance) can be used to determine stoing criteria for iterative solution techniques. terative convergence to machine zero is desirable, but for comlex geometry and conditions it is often not ossible. Three or four orders of magnitude dro in solution residual to a level of 0-4 is more liely for these cases. ethods for estimation of iterative errors and uncertainties can be based on grahical, as discussed below, or theoretical aroaches and are deendent on the tye of iterative convergence: (a) oscillatory; (b) convergent; or (c) mixed oscillatory/convergent. For oscillatory iterative convergence (a), the deviation of the variable from its mean value rovides estimates of the iterative uncertainty based on the range of the maximum S and minimum S L values = ( S S L ) (7) m 8

13 For convergent iterative convergence (b), a curve-fit of an exonential function can be used to estimate or and as the difference between the value and the exonential function from a curve fit for large iteration number = S F = S F, = 0 m F For mixed convergent/oscillatory iterative convergence (c), the amlitude of the solution enveloe decreases as the iteration number increases, the solution enveloe is used to define the maximum S and minimum S L values in the th iteration, and to estimate or and m = S = ( ( S S An increase in the amlitude of the solution enveloe as the iteration number increases indicates that the solution is divergent. Estimates of the iterative error based on theoretical aroaches are resented in Ferziger and Peric (997) and involve estimation of the rincial eigenvalue of the iteration matrix. The aroach is relatively straightforward when the eigenvalue is real and the solution is convergent. For cases in which the rincial eigenvalue is comlex and the solution is oscillatory or mixed, the estimation is not as straightforward and additional assumtions are required. S S L L ) ), = 0 (8) (9) 3..3 onotonic onvergence: eneralized Richardson Extraolation For monotonic convergence, i.e., condition (i) in equation (6), generalized RE is used to estimate or and. RE is generalized for J inut arameters and use of correction factors to account for the effects of higher-order terms and defining and estimating errors and uncertainties, as summarized in the following. Aendix B rovides a detailed descrition. eneralized RE begins with equation (4). The error terms on the right-hand-side of equation (4) are of nown form (i.e., ower series exansion in x ) based on analysis of the modified (A.6) and numerical error (A.9) equations, as shown in Aendix A equation (A.), which is written below as a finite sum (i.e., error estimate) and for the th arameter and mth solution m n ( i) = ( x ) g (0) i= (i) n = number of terms retained in the ower series, owers corresond to order of (i) accuracy (for the ith term), and g are referred to as grid functions which are a m 9 ( i)

14 function of various orders and combinations of derivatives of S with resect to x. Substituting equation (0) into equation (4) results in n i= i i S ˆ ( ) ( ) = S + ( x ) g + () m m J =, Subtraction of multile solutions where inut arameter eliminates the equations for S, terms in equation () since m m (i), and g (i). This assumes m x is uniformly refined is indeendent of x and rovides and (i) (i) g are also indeendent of x. Since each term (i) contains unnowns, m=n+ solutions are required to estimate the numerical benchmar S and the first n terms in the exansion in equation () (i.e., for n=, m=3 and for n=, m=5, etc). The accuracy of the estimates deends on how many terms are retained in equation (0), the magnitude (imortance) of the higher-order terms, (i) (i) and the validity of the assumtion that and g are indeendent of x. For sufficiently small x, the solutions are in the asymtotic range such that higher order (i) (i) terms are negligible and the assumtion that and g are indeendent of x is valid. However, achieving the asymtotic range for ractical geometry and conditions is usually not ossible and m>3 is undesirable from a resources oint of view; therefore, methods are needed to account for effects of higher-order terms for ractical alication of RE. (i) Additionally, methods may be needed to account for ossible deendence of and on x inut arameter, i.e.,, although not addressed herein. sually = is estimated for the finest value of the corresonding to the finest solution S. For m=3, only the leading-order term can be evaluated. Equations are obtained for and order-of-accuracy Aendix B includes results for m=5. (i) g = = RE () r ln( 3 ) = (3) ln( r ) Aendix rovides verification for two analytical benchmars (one-dimensional wave and two-dimensional Lalace equations). ultile solutions were used to evaluate the RE error estimates, including the effects of higher-order terms. Solving for the firstorder term is relatively easy since evaluation of equations () and (3) only requires that the m=3 solutions are monotonically convergent, even if the solutions are far from the asymtotic range and equations () and (3) are inaccurate. Solving for the higher-order terms (i.e., second-order term) is more difficult since evaluation of the m=5 solutions for ( i=,) ( =,) S,, and g i additionally requires that the solutions are relatively close to the asymtotic range, i.e., within about 6% of the theoretical order of accuracy based on the modified equation and th th q. 0

15 The solutions show that equation () has the correct form, but the order of accuracy is oorly estimated by equation (3) excet in the asymtotic range. Therefore, one aroach is to correct equation () by a multilication correction factor to account for the effects of higher-order terms. Two correction factors were investigated r = (4a) est r q est ( 3 / r )( r ) ( 3 / = r est )( r + est q est est est q est q est ( r r )( r ) ( r r )( r ) ) (4b) est and est q are estimates for the st and nd term order of accuracy estimated values can be based either on and q th th () and (). The or solutions for simlified geometry and conditions. n either case, referably including the effects of grid stretching. Equation (4a) roughly accounts for the effects of higher-order terms by relacing with thereby roviding an imroved single-term estimate. Equation (4b) more est rigorously accounts for higher-order terms since it is derived from the two-term estimate with st and nd term order of accuracy and relaced by and q. Equation () (4b) simlifies to equation (4a) in the limit of the asymtotic range. Both correction factors only require solutions for three arameter values. < or > indicates that the leading-order term over redicts (higher-order terms net negative) or under redicts (higher-order terms net ositive) the error, resectively. given by equation (4) is fairly universal in that it only imlicitly deends on geometry and conditions. However, is based on results from only two linear analytical benchmars and additional benchmars (esecially non-linear) are needed to confirm the universality of equation (4) or to rovide alternative forms. ombining equation () and (4) rovides an estimate for effects of higher-order terms () est est accounting for the = = RE (5) r The estimate includes both sign and magnitude. Equation (5) is used to estimate or and deending on how close the solutions are to the asymtotic range (i.e., how close is to ) and one s confidence in equation (5). There are many reasons for lac of confidence, esecially for comlex three-dimensional flows. Point variables invariably are not uniformly convergent, which is articularly evident near inflection oints and zero crossings. Equations (4) and (5) need further testing both for additional analytical benchmars (as already mentioned) and ractical alications. Also alternative strategies for including effects of higher-order terms may be ust as viable. Note that equation (5) differs

16 significantly from the roosed by Roache (998). Herein = (, r,,, q ), whereas in the, est est is a constant referred to as a factor of safety F S which equals.5 for careful grid studies and 3 for cases for which only two grids are used. For sufficiently less than or greater than and lacing confidence, estimated, but not and is. Based on the analytical benchmar studies (Aendix ), it aears that equation (5) can be used to estimate the uncertainty by bounding the error by the sum of the absolute value of the corrected estimate from RE and the absolute value of the amount of the correction = + (6) RE ( ) RE For sufficiently close to and having confidence, Equation (5) is used to estimate the error and are estimated., which can then also be used in the calculation of S [in equation (0)]. The uncertainty in the error estimate is based on the amount of the correction Note that in the limit of the asymtotic range, = ( ) RE (7) =, = * = RE, and = Oscillatory onvergence For oscillatory convergence, i.e., condition (ii) in equation (6), uncertainties can be estimated, but not the signs and magnitudes of the errors. ncertainties are estimated based on determination of the uer ( S ) and lower ( S L ) bounds of solution oscillation, which requires more than m=3 solutions. The estimate of uncertainty is based on half the solution range 3..5 Divergence = ( S S L ) (8) For divergence, i.e., condition (iii) in equation (6), errors and or uncertainties can not be estimated. The rearation and verification stes must be reconsidered. mrovements in iterative convergence, arameter secification (e.g., grid quality), and/or FD code may be required to achieve converging or oscillatory conditions. 3.3 alidation alidation is defined as a rocess for assessing modeling uncertainty S by using benchmar exerimental data and, when conditions ermit, estimating the sign and

17 magnitude of the modeling error S itself. Thus, the errors and uncertainties in the exerimental data must be considered in addition to the numerical errors and uncertainties discussed in Section 3.. Aroaches to estimating exerimental uncertainties are resented and discussed by oleman and Steele (999). The validation methodology of oleman and Stern (997) which roerly taes into account the uncertainties in both the simulation and the exerimental data is described in this section. The methodology is also demonstrated using an estimated numerical error and corrected simulation and validation uncertainty values ethodology The validation comarison for a simulated and measured result r that is a function of the variable X is shown in figure. The exerimentally determined r-value of the ( X i, r i ) data oint is D and, as before, the simulated r-value is S. Recall from equation () that the simulation error S is the difference between S and the truth T. Similarly, the error D in the data is the difference between D and the truth T, so setting the simulation and exerimental truths equal results in D = S (9) The comarison error E is defined as the difference of D and S D S E = D S = = + + ) (30) D S D ( SA SPD with S decomosed into the sum of SPD, error from the use of revious data such as fluid roerties, and SA, error from modeling assumtions. Thus E is the resultant of all the errors associated both with the exerimental data and with the simulation. For the * aroach in which no estimate of the sign and magnitude of is made, all of these errors are estimated with uncertainties. (As will be shown, during the validation rocess an estimate of the sign and magnitude of SA can be made under certain conditions.) f X i, ri, and S share no common error sources, then the uncertainty E in the comarison error can be exressed as or E E E = D + S = D + D S = E D SA SPD where subscrits are used in the same manner as for the 's. deally, we would lie to ostulate that if the absolute value of E is less than its uncertainty E, then validation is achieved (i.e., E is zero considering the resolution imosed by the noise level E ). n reality, the authors now of no aroach that gives S (3) (3) 3

18 an estimate of SA, so E cannot be estimated. That leaves a more stringent validation test as the ractical alternative. f the validation uncertainty is defined as the combination of all uncertainties that we now how to estimate (i.e., all but SA), then = = + + E SA f E is less than the validation uncertainty, the combination of all the errors in D and S is smaller than the estimated validation uncertainty and validation has been achieved at the level. is the ey metric in the validation rocess. is the validation noise level imosed by the uncertainties inherent in the data, the numerical solution, and the revious exerimental data used in the simulation model. t can be argued that one cannot discriminate once E is less than this; that is, as long as E is less than this, one cannot evaluate the effectiveness of roosed model imrovements. f the corrected aroach of equations (3)- (6) is used, then the equations equivalent to equations (30) and (33) are E D SPD (33) = D = + + ) (34) S for the corrected comarison error and E D SA ( SA SPD = = + + for the corrected validation uncertainty. Note that S and E can be either larger or smaller than their counterarts S and E, but and should be smaller than E and D E SPD, resectively, since S N should be smaller than. For the data oint (, ) X i r i, D should include both the exerimental uncertainty in r i and the additional uncertainties in r i arising from exerimental uncertainties in the measurements of the n indeendent variables ( X in X i. The exression for D that should be used in the ( ) calculation is then D ) i i ( ) X i S N (35) n r ri = + = X (36) n some cases, the terms in the summation in equation (36) may be shown to be very small, using an order-of-magnitude analysis, and then neglected. This would occur in situations in which the values are of "reasonable" magnitude and gradients in r are X small. n regions with high gradients (e.g., near a surface in a turbulent flow), these terms may be very significant and the artial derivatives would be estimated using whatever X, data is available. ( ) i r i There is also a very real ossibility that measurements of different variables might share identical bias errors. This is easy to imagine for measurements of x, y, and z. Another ossibility is D and S sharing an identical error source, for examle if the same 4

19 density table (curve fit) is used both in data reduction in the exeriment and in the simulation. n such cases, additional correlated bias terms must be included in equation (3), (3), (33), and (35). To estimate SPD for a case in which the simulation uses revious data D i in m instances, one would need to evaluate where the SPD = m i= S D i ( ) D are the uncertainties associated with the data. i D i (37) 3.3. Single FD ode onsideration of equation (3) shows that () the more uncertain the data, and/or () the more inaccurate the code (greater and SPD ), the easier it is to validate a code, since the greater the uncertainties in the data and the code redictions, the greater the noise level. However, if the value of is greater than that designated as necessary in a research/design/develoment rogram, the required level of validation could not be achieved without imrovement in the quality of the data, the code, or both. Also, if and SPD are not estimated, but E is less than D, then a tye of validation can be argued to have been achieved, but clearly as shown by the resent methodology, at an unnown level. f there is a rogrammatic validation requirement, denote it as reqd since validation is required at that uncertainty level or below. From a general ersective, if we consider the three variables, E, and there are six combinations (assuming none of the three variables are equal):. E < < reqd reqd. E < reqd < 3. reqd < E < 4. < E < reqd (38) 5. < reqd < E 6. reqd < < E n cases, and 3, E < ; validation is achieved at the level; and the comarison error is below the noise level, so attemting to decrease the error SA due to the modeling assumtions in the simulation is not feasible from an uncertainty standoint. 5

20 n case, validation has been achieved at a level below from a rogrammatic standoint. reqd, so validation is successful n cases 4, 5 and 6, < E, so the comarison error is above the noise level and using the sign and magnitude of E to estimate SA is feasible from an uncertainty standoint. f << E, then E corresonds to SA and the error from the modeling assumtions can be determined unambiguously. n case 4, validation is successful at the E level from a rogrammatic standoint. A similar comarison table can be constructed using E,, and reqd. Since E can be larger or smaller than E, but should always be less than, the results for a given corrected case are not necessarily analogous to those for the corresonding uncorrected case. That is, a variable can be validated in the corrected but not in the uncorrected case, or vice versa. However, the band E ± should always give a E smaller (therefore better) range within which the true value of E lies than the band E ± E, assuming that one s confidence in using the estimate is not mislaced. Furthermore, for cases 4, 5, and 6, one can argue that E more liely corresonds to SA. n general, validation of a code's redictions of a number (N) of different variables is desired, and this means that in a articular validation effort there could be N different E, E,,, and reqd values and (erhas) some successful and some unsuccessful validations. For each variable, a lot of the simulation rediction versus X comared with the ( X i, r i ) data oints gives a traditional overview of the validation status, but the interretation of the comarison is greatly affected by choice of the scale and the size of the symbols. A lot of ± ( ± ) and E (E ), and reqd (if nown) versus X for each variable is articularly useful in drawing conclusions, and the interretation of the comarison is more insensitive to scale and symbol size choices. * omarison of ultile odes and/or odels When a validation effort involves multile codes and/or models, the rocedure discussed above -- comarison of values of E and (and if nown) for the N variables -- should be erformed for each code/model. Since each code/model may have a different, some method to comare the different codes /models erformance for each variable in the validation is useful. The range within which (95 times out of 00) the true value of E lies is E ± E. From equation (3), when SA is zero then = E, so for that ideal condition the maximum absolute magnitude of the 95% confidence interval is given by E +. omarison of the ( E + ) s for the different codes/models then shows which has the smallest range of reqd 6

21 liely error assuming all SA s are zero. This allows aroriate comarisons of (low E)/(high ) with (high E/low ) codes/models. A similar discussion holds if the corrected values are used Predictions of Trends n some instances, the ability of a code or model to redict the trend of a variable may be the subect of a validation effort. An examle would be the difference in drag for two shi configurations tested at the same Froude number. The rocedure discussed above -- comarison of E and for the drag -- should be erformed for each configuration. The difference in drag for the two configurations should then be considered as the variable that is the subect of the validation. As discussed in oleman and Steele (999), because of correlated bias uncertainty effects in the exerimental data the magnitude of the uncertainty in may be significantly less than the uncertainty in either of the two exerimentally determined drag values. This means that the value of for may be significantly less than the 's for the drag values, allowing for a more stringent validation criterion for the difference than for the absolute magnitudes of the variables. hoice of the corrected or uncorrected aroach should be made on a secific case-by-case basis orrected vs. ncorrected Simulation Results f a validation using the corrected aroach is successful at a set condition, then if one chooses to associate that validation uncertainty level with the simulation's rediction at a neighboring condition that rediction must also be corrected. That means enough runs are required at the new condition to allow estimation of the numerical errors and uncertainties. f this is not done, then the comarison error E and validation uncertainty corresonding to the use of the uncorrected S and its associated (larger) should be the ones considered in the validation with which one wants to associate the rediction at a new condition. (Whether to and how to associate an uncertainty level at a validated condition with a rediction at a neighboring condition is very much unresolved and is ustifiably the subect of much debate at this time.) As discussed in Section 3.3., however, the band E ± E should always give a smaller (therefore better) range within which the true value of E lies than the band E ± E, assuming that one s confidence in using the estimate is not mislaced. * 7

22 4.0 Examle for RANS FD ode Examle results of verification and validation are resented for a single FD code and for secified obectives, geometry, conditions, and available benchmar information. The FD code is FDSHP-OWA, which is a general-urose, multi-bloc, high erformance comuting (arallel), unsteady RANS code (Paterson et al, 998; Wilson et al., 998) develoed for comutational shi hydrodynamics. The RANS equations are solved using higher-order uwind finite differences, PSO, Baldwin-Lomax turbulence model, and exact and aroximate treatments, resectively, of the inematic and dynamic free-surface boundary conditions. The obectives are to demonstrate the usefulness of the roosed verification and validation rocedures and methodology and establish the levels of verification and validation of the simulation results for an established benchmar for shi hydrodynamics FD validation. 4. eometry, onditions, and Benchmar Data The geometry is the Series 60 cargo/container shi. The Series 60 was used for two of the three test cases at the last international worsho on validation of shi hydrodynamics FD codes (FD Worsho Toyo, 994). The conditions for the calculations are Froude number Fr = 0.36, Reynolds number Re = 4.3x0 6, and zero sinage and trim. These are the same conditions as the exeriments, excet the resistance and sinage and trim tests, as exlained next. The variables selected for verification and validation are resistance T (integral variable) and wave rofile ζ (oint variable). The benchmar data is rovided by Toda et al. (99), which was also the data used for the Series 60 test cases at the FD Worsho Toyo (994). The data includes resistance and sinage and trim for a range of Fr for the model free condition (i.e., free to sin and trim); and wave rofiles, near-field wave attern, and mean velocities and ressures at numerous stations from the bow to the stern and near wae, all for Fr = (0.6, 0.36) and the zero sinage and trim model fixed condition. The data also includes uncertainty estimates, which were recently confirmed/udated by Longo and Stern (999) closely following standard rocedures (oleman and Steele, 999). The resistance is nown to be larger for free vs. fixed models. Data for the Series 60 indicates about an 8% increase in T for the free vs. fixed condition over a range of Fr including Fr=0.36 (Ogiwara and Kaatani, 994). The Toda et al. (99) resistance values were calibrated (i.e., reduced by 8%) for effects of sinage and trim for the resent comarisons. 4. omutational rids rid studies were conducted using four grids (m=4), which enables two searate grid studies to be erformed and comared. rid study gives estimates for grid errors and uncertainties on grid using the three finest grids -3 while grid study gives estimates for grid errors and uncertainties on grid using the three coarsest grids -4. The results for grid study are given in detail and the differences for grid study are also mentioned. The grids were generated using the commercial code RDEN (Pointwise, 8

23 nc.) with consideration to toology; number of oints and grid refinement ratio r ; nearwall sacing and turbulence model requirement that first oint should be at y + <; bow and stern sacing; and free-surface sacing. The toology is body-fitted, H-tye, and single bloc.. The sizes of grids (finest) through 4 (coarsest) are 87x78x43 = 876,, 0x5x3 = 37,78, 44x36x = 4,048, and 0x6x6 = 4,06, and the grid refinement ratio r =. lustering was used near the bow and stern in the ξ direction, at the hull in the η-direction, and near the free surface in the ζ-direction. The y + values for grids -4 were about 0.7,,.4, and, resectively. About twice the number of grid oints in the η-direction would be required to achieve y + <.0 for all four grids -4 (i.e., roughly,800,000 oints on the finest grid). With grid refinement ratio r =, only grids and were generated. rids 3 and 4 were obtained by removing every other oint from grids and, resectively (i.e., the grid sacing of grids 3 and 4 is twice that of grids and, resectively). rids and were generated by secifying the grid sacing at the corners and number of oints along the edges of the comutational blocs. The faces of the comutational blocs were smoothed using an ellitic solver after which the coordinates in the interior were obtained using transfinite interolation from the bloc faces. rid was generated from grid by increasing the grid sacing and decreasing the number of comutational cells in each coordinate direction at the corners of the blocs by a factor r. A comarison of the four grids at the free surface lane is shown in figure along with comuted wave elevation contours 4.3 erification and alidation of ntegral ariable: Resistance erification. erification was erformed with consideration to iterative and grid convergence studies, i.e., = + and = +. terative convergence was assessed by examining iterative history of shi forces and L norm of solution changes summed over all grid oints. Figure 3 shows a ortion of the iterative history on grid. The ortion shown reresents a comutation started from a revious solution and does not reflect the total iterative history. Solution change dros four orders of magnitude from an initial value of about 0 - (not shown) to a final value of 0-6. The variation in T is about 0.07%S over the last eriod of oscillation (i.e., = 0.07%S ). terative uncertainty is estimated as half the range of the maximum and minimum values over the last two eriods of oscillation (see figure 3c). terative histories for grids -4 show iterative uncertainties of about 0.0, 0.03, and 0.0%S, resectively. The level of iterative uncertainties for grids -4 are at least two orders of magnitude less than the corresonding grid uncertainties, whereas the iterative uncertainty for grid is only one order of magnitude smaller than the grid error. For all four grids the iteration errors and uncertainties are assumed to be negligible in comarison to the grid errors and uncertainties for all four solutions (i.e., << and << such that = and = ). The results from the grid convergence study for T are summarized in tables and 3. The solutions for T indicate the converging condition (i) of equation (6) with 9