The process of establishing credibility in computational model predictions via verification and validation (V&V) encompasses a wide range of activities. Those activities are focused on collecting evidence that the model is adequate for the intended application and that the errors and uncertainties are quantified. In this work, we use the predictive capability maturity model (PCMM) as an organizing framework for evidence collection activities and summarizing our credibility assessment. We discuss our approaches to sensitivity analysis, model calibration, model validation, and uncertainty quantification and how they support our assessments in the solution verification, model validation, and uncertainty quantification elements of the PCMM. For completeness, we also include some limited assessment discussion for the remaining PCMM elements. Because the computational cost of performing V&V and the ensuing predictive calculations is substantial, we include discussion of our approach to addressing computational resource considerations, primarily through the use of response surface surrogates and multiple mesh fidelities.

References

1.
Oberkampf
,
W.
, and
Roy
,
C.
,
2010
,
Verification and Validation in Scientific Computing
,
Cambridge University Press
,
New York
.
2.
Hu
,
K.
,
2013
, “
2014 V&V Challenge: Problem Statement
,” Sandia National Laboratories, Albuquerque, NM and Livermore, CA,
Technical Report No. SAND2013-10486P
.
3.
Adams
,
B. M.
,
Bauman
,
L. E.
,
Bohnhoff
,
W. J.
,
Dalbey
,
K. R.
,
Eddy
,
J. P.
,
Ebeida
,
M. S.
,
Eldred
,
M. S.
,
Hough
,
P. D.
,
Hu
,
K. T.
,
Jakeman
,
J. D.
,
Swiler
,
L. P.
,
Stephens
,
J. A.
,
Vigil
,
D. M.
, and
Wildey
,
T. M.
,
2014
, “
Dakota, a Multilevel Parallel Object-Oriented Framework for Design Optimization, Parameter Estimation, Uncertainty Quantification, and Sensitivity Analysis: Version 6.1 Users Manual
,” Sandia National Laboratories, Albuquerque, NM, Technical Report No. SAND2014-4633.
4.
Adams
,
B.
,
Ebeida
,
M.
,
Eldred
,
M.
,
Jakeman
,
J.
,
Swiler
,
L.
,
Bohnhoff
,
W.
,
Dalbey
,
K.
,
Eddy
,
J.
,
Hu
,
K.
,
Vigil
,
D.
,
Bauman
,
L.
, and
Hough
,
P.
,
2011
, “
Dakota, a Multilevel Parallel Object-Oriented Framework for Design Optimization, Parameter Estimation, Uncertainty Quantification, and Sensitivity Analysis
,” Sandia National Laboratories, Albuquerque, NM and Livermore, CA, Technical Report No. SAND2011-9106.
5.
Oberkampf
,
W.
,
Pilch
,
M.
, and
Trucano
,
T.
,
2007
, “
Predictive Capability Maturity Model for Computational Modeling and Simulation
,” Sandia National Laboratories, Albuquerque, NM and Livermore, CA,
Technical Report No. SAND2007-5948
.
6.
Montgomery
,
D.
, and
Runger
,
G.
,
1994
,
Applied Statistics and Probability for Engineers
,
Wiley
,
New York
.
7.
Hahn
,
G.
, and
Meeker
,
W.
,
1991
,
Statistical Intervals—A Guide for Practitioners
,
Wiley
,
New York
.
8.
Computer Software by Minitab, Inc.
, “
Minitab 17 Statistical Software
,” www.minitab.com
9.
Howe
,
W.
,
1969
, “
Two-Sided Tolerance Limits for Normal Populations—Some Improvements
,”
J. Am. Stat. Assoc.
,
64
(
326
), pp.
610
620
.
10.
Romero
,
V.
,
Swiler
,
L.
,
Urbina
,
A.
, and
Mullins
,
J.
,
2013
, “
A Comparison of Methods for Representing Sparsely Sampled Random Quantities
,” Sandia National Laboratories, Albuquerque, NM and Livermore, CA,
Technical Report No. SAND2013-4561
.
11.
Iman
,
R. L.
, and
Shortencarier
,
M. J.
,
1984
, “
A Fortran 77 Program and User's Guide for the Generation of Latin Hypercube Samples for Use With Computer Models
,” Sandia National Laboratories, Albuquerque, NM,
Technical Report No. NUREG/CR-3624, SAND83-2365
.
12.
Saltelli
,
A.
,
Tarantola
,
S.
,
Compolongo
,
F.
, and
Ratto
,
M.
,
2004
,
Sensitivity Analysis in Practice: A Guide to Assessing Scientific Models
,
Wiley
,
New York
.
13.
Dennis
,
J. E.
,
Gay
,
D. M.
, and
Welsch
,
R. E.
,
1981
, “
ALGORITHM 573: NL2SOL—An Adaptive Nonlinear Least-Squares Algorithm
,”
ACM Trans. Math. Software
,
7
(
3
), pp.
369
383
.
14.
Xiu
,
D.
,
2010
,
Numerical Methods for Stochastic Computations: A Spectral Method Approach
,
Princeton University Press
,
Princeton, NJ
.
15.
Hastie
,
T.
,
Tibshirani
,
R.
, and
Friedman
,
J.
,
2001
,
The Elements of Statistical Learning: Data Mining, Inference, and Prediction: With 200 Full-Color Illustrations
,
Springer-Verlag
,
Berlin
.
16.
Bichon
,
B.
,
Eldred
,
M.
,
Swiler
,
L.
,
Mahadevan
,
S.
, and
McFarland
,
J.
,
2008
, “
Efficient Global Reliability Analysis for Nonlinear Implicit Performance Functions
,”
AIAA J.
,
46
(
10
), pp.
2459
2468
.
17.
MacKay
,
D.
,
1998
, “
Introduction to Gaussian Processes
,”
Neural Networks and Machine Learning
, Vol.
168
,
C. M.
Bishop
, ed., Springer, Berlin, pp.
133
165
.
You do not currently have access to this content.