On April 30, Paridhi Athe successfully defended her PhD dissertation, A framework for Predictive Capability Maturity Assessment of Computer Simulation Codes. Paridhi’s committee consisted of her advisor, Nam Dinh, and members, Abhinav Gupta, Maria Avramova, David Kropaczek, and Robert Youngblood.

Abstract

ATHE, PARIDHI. A Framework for Predictive Capability Maturity Assessment of Computer Simulation Codes. (Under the direction of Dr. Nam Dinh).

Rapid growth, affordability of computing power, and capabilities of modeling and simulation tools have brought tremendous advances in the field of engineering and science. In nuclear engineering, modeling and simulation tools are extensively used to support the decisions regarding design, operation, and safety of nuclear power plants. Consequently, comprehensive methodologies and procedures were developed to guide the evaluation of modeling and simulation tools for the intended use.

It becomes evident from the quality of data and models that both simulation models and experiments provide an approximate representation of the real application. Given this scenario of uncertainty and incomplete information, it has proven difficult to make a confident decision regarding the adequacy of a simulation code for an intended application, particularly in a high-consequence nuclear safety analysis. The “Predictive Capability Maturity Model” (PCMM) is a decision model developed by Sandia National Laboratories which provides criteria for assessing the maturity of a tool based on the consequence of its application. Although comprehensive in structure, PCMM remains a high-level guidance. It needs to have a formal structure that can provide clarity, traceability, and ability to integrate information for thorough confidence assessment.

This work presents a formalized and computerized framework for the assessment of decision regarding the adequacy of a simulation tool for an intended application. The adequacy of a simulation code for an intended application is determined by verification, validation and uncertainty quantification (VVUQ) of code. Therefore, the decision regarding code adequacy is dependent on the assessment of different attributes that govern the verification, validation and

uncertainty quantification of code. In this work, the focus is on code validation. Therefore, the framework is developed and illustrated from the perspective of decision regarding the validation assessment of code. This framework includes different process quality assurance (PQA) factors related to the assessment process. These process quality assurance factors warrant the trustworthiness of the evidence and help in checking people and process compliance with respect to the standard requirements.

The proposed framework is developed using an argument modeling technique called Goal Structuring Notation (GSN). Goal structuring notation facilitates structural knowledge representation, information abstraction, evidence incorporation and provides a skeletal structure for quantitative maturity assessment. The decision schema for the development of the decision model is based on the PCMM and Analytic Hierarchy Process (AHP), and formalized using Goal structuring notation. Each decision attribute is formulated as a claim, where the degree of validity of the claim (attribute’s assessment) is expressed using different maturity levels. The GSN representation of the decision model is transformed into a confidence network to provide evidence-based quantitative maturity assessment using the Bayesian network. A metric based on expected utility of maturity levels, called expected distance metric, is proposed to measure the distance between target maturity and achieved maturity on a scale of 0 to 1. Expected distance metric helps in comparing the assessment of different assessment attributes and identification of major areas of concern in terms of modeling capability, data needs, and quality of assessment process.

Practical application of the framework is demonstrated by two case studies. The first case study is focused on validation assessment of a thermal-hydraulic code for a challenge problem called Departure from Nucleate Boiling (DNB). The second case study is focused on assessment of multiphysics codes for another challenge problem called CRUD Induced Power Shift (CIPS).