A Characterization Taxonomy for Integrated Management of Modeling and Simulation Tools

A Characterization Taxonomy for Integrated Management of Modeling and Simulation Tools Bobby Hartway AEgis Technologies Group 631 Discovery Drive Huntsville, AL 35806 256-922-0802 bhartway@aegistg.com Danny Thomas AEgis Technologies Group 631 Discovery Drive Huntsville, AL 35806 256-922-0802 danny.thomas@aegistg.com Lisa Cain AEgis Technologies Group 631 Discovery Drive Huntsville, AL 35806 256-922-0802 abray@aegistg.com Joe Hale NASA Marshal Space Flight Center Huntsville, AL 256-655-6883 joe.hale@nasa.gov Keywords: Modeling and Simulation Tools, NASA, Integrated Management, Taxonomy, Database schema ABSTRACT: The NASA Exploration Program employs a large number of modeling and simulation (M&S) assets for complex analysis of sophisticated spacecraft design and operations. Many are existing M&S that have a legacy of productive use. These M&S vary greatly in technologies addressed, methods of analysis employed, rigor of documentation, user support available, extent of testing done, and configuration management employed. Integrated management of these M&S assets requires an ability to answer the following key questions: Do we know what M&S we have (where and when)? Do we know what M&S we need (where and when)? Do we know if what we have matches what we need? Do we know who is working on and using what asset, where, when, how, and why, and what the status is? Do we know, and are we tracking, the VV&A requirements and status of these assets? The answers to these questions might be found in an M&S asset inventory catalog, library, database, repository, or even a project workflow server but all such information stores rely on the same fundamental requirement for a common, well-structured, M&S asset characterization schema, which in turn requires a common M&S asset characterization taxonomy. This characterization taxonomy provides a critical common terminology for specifying analysis data needs, specifying requirements (including VV&A) for M&S matching these data needs, and specifying M&S asset-survey templates that will properly inventory existing M&S assets, their applicability, and their availability. This paper describes an initial draft taxonomy created to accomplish the above requirements in support of the integrated management of M&S assets for NASA s Space Exploration Program. This taxonomy supports effective information exchange mechanisms requiring description, search, retrieval, matching, evaluation, and application of M&S assets among diverse organizations and multiple disciplines.

1.0 Multidimensional M&S Challenge of NASA Constellation Program NASA s vision for space exploration encompasses a broad range of human and robotic missions, including missions to the Moon, Mars, and destinations beyond. This endeavor requires a challenging multi-dimensional application of Modeling and Simulations (M&S) to conduct performance analyses; validate requirements; evaluate designs and associated technology trades, evaluate hardware and software implementation and interfaces; and support element and system integration. NASA uses a comprehensive set of M&S to support both technical and programmatic decisions across multiple organizations, at multiple levels, throughout the system development lifecycle. Figure 1.0-1 provides an overview of this multidimensional M&S challenge. This challenge can be likened to the failure of the Tower of Babel, or to the success of the evolution of Chemistry from Alchemy, by the single factor of whether or not a common language (or characterization taxonomy) can be established for the management of M&S assets across the multiple organizations, disciplines and missions of the Constellation program. The upper part of Figure 1.0-1 is divided into three sections across the page. The left side depicts the hierarchical breakdown of architectural elements comprising the systems that must be designed and built to support constellation campaign operations (in the middle). A campaign may have multiple missions, and each mission is made up of multiple operational segments. Architectural elements are used in multiple combinations and re-combinations across these segments to accomplish the sequence of segment operations. The design and operational details of each campaign will evolve across program time, from initial concepts to launch and space operations, as depicted on the right side. This program evolution is planned in program phases, with each phase requiring increasing detail and confidence from the M&S used to support the design reviews and decision-making processes. The lower part of the figure shows 1) the organization viewpoints of, and 2) the M&S application requirements for, design and operational evaluations supporting program decision making across program phases and milestone reviews. Each of the concepts introduced by this figure are further defined on following pages. Figure 1.0-1: The Multidimensional M&S Challenge of NASA Constellation Program

1.1 Dimensions of System Architectures The NASA Constellation Architecture is comprised of multiple classes of systems, each with its own hierarchical breakdown. These system classes include Crewed Systems, Launch Systems, Space Communication Systems, Robotic Systems, Surface Exploration Systems, and so forth, as shown on the left hand side of Figure 1.1-1. For any given analysis or study, the subjects included and excluded at a given level of detail is called the system (study) scope. Scope and detail are orthogonal dimensions. Recall that, as shown in the upper-middle of previous Figure 1.0-1, these systems, with their elements and components, are employed in one or more operational campaigns, missions, segments, and associated environmental domains. Also recall that, as the program evolves, decision-making risks change, and the amount of detail and confidence required from M&S used for decision-supporting analyses must be commensurate with the level of decision risk. The intended use of M&S also determines the amount of model maturity and confidence required. For example, concept evaluation studies require much less fidelity and accuracy than M&S used for in-the-loop tests and analyses. Figure 1.1-1 depicts how model maturity or fidelity ranges across a confidence scale from conceptual (on the left), to operational emulation and in-service analysis (on the right). Note that M&S fidelity is orthogonal to system scope and system levels. This means that any given application of M&S may characterized by scope, level, and fidelity. at each phase of the program. In other words, at any given time in the program evolution, any architectural entity may have one or more of its own unique M&S representation maturity - any of which may be in independent use for unique analyses in any discipline area. This places a very demanding requirement on M&S configuration management (CM), and on data management (DM) of the data inputs and outputs from any M&S asset. The degree of confidence bestowed on an M&S asset is established by a verification, validation, and accreditation (VV&A) activity, which is described in an accompanying paper [06F-SIW-090]. Following sections in this paper show the context of VV&A as an M&S asset characterization factor. Figure 1.1-1: Orthogonality of Constellation System Levels, System Scope, and M&S Fidelity Dimensions

1.2 Dimensions of Constellation Operations Figure 1.2-1 shows the essential dimensions and interrelationships for constellation operations. Constellation operations consist of a series of exploration campaigns (e.g. return to the moon), with each campaign comprised of one or more missions. Each mission is further subdivided into a sequence of operational segments across the mission s time sequence. A mission may have multiple, overlapping segments, with architectural staging interdependencies between them. Segment operations may be further classified as to their operational environment domains, as shown on the left hand side of the figure. Each environmental domain has their own unique physics and operational constraints associated with them. M&S designs must account properly for operational domain attributes, and for the multiple combinations of interfaces between architectural elements across mission segments. Figure 1.2-1: The Dimensions of NASA Constellation Program Operations 1.3 Dimensions of Program Planning NASA s program development lifecycle is designed for an incremental buildup of essential Constellation systems and their components across six well defined program phases and associated management review cycles, from concept definition to flight operations. Each of these major reviews is preceded by one or more design analysis cycles (DACs), each of which has explicitly defined analysis tasks to support the design, trades, development, build, integrate, test, and evaluate activities associated with each program segment. M&S assets must be accredited to provide the appropriate accuracy of data for each DAC analysis need. Figure 1.3-1 shows the essential elements of this program review sequence and integration. Figure 1.3-1: The Dimensions of NASA Constellation Program Planning 2.0 Objectives for Integrated M&S Asset Management Across Multiple Program Dimensions There are two key management objectives for the successful integrated M&S asset management across the multiple program dimensions illustrated in the previous section. The primary objective is to assure that M&S are applied correctly in every dimension. This in turn requires that the proper M&S asset, among all available, is chosen for application to the analysis problem. This requires the matching of a precise analysis problem description to a precise M&S capabilities description (see Section 3). If the proper M&S asset is matched to the analysis, it must still be assured that the M&S asset is designed, built, and working properly. If both these are true, then it must still be assured that the M&S asset is used properly. This means that the analyst using the asset must be qualified to use the asset on the subject problem. The secondary objective is to assure the effective utilization of M&S resources across all program dimensions. This requires M&S asset management information be collected, organized, and made available to all M&S participants, in all program dimensions. Any gaps between M&S-needs and M&S qualified-asset availability will reveal requirements for M&S asset development or acquisition. Any overlaps indicate redundancies leading to opportunities for establishing common assets instead of duplicate assets. Resolution of such gaps and overlaps will assure efficient M&S resource utilization. The ability to gather appropriate M&S asset management information to accomplish these objectives is totally dependent upon a common M&S characterization taxonomy, as explained in the next section.

3.0 The Steps for Solving the M&S Management Information Challenge The successful multidimensional management of M&S assets for the NASA Constellation program is critically dependent upon a common way to characterize the analysis-needs for M&S consistently with characterization of the inventory assets for M&S, so that available assets may be properly matched to M&S needs. Figure 3.0-1 illustrates this requirement. The right side of the figure lists the M&S information management needs, and the left side of the figure shows the information management process interrelationships between these needs. The information needs are subdivided into two groups, A) M&S ASSET management information needs, and B) M&S WORKFLOW management information needs. M&S asset management is concerned with data related to technical qualifications & applications of M&S assets. Workflow management is concerned with operational and organizational data related to the status of development, accreditation, distribution, application, and maintenance activities for M&S assets. The M&S asset and workflow information needs listed in the two boxes are necessary but insufficient. They are insufficient if they are not accomplished using a common characterization taxonomy for M&S needs and assets. This means there is a missing step (step 0) to provide this common M&S characterization taxonomy. The left hand side of the figure puts these seven steps (0 through 6) into a process flow chart showing their sequence and interdependencies, and shows how the M&S Management Database (MSDB) enables the information sharing among all M&S participants in all dimensions of M&S applications. This database satisfies the management information requirement of step 6. In summary, all aspects of M&S asset management and M&S workflow management are critically dependent upon a common M&S characterization taxonomy (Step 0). This taxonomy must be consistent across all dimensions of the M&S management challenge described in previous sections. The next section describes typical information sharing activities enabled by the MSDB. This discussion is followed by a strategy leading up to the design of the M&S Characterization Taxonomy. Figure 3.0-1: Seven steps for solving the Multidimensional M&S Management Information Challenge

4.0 M&S Characterization Taxonomy Required for Design of an M&S Management Database (MSDB) Figure 4.0-1 shows how the M&S asset management and workflow management process described in the previous section is supported by the M&S Management Database (MSDB). The right-hand side of the figure provides some examples of how the MSDB, built upon the common M&S characterization taxonomy, supports a variety of M&S participants across multiple dimensions of Constellation M&S management. The database use activities illustrated on the right side of the figure are divided into two types, A) technical M&S activities (down the left half), and B) M&S support activities (down the right half). Technical activities are those associated with the selection, development/upgrade, application, and VV&A of M&S assets. Support activities are those associated with the management of M&S workflow activities and development, support, and maintenance of the MSDB. The left hand side of Figure 4.0-1 shows the interrelationships of M&S management information, from needs (at the top) through development and application (at the bottom). The M&S characterization taxonomy (step 0) is a critical factor behind every information function. Step 1 is the definition and quantification of M&S needs, derived from DAC Task Description Sheets (TDSs). Step 2 is the establishment of an M&S asset inventory, by using survey forms based upon the characterization taxonomy. Step 3 is the match-up of M&S assets with M&S needs now possible due to the common taxonomy. Step 3 reveals matches, gaps, and overlaps, providing information for the M&S Support Plan (MSSP), which is developed in Step 4. Step 5 encompasses all activities (and corresponding status information) related to the acquisition, development, VV&A, test, application, and maintenance of M&S assets. Step 6 encompasses the specification, development, implementation, maintenance, and user-support functions for the MSDB. Figure 4.0-1: Common M&S Characterization Taxonomy Supports M&S Asset Management and Workflow Management Database Design

5.0 M&S Asset and Workflow Management Across Dimensions of M&S Lifecycle Activities Figure 5.0-1 expands upon the M&S asset management and workflow information management described in Section 4.0, and shows how this information is related to M&S lifecycle activities for M&S application, development, and VV&A. The left side of the figure repeats the M&S management process flow described in previous sections. The right side of the figure is divided into three M&S lifecycle processes, all interrelated; 1) M&S asset application process, 2) M&S development (or upgrade) process, and 3) M&S VV&A process. Each of these M&S activities has it s own unique M&S community of practice and information needs, which comprise the aggregate needs for the management information support by the MSDB. The common M&S characterization taxonomy assures consistency of M&S management information across all M&S communities of practice, all M&S needs and applications, and all M&S lifecycle activities. 5.1 M&S Asset Application For the M&S asset application process, key M&S requirements come from the data analysis needs specified in analysis task description sheets (TDSs) aligned with each Design Analysis Cycle (DAC). Any chosen M&S asset must be accredited for this use, and the M&S operators and data analysts must be qualified as well. The decision maker who consumes the data reports produced from this M&S data may be the accreditation and qualification authority. Data management (DM) must be maintained on the data flow paths throughout an analysis, and configuration management (DM) must be maintained on all M&S assets used. 5.2 M&S Asset Development Any M&S acquisition, development, or upgrade activity in support of each specified M&S need must be synchronized with the corresponding VV&A application activities. The M&S Support Plan (MSSP) document (shown in the lower left) is the key source for M&S asset development and management. There is an MSSP for each Constellation project, and it is updated for each program development phase. 5.3 M&S VV&A The extensiveness of VV&A activities required for any M&S asset are determined in accordance with the risk associated with the intended application. This is explained in accompanying paper [06F-SIW-090]. The VV&A activities must be planned to synchronize with the M&S asset development and M&S asset application, as shown in the process flow of the figure. Figure 5.0-1: M&S Asset Management Across Dimensions of M&S Lifecycle Activities

6.0 M&S Ontology Precedes Development of M&S Characterization Taxonomy Figure 6.0-1 shows the basic layout of an ontology used to support the development of the common characterization taxonomy for M&S assets. This ontology diagram emphasizes M&S haves versus M&S needs. Understanding these information relationships precedes the development of a common taxonomy. This ontology is a small subset of an overall M&S characterization ontology that supporting the development of the M&S characterization taxonomy. The figure is divided into five blocks of M&S asset attributes. Each block shows a different view of the M&S asset and its attributes. Collectively, this information is useful to search for assets with certain characteristics, match assets with needs, and manage assets across all dimensions. The middle block, M&S VV&A, shows how simulations are comprised of software-coded models, algorithms, and an execution framework. A simulation also has setup data, input data, and output data, all correlated with the data needs of the analysis problem for which it is used. VV&A assessment of an M&S asset accounts for the correctness of these factors, as well as the quality and correctness of the M&S artifacts in the lower right block. The lower right block, M&S Management Information, shows the characteristics of an M&S asset in terms of status information (artifacts) about how the asset has been (or is being) used, and it s development and VV&A status. The lower left block (grayed-out) shows Software V&V activities for the models, algorithms, and execution software framework for a simulation. Software V&V is a separate function from M&S VV&A, and the degree of software V&V is another factor considered in assessing the VV&A status of an M&S asset. The upper horizontal block (blue outline) shows M&S characteristics key to the technical and analytical needs that an M&S asset must satisfy to support an analysis problem. The upper left block addresses the information needed for analyst/user qualification to operate the simulation for a specific analysis-need on a specific analysis problem. Note that an analyst qualified to operate a simulation is not necessarily an analyst qualified to work the analysis problem, and vice versa. However, it is quite common for one analyst to be qualified for both. The next section provides other, complimentary ontological views. Figure 6.0-1: M&S Ontology Of Haves Versus Needs Precedes Development M&S Taxonomy

6.1 Further Ontological Views of M&S Characterization Figure 6.1-1 shows additional ontological views of M&S characterization data. The upper left block shows the context of program decision-data needs. The next block down shows the context of problem domain (systems engineering) data needs, which drives the right hand block for data-confidence requirements, and the input and setup conditions for the technical/analytical M&S application requirements in the next block down. The lower right block shows how VV&A requirements come from the data-confidence needs in the block above it, and are also linked to the organizational/ disciplines domain in the lower left block. This figure, and the previous figure, show how M&S characterization data attributes are interrelated, thus supporting the development of a common M&S characterization taxonomy, and supporting the establishment of a schema for the MSDB. In summary, a multidimensional mapping of the complete M&S lifecycle, and associated activities in related communities of practice, supports an understanding of M&S asset management and workflow information needs, which leads to development and understanding of an M&S characterization ontology, which leads to an development of an M&S characterization taxonomy. Figure 6.1-1: Further ontological views of M&S characterization

7.0 Draft M&S Characterization Taxonomy Figure 7.0-1 shows the top level view of the draft M&S characterization taxonomy resulting from the M&S dimensions, processes, and ontology discussed in previous sections. For convenience, the taxonomy is divided into two main sections, 1) Identifying Characteristics (on the left side) and 2) Confidence Characteristics (on the right side). Identifying characteristics are those M&S attributes that help define and describe the nature of the asset as well as define and describe who is involved (and how) in an asset s development and use. Confidence characteristics are those M&S attributes that are associated with assessing the ability, of any given M&S asset application and execution, to generate data at a confidence level that is commensurate with the analysis needs. The use of this set of characteristics in conducting M&S VV&A for the NASA Constellation program is discussed in a companion paper entitled Incremental Quantification of VV&A Levels for Integrated Management of M&S Tools [06F-SIW-090]. Figure 7.0-2 provides the partially expanded detail of the M&S characterization taxonomy. Figure 7.0-1: Top Level View of Draft M&S Characterization Taxonomy for NASA Constellation Program 8.0 Summary The NASA Constellation Program is a large, diverse enterprise, encompassing multiple locations, missions, communities of practice, products, technologies, and development lifecycles. Accordingly, effective integrated management and controlled quality of M&S resources supporting this program requires a cohesive plan across multiple dimensions. This in turn requires a common M&S characterization taxonomy that will serve across multiple communities of M&S planners, developers, and users. A common M&S characterization taxonomy provides a critical common language for specifying analysis data needs, specifying requirements for M&S tools supporting these data needs, specifying M&S assetsurvey templates that will properly inventory existing M&S assets, their applicability, and their availability, and supporting the effective match-up of analysis needs to M&S needs to M&S assets. The inter-community communication consistency provided by a common M&S characterization taxonomy supports greater visibility and control, and thus effectiveness, for integrated management of M&S asset development and application across the Constellation program. This paper has described the development of a common M&S characterization taxonomy being developed for NASA s Constellation program to support effective integrated management of M&S. In summary, a multidimensional mapping of the complete M&S lifecycle, and associated activities in related communities of practice, provides an understanding of M&S asset management and workflow information needs, which leads to development and understanding of an M&S characterization ontology, which leads to development of a common M&S characterization taxonomy, which leads to a relational database schema supporting an M&S management information database design, which supports the effective integrated management of M&S assets across the Constellation program enterprise and lifecycle.

Figure 7.0-2 Partially Expanded detail of the draft M&S characterization taxonomy.

Author Biographies BOBBY HARTWAY is a Senior Research Scientist with AEgis Technologies Group in Huntsville, Alabama. He has developed a new paradigm for simulation characterization and requirements development for space and defense systems. He is using this paradigm to support NASA s activities for integrated management of modeling and simulation. DANNY THOMAS is a Senior Research Scientist with AEgis Technologies Group in Huntsville, Alabama. He is supporting NASA s effort to institute consistent management practices for simulation development and use. He has developed simulations for space and defense. LISA CAIN is a Systems Engineer with AEgis Technologies Group in Huntsville, Alabama. She supports the Exploration Mission Space Directorate (ESMD) Integrated Modeling and Simulation (IM&S) VV&A Program. Ms. Caine earned a BS degree in Engineering and Information Systems from Michigan State University and a Masters in Project Management from Keller Graduate School of Management in Chicago. She is certified as a Project Management Professional by the Project Management Institute (PMI). JOE HALE is currently Lead for the Exploration Systems Mission Directorate s Integrated Modeling and Simulation Verification, Validation, and Accreditation activity. Prior to that he was the Lead Systems Engineer for the Next Generation Launch Technologies Advanced Engineering Environment. Much of his prior work at MSFC was as a Human Factors Engineer, working various projects, including Spacelab and the International Space Station. He spent five years as Team Lead for the Human Engineering and Analysis Team. Mr. Hale is a Certified Human Factors Professional (CHFP) (Board of Certification in Professional Ergonomics), is a founding member and first president of the Tennessee Valley Chapter of the Human Factors and Ergonomics Society (HFES).