A Case Study of the Systems Engineering Process in Healthcare Informatics Quality Improvement By Ali M. Hodroj Project Report submitted to the Faculty of the Maseeh School of Engineering and Computer Science at Portland State University in partial fulfillment of the requirements for the degree of Master of Engineering In Systems Engineering Approved: Professor Herman Migliore, Director of Systems Engineering 1
Table of Contents Contents ABSTRACT...3 INTRODUCTION...4 1.0 PROBLEM STATEMENT... 4 1.1 OBJECTIVE AND PROCEDURE... 4 1.2 PROBLEM CONTEXT... 4 1.3 JUSTIFICATION FOR A NEW SYSTEM... 4 SYSTEMS APPROACH TO HEALTHCARE INFORMATICS...5 2.0 HEALTHCARE AS A COMPLEX SYSTEM... 5 2.1 [OMITTED PROPRIETARY INFORMATION]... 5 SYSTEMS ENGINEERING PROCESS...6 3.0 SYSTEMS ENGINEERING OVERVIEW... 6 3.1 PROCESS OVERVIEW... 6 3.2 CONCEPT/CONTEXT EXPLORATION VIEW... 9 3.3 FUNCTIONAL ANALYSIS VIEW... 10 3.4 REQUIREMENTS ANALYSIS VIEW... 10 3.5 ARCHITECTURAL ANALYSIS VIEW... 11 3.6 TEST VIEW... 12 SYSTEM CONCEPT... 14 4.0 CURRENT SYSTEM... 14 4.1 MISSION STATEMENT... 14 4.2 PROPOSED SYSTEM CONCEPTUALIZATION... 14 4.3 CONCEPT DEFINITION... 14 4.4 ITERATION 2: HIGH-LEVEL SYSTEM... 14 4.5 ITERATION 3: [OMITTED PROPRIETARY INFORMATION]... 15 4.6 ITERATION 4: [OMITTED PROPRIETARY INFORMATION]... 15 4.7 ITERATION 5: [OMITTED PROPRIETARY INFORMATION]... 16 4.8 ITERATION 6: [OMITTED PROPRIETARY INFORMATION]... 16 CONCLUSION... 18 REFERENCES... 19 2
Abstract A systems engineering approach was used to improve and develop a clinical quality improvement system for a major biomedical informatics solutions provider. Development efforts spanned the system life cycle and involved project stakeholders. Requirements were developed and alternative designs where conceptualized. The best alternative design concept was selected based on research and analysis in the view of requirements. A detailed design was embodied in a prototype system that was evaluated and found not to meet requirements completely. The design was revised, including a tradeoff where total system value as well as additional trade-off criteria were incorporated to satisfy a constraint regarding leveraging an existing product within the healthcare provider market space. The revised system design was validated and approved by project stakeholders within the company. The project progressed through different iterative systems engineering life cycle phases, where the conceptualized solution was viewed in context of requirements, functions, possible architectures, and validation and verification approaches. The same four-fold process was applied recursively at each sub-system to facilitate the proper selection of solution concepts at different layers of abstraction. The purpose of this paper is to present a case study of the systems engineering process and how was it was applied to ensure that the proposed system met all project stakeholder requirements. 3
Introduction 1.0 Problem Statement This masters project report is the outcome of an academic case study and industrial work conducted in the light of the Master of Engineering program in Systems Engineering at Portland State University. The scope of this project encompasses the process of following a systems approach to the design of a healthcare quality improvement system based on the implementation of a quality function deployment process, a systems engineering tool, through clinical quality reporting of clinical performance measures for healthcare providers. The process consists of (1) systems requirements definition, (2) specification and collection of data through systems modeling, (3) design, validation, and verification of appropriate system models, (4) trade-off analysis for selecting the most balanced design alternatives, and finally (5) the development of implementation and evaluation plans for meeting the target system functions, quality attributes, and performance metrics. 1.1 Objective and Procedure The objective is to follow a systems approach at implementing a performance measurement framework that can report on healthcare delivery quality as well as indicators for clinical quality improvement. Form a systems engineering perspective, a performance measure is a statistic computed from information in a given system trace. 1.2 Problem Context 1.3 Justification for a new System 4
Systems Approach to Healthcare Informatics 2.0 Healthcare as a Complex System The decision to take a systems approach towards a healthcare informatics system stems from the fact that the US Healthcare is itself a complex system of many interdependencies and heterogeneous components. The United States has a unique system of healthcare delivery. It is unlike any other health care system in the world. Most developed countries have national health insurance programs run by the government and financed through general taxes. Almost all citizens in such countries are entitled to receive healthcare services. Such is not the case in the United States, where not all Americans are automatically covered by health insurance. The US healthcare delivery system is not a formal system in the true sense, even though it is called a system when reference is made to its various features, components, and services. The US healthcare system is unnecessarily fragmented due to continuous periodic changes, mainly in response to concerns with cost, access, and quality [8]. Figure 1 demonstrates the complexity of healthcare delivery in the United States. Many organizations and individuals are involved in healthcare. These range from educational and research institutions, medical suppliers, insurers, payers, and claims processors to healthcare providers. Figure 1: US healthcare system structure 2.1 5
Systems Engineering Process 3.0 Systems Engineering Overview Systems engineering is a methodology and framework for managing technological complexity. The application of systems engineering revolves around three core principles: systems management, requirements/architecture definition, as well as systems integration and verification. Systems engineering focuses on the system as a whole by holistically emphasizing its total operation. It assumes a non-reductionist engineering perspective by looking at a system from the outside through its interactions with other systems and the environment that it operates within. Such approach demands not only the engineering design of the system, but also the external factors that govern and constrain the design. Systems engineering also adopts a management perspective in order to control the total system development effort for the purpose of achieving an optimum balance of all system elements. It is a process of transforming an operational need into a description of systems parameters and integrates such parameters to optimize the overall system effectiveness. 3.1 Process Overview The systems engineering process applied in this project is adapted from a recursive systems engineering lifecycle pattern, CCFRAT: Context, Concept, Functions, Requirements, Answers, and Test [1]. CCFRAT is a hierarchical systems engineering methodology applied at every level of system abstraction, design, and decomposition. This methodology is used to develop the system boundaries at each level of system interaction, abstraction, and/or design. The CCFRAT process is a modification of the Function, Requirement, Answer, and Test (FRAT) simplified systems engineering process [3]. The process is named Context, Concept, FRAT (CCFRAT), after the six views of any system required by this method. The CCFRAT model was developed by adding two additional holistic views to the classic FRAT model. The new holistic views are the context view and the concept view. These two new views focus on the value set constraining the total system context and the controlling concept set associated with the relationships contained in the core design process [1]. With these six views in place, the systems engineering process proceeds in a recursive, stepwise fashion to design, model, and develop systems at lower and lower levels of details until the system has been fully described and modeled. In other words, the conceptual, contextual, functional, requirements, architecture, and validation sum of all views at the lowest level of details shall be equivalent to those six views at the highest level of detail. The first step in a CCFRAT process is the establishment of the system context, including a definition of a current system abstraction frame. The system context view is focused on the external relationships and connections associated with the system of interest. This view records the driving need for the system as well as the system value network, external stakeholders and controlling value sets. The figure below provides a conceptual model of the system context view: 6
Figure 2: Conceptual model of a system context view Once the system context is established, a conceptual view of the system is derived. The conceptual view focuses on the system interest as well as its current level of abstraction, level of decomposition and abstraction boundary for a specific set of system concepts. The core four system views, function view, requirements view, architecture view, and test view can be evaluated, analyzed, documented, and modeled in a number of ways. The concept view establishes a wellcoordinated and executable set of rules for system definition and analysis. The concept view details the semantics, processes, and approach used to evaluate and analyze the system represented in the current context. The following figure outlines the conceptual model of a system context view: Figure 3: Conceptual model of a system concept view 7
As the six views (Context, Concept, Functions, Requirements, Architecture, and Test) are being developed, a logical mapping between each of the layers is established as shown in Figure 4 below: Context, Concept, Functions, Requirements, Answers, Tests CCFRAT CCFRAT CCFRAT CCFRAT CCFRAT CCFRAT CCFRAT CCFRAT CCFRAT governs System Concept C Sub-System Concept C.1 Sub-System Concept C.2 + + Sub-System Concept C.3 System Function F Concept C.1.1 Concept C.1.2 Concept Concept + C.2.1 C.2.2 + Concept C.3.1 Concept C.3.2 Sub-system Function F.1 Sub-system Function F.2 + + Sub-system Function F.3 + + Function F.1.1 Function F.1.2 Function F.2.1 Function F.3.1 Function F.3.2 Figure 4: CCFRAT recursive systems engineering process Each system in a CCFRAT process is has six views that are used to identify the logical relationships important to the system at each level of abstraction and development. These six views are, the context view (defines the system context and meta-system), the concept view (defines the conceptual relationship inside the system), functional view (defines what the system does), requirements view (defines how well the functions are done), the answer view (defines how the function is performed), and the test view (defines how you know that the answer meets the functions and stated requirements). As CCFRAT methods and processes are used to recursively model and design the system of interest, system models and descriptions are developed at lower and lower levels of detail. At the same time, the number of unique system solutions is reduced and the design space is further constrained. At each abstraction level the system is modeled and evaluated against the given concept and value constraint context. 8
Figure 5: CFRAT process used in this case study This CCFRAT systems engineering approach was followed, but tailored as indicated in Figure 5 to better suit the scale of the this project and the incremental approach used to develop the overall target system through a sequence of individual iterations. For example, the Context and Concept stage where combined in an overall Concept Exploration phase based on the scope of each individual sub-system. There were also continuous opportunities to involve key end users throughout different project phases to validate that the system would meet their needs. The concept of operations was incrementally updated when needed to reflect the specific effects of individual iterations as they were concluded. 3.2 Concept/Context Exploration View The system concept/context view details the relationships between the system functions and architectural design as well as specifying the appropriate level of abstraction for system modeling. Both concept and context views are combined together to emphasize the system boundaries and provide a common abstraction layer for the complete system under development. With these two holistic views in place, relationship models, structural models, and other types of non-hierarchical system interactions can be more readily evaluated without undergoing detailed functional analysis of each component in the proposed system. Figure 6 below shows how this view is decomposed at different levels of abstraction. 9
Figure 6: recursive concept decomposition at different levels of abstraction 3.3 Functional Analysis View Functional analysis creates a function hierarchical pattern that has equivalent functional behavior at each level in the functional abstraction hierarchy. At every level of abstraction in the system, functional analysis is performed to decompose the system under analysis into a set of functions that satisfy the original system objective. Figure 7 below shows how this view is decomposed at different levels of abstraction. Figure 7: recursive functional decomposition at different levels of abstraction 3.4 Requirements Analysis View The purpose of the requirements analysis view is to obtain a thorough and detailed understanding of the business need and constraints at every level of abstraction in the system, and to break it down into a discrete set of measurable requirements, which are then clearly defined, reviewed, and agreed upon with the project stakeholders. The primary goal of this view is to create a detailed set of specifications on how well the system must perform. These specifications form the constraints, non-functional requirements, and quality attributes of the system. Within our CFRAT process, the requirements are used to construct an architecture utility tree which is later used for criteria to perform trade-off analysis between different proposed solution architectures. Figure 8 below shows how this view is decomposed at different levels of abstraction. 10
Figure 8: recursive requirements decomposition at different levels of abstraction 3.5 Architectural Analysis View The architectural analysis view is two-fold: (1) identification of the appropriate solution answers to which all current system functions can be allocated and requirements can be met, and (2) identification of a set of architecture alternatives that are compared and synthesized through an architecture trade-off analysis method. This phase evaluates the solution concepts in order to understand the consequence of each choice decision with respect to meeting requirements and system functions. A result of this phase is identifying areas of potential risks within the architecture of the complex system. In this view there are three main activities performed at each level of abstraction: (1) identification of architectural approaches (solution concepts/answers) that meet the defined functions in the Functions view, (2) generation of a quality attribute tree that maps to the Requirements view, and (3) analyzing and comparing architectural approaches through tradeoff and sensitivity analysis where the quality tree attributes are prioritized and used as the main trade-off analysis criteria. This process is an adaptation of the Software Engineering ATAM (Architectural Analysis and Trade-off Method) pioneered by the Software Engineering Institute (SEI) at Carnegie Mellon University [2]. The principal benefit of this method is the ability to see if a proposed overall architecture will live up to the requirements expectations. The trade-off analysis helps one understand the strengths and weaknesses of the system. More so, this view within our process forms the core of applying systems engineering principles where constraints are put under investigation for two primary reasons: (1) to investigate and prioritize the importance of each constraint, and (2) to investigate the interdependencies between constraints and how the relationship between them affects the decision to select the appropriate solution concept. The result of this is to provide the systems engineer with a set of trade-off points acting as gauges that the stakeholders can monitor every time a solution concept is evaluated. The following sections summarize three main categories that requirements are categorized under in order to generate a utility tree for trade-off analysis End User Quality Attributes Performance The degree to which a system or component accomplishes its designated functions within a given set of constraints, such as speed, accuracy, or memory usage. Availability The degree to which a system or component is operational and accessible when required for use. Usability The ease with which a user can learn to operate, prepares inputs for, and interprets outputs of a system or component. 11
Security The degree to which a system is able to protect its data, restrict access, as well as authorize and authentication certain users. Engineering Quality Attributes Maintainability The ease with which a system or component can be modified to correct faults, improve performance or other attributes, or adapt to a changed environment. Portability The ease with which a system or component can be transferred from one hardware or software environment to another. Reusability The degree to which a system or other work product can be used in more than one system. Testability The degree to which a system or component facilitates the establishment of test criteria and the performance tests to determine whether those criteria have been met. Business Quality Attributes Time to Market The length of time it takes from a product being conceived until it s available for production. Cost and Benefits Potential costs and revenues that may be incurred or generated as a result of selecting a specific system or component. Project Lifetime The intended lifetime of the system in production. Targeted Market The target market segment that the system will operate in. Legacy Integration The ability to integrate the proposed system or component with current existing systems. 3.6 Test View The purpose of the test view within this process is to identify the verification and validation requirements for the system. Verification is the confirmation that the selected solution concept fulfills the requirements and functions allocated to it. Validation is the confirmation that requirements are correct and that the functions and requirements, when taken in combination, satisfy the system-level mission needs. Validation and verification activities take place as part of each CFRAT phase within our systems engineering process. The approach to verification and validation is accomplished through several methods: Testing Exercising and observing system behavior by comparing its outputs to expected outcomes. Demonstration Prototyping of the system through an engineering effort to implement a subset of its functionality that can be used to verify whether the concept is technically feasible or not. Similarity Comparing computational results from the system or component against results obtained from an identical system. 12
Analysis/Inspection The analysis of the static system representation to discover problems or potential areas of risk. Simulation and Modeling The simplified representation of the system behavior through a computational model in order to develop a level of understanding of the interaction of the parts of the system as well as the system as a whole. 13
System Concept 4.0 Current System 4.1 Mission Statement 4.2 Proposed System Conceptualization 4.3 Concept Definition After the high-level stakeholder needs where identified and prioritized, alternative concepts were defined at the highest level of abstraction in the system. Conceptual designs where developed from these alternatives for consideration resulting in the following possible concepts: 4.4 Iteration 2: High-Level System Context Functions Requirements Architectures Utility Tree 14
4.5 Iteration 3: Concept Functions Requirements Architectures Test 4.6 Iteration 4: Functions Requirements Architectures Utility Tree 15
Test 4.7 Iteration 5: Functions Requirements Architectures Utility Trees Test 4.8 Iteration 6: Context [Omitted Proprietary Information] Functions Requirements Architectures Utility Tree 16
Test 17
Conclusion In this case study, systems engineering was used to capture all stakeholder needs in the most effective way possible. This ensures that all requirements were gathered and consistent across all stakeholder needs and in alignment with the primary system objective. The definition of the system was carried out through a hierarchical recursive systems engineering process. The definition of the system was completed through conceptual, functional, requirements, architectural, and validation decomposition. At each decomposition layer in the system hierarchy, system views were based-lined with customers and stakeholders before the next level of analysis is started. Roll-up reviews were also conducted to make sure that the top-level system objectives have not been modified in the subsequent analysis. The end result yielded a definition of a system that is sufficient enough to provide the development team with the definition of the system to be delivered and how it will operate in a production environment. The utilization of a systems approach to the development of a clinical performance measurement system proved to be successful and recommended. The end result of the approach was meeting all stakeholder needs in an iterative fashion while reducing system complexity and balancing the system s quality attributes in the most effective way possible. 18
References [1] Simpson, J.J. and Simpson, M.J., "System Integration Frameworks." Proceedings of the 15 th Annual International Symposium of the International Council on Systems Engineering (Rochester, NY, July, 2005). [2] Abowd, G., Bass, L., Clements, P., Kazman, R., Northrop, L., and Zaremski, A. Recommended Best Industrial Practice for Software Architecture Evaluation. Technical Report, Software Engineering Institute, CMU/SEI-96-TR-025, ESC-TR-96-025, January 13, 1997. [3] Mar, B.W., and Morais, B.G., "FRAT A Basic Framework for Systems Engineering." Proceedings of the Twelfth Annual International Symposium of the International Council on Systems Engineering, Las Vegas NV, 2002. [4] Drools. online. http://labs.jboss.com/drools/. [5] Physicians Quality Reporting Initiative. online. http://www.cms.gov/pqri/ [6] Linder, J.A., Ma, J., Bates, D.W., Middleton, B., Stafford, R.S. Electronic Health Record Use and the Quality of Ambulatory Care in the United States, Archives of Internal Medicine, July 9, 2007; 167 (13): 1400-1405. [7] O Toole MF, Kmetik KS, Bossley H, Cahill J, Kotsos T, et al. Electronic Health Record Systems: The vehicle for implementing performance measures. American Heart Hospital Journal. 2005 (3):88-93. [8] McGlynn E, Asch S, Adams J, et al. The quality of health care delivered to adults in the United States. N Engl J Med. 2003;348(26):2635 45. [9] Institute of Medicine (U.S.). Committee on Quality of Health Care in America. Crossing the quality chasm: a new health system for the 21st century.washington, D.C.: National Academy Press; 2001. 19