ENGINEERING CHANGE MANAGEMENT - VERIFICATION, VALIDATION, AND TESTING PLANNING TOOL DEVELOPMENT

Transcription

1 Proceedings of TMCE 2014, May 19-23, 2014, Budapest, Hungary, Edited by I. Horváth, Z. Rusák Organizing Committee of TMCE 2014, ISBN ENGINEERING CHANGE MANAGEMENT - VERIFICATION, VALIDATION, AND TESTING PLANNING TOOL DEVELOPMENT Keith Phelan Mechanical Engineering Clemson University United States of America [email protected] Joshua D. Summers Prof. of Mechanical Engineering Design Clemson University United States of America [email protected] Paolo Guarneri Mechanical Engineering Polytechnic University of Milan Italy [email protected] ABSTRACT The majority of products will undergo numerous changes over the course of their product lifecycles, many of which will occur while the products are already in production. Research has shown that these engineering changes can propagate throughout the product of system and affect the functionality of the other components. Previous research in this lab has developed a 7-step method to generate a validation, verification and testing (VV&T) plan to assist change engineers in mitigating the negative effects of engineering change propagation. The proposed method can be a time-consuming and exhaustive process to execute, depending on the scope of the engineering change and the complexity of the system/product in question. As such, a computational support tool was developed that will assist the change engineering in the implementation of the method. The support tool will also assist in the documentation of the engineering change process and will help to minimize the opportunities for human error that can occur. In order to validate the execution of the support tool, an industry case study was used from the original research for the development of the 7-step method. The computational support tool successfully aided the development of a VV&T plan for the execution of an engineering change in an industry case study. However, additional testing is necessary to validate the claims. KEYWORDS Engineering change, change propagation, engineering change management, documentation support 1. INTRODUCTION Over the course of its lifecycle, a typical product will undergo a large variety of changes. Each of these changes will have a direct effect on the performance of the product, but could also cause additional changes to other components within the system due to the interactions between the changed component and the affected components. Previous research [1] has led to the development of a method for analysing how changes can propagate through the system and developing a validation, verification and testing plan to mitigate the negative effects of the change. The purpose of this paper is to propose a computational support tool to assist in the implementation of the aforementioned planning method. In the following sections, the authors will provide a background to the engineering change process and common engineering change management practices. Section 4 will consist of a discussion of the previous research in engineering change propagation. Sections 5 and 6 discuss the development of the support tool and its implementation, respectively. Finally, an evaluation of the usefulness of the tool and the conclusions and future work are presented. 587

2 2. ENGINEERING CHANGE In defining the term engineering change, three common definitions are as follows: An engineering change (EC) is a modification to a component of a product, after the product has entered production [2] [Engineering changes are] the changes and modifications in forms, fits, materials, dimensions, functions, etc. of a product or a component [3] Engineering change orders (ECOs) [are] changes to parts drawings or software that have already been released [4] For the purposes of this research, the first definition of engineering change is used, focusing on the changes to products that have already entered production. It should be noted that none of the above definitions take into account the origin or size of the change. Therefore, it is important to add that an engineering change can be any size or type and can involve any number of people or amount of time. It is also important to consider that engineering changes can occur during any stage of a product lifecycle [5]; the stage at which the change occurs can greatly affect the actions required by the company. Oftentimes, companies will use different terminology to describe the engineering change and will use different resources in executing the change [6]. In most cases, the processes will be similar, but the formality of the process used will increase as the level of development increases (often due to the increased number of resources and/or outside agencies involved in the product). Because this research is focused on products already in production, it is expected that the engineering change process to be more formal and complex that if the change occurred earlier in the design process. A comprehensive example of the engineering change process outline can be found in Figure 1. The process is as follows: 1. A request for an engineering change is made, typically on a standard, company form. The engineering change must include the reasoning, priority, type and what components may be affected. The engineering change is sent to a change-controller and is entered into a database. 2. Possible solutions are developed, and, typically, only one solution is further evaluated. This is likely due to time constraints or the acceptance of Figure 1 A generic engineering change process [7] the solution as the obvious answer. 3. The next step is to determine the impact of the engineering change. Factors that must be considered include, but are not limited to: costs, supplier relationships, and scheduling. The later the engineering change occurs in the design process, the more significant the impacts will likely be. 4. Once selected/evaluated, the engineering change must be approved. Many companies utilize an Engineering Change Board/Committee to review the changes, analyzing the costs vs. benefits before granting approval. The Engineering Change Board should consist of company staff members from a variety of functions related to product development, including: product design, manufacturing, supply, marketing, finance, etc. 5. Depending on the nature of the change and the stage of the product life-cycle, the implementation can be immediate or phased over 588 Keith Phelan, Summers, Guarneri

3 time. Paperwork regarding the product/affected components must also be updated at this time. Ensuring that only the correct documentation is available is one of the major current issues regarding engineering changes [4]. 6. One of the most important steps in the engineering change process is the review period, to ensure that the intended effects were achieved as expected. The documents used to support the engineering change process have a variety of names depending on the authors. Some common titles for the documents include Engineering Change Request (ECR), Engineering Change Notice (ECN), Engineering Change Order (ECO), and Engineering Change Proposal (ECP). Despite the discrepancy in terminology, the names all refer to two types of documents: those used to propose a change in a given product (ECR/ECP), and those used to document an engineering change approval and implementation (ECO/ECN).. A common misconception is that all engineering changes occur as a result of technical necessity. However, in a study of German manufacturing firms [8], only 40-60% of engineering changes were technically necessary. In some instances the benefits of executing an engineering change are not cost or performance related, but rather to maintain the goodwill of a key customer [6]. It is therefore important to differentiate between the essential and meaningless changes. As such, it has been shown that changes can be classified into two categories: emergent (product origination e.g. errors/fixes) or initiated (outside the product e.g. customer requests) [9,10]. Common examples of emergent changes include: - Error correction: mistakes made during design that are identified at any point during the product life-cycle - Safety: any situation in which a product must be changed due to not meeting safety requirements; this can include any unintended uses that may be hazardous - Change of function: any situation where the product does not fulfil its functional requirements; this could be due to environment, use, etc. - Product quality problems: problems with rework are often the result of poor manufacturing or assembly processes/instructions Initiated changes include improvements on existing designs and can occur for any number of reasons. As such, in a review of the causes of engineering changes [6], it was more useful to discuss the parties that might initiate change. These include: customers, sales and marketing, product support, production, suppliers, product engineering, company management, and legislators. In a series of studies on aeroengines and drilling machinery [11], it was found that product improvements (initiated changes) were more likely to be the cause of an engineering change early in the product life-cycle, whereas error correction (emergent changes) were more likely to be a factor later in the product life-cycle. In a similar study of an automotive OEM [12], it was found that 77% of changes were due to internal reasons (notably, documentation error and other error rectifications), and 23% due to external forces. It has been found that engineering changes can affect the cost, scheduling and planning of an impacted product [6]. With regards to the costs of implementing an engineering change, many researchers have made use of the Rule of Ten [8], which states that the associated costs increase, on average, by a factor of 10 between each stage of the design process. Terweisch and Loch [4] further decompose project costs into three categories: design, changes in prototype tools and changes in production tools. For example, a specific change in an automotive company affected the product tooling and cost $190,000, while a different change, much earlier in the design process, affected the same component and only cost $10,000 [8]. The increase in costs for products late in the product life-cycle is largely due to the increased number of resources (material, personal, etc.) involved in a given product. For example, a change to a design already in production would likely result in product recalls. During the design phase, engineering changes can result in information deficiencies, even if documentation is kept up-to-date; this can result in design teams working off of outdated data and can lead to a significant amount of rework later on [8,13]. Because of the global nature of modern product development, the design environment has become more distributed, leading to an increase in the possibility of such information gaps [14]. ENGINEERING CHANGE MANAGEMENT - VV&T PLANNING TOOL DEVELOPMENT 589

4 Oftentimes, the impacts of change can be exacerbated due to the effects of change propagation. Change propagation is defined as a phenomenon where one change initiates a series of additional changes [15], and often occurs as a result of interconnectedness between components of a project. Three different types of relationships have been identified that can lead to change propagation: between components and manufacturing, components with the same sub-system and between components in different sub-systems. Shankar, et al. [12] expanded upon the first type of propagation (between components and manufacturing) to include all of the functional silos within a company (manufacturing, packaging, transportation of goods, marketing). 3. ENGINEERING CHANGE MANAGEMENT There has been a large amount of research conducted on ways to mitigate the effects and/or occurrences of engineering change. The research can be categorized according to the following types of mitigation: tools for documentation, tools for decision-making, and engineering change coping strategies Documentation tools The first type of tools to be discussed involves those used for assistance in documentation and managing the work flow of the engineering change process. It is widely agreed that such tools are necessary to effectively and efficiently execute engineering changes [3,16]. Engineering change management systems that are primarily paper-based are typically inefficient in that the information is largely centralized; as the number of engineering changes of a product increases, the situation is compounded [16]. Therefore, there has been a significant trend towards computer-based systems for handling/documenting engineering changes. Huang and Mak [17] use the following classification method for computer-based tools: - Dedicated engineering change management systems. They include databases of engineering change activities and can generate engineering change forms. - Computer aided Configuration Management systems. These systems are software-based engineering change management systems and allow the user to address product structuring and versioning. - Product Data Management (PDM) or Product Life-cycle Management (PLM) systems. These systems incorporate all of the above functionalities and also are able to encompass all stages of the product life-cycle, such as product planning. Typically, the size/scope of these systems requires that they be developed outside of the company by software design companies. There is great potential for this type of system, but as of yet is unrealized. The increase in the use of computer networking in company infrastructures has led to an increase in academic research into computer-based change management systems. One example, a stand-alone, web-based system for managing the engineering change process, has been developed at the University of Hong Kong s Department of Industrial and Manufacturing Systems Engineering [18]. Reddi [19,20] presents a framework for engineering change management based on Service Oriented Architecture that allows for an agile ECM process to be used in a collaborative environment. Despite the prevalence of commercially available engineering change management software packages, it has been found that few companies have moved to integrate these systems [8]. Some possible reasons behind this are [17]: - Companies do not realize the systems are available - Available systems do not meet the needs of the user - Available systems are not worth the difficulty to implement - The systems require too much data input to be time-effective - The technology does not fulfil its functions as promised In their study of three Swedish engineering companies [21], it was found that none of the companies utilized the benefits of computer-based support to their full potential. However, it is understood that at the time of the report that all of the companies were investing in computer-based systems. The biggest question was whether it was more efficient to develop their own software or to revise commercially available software for use by the 590 Keith Phelan, Summers, Guarneri

5 company. In a similar review of two British companies [16], the companies felt that adapting a commercially available system would be more expensive and time-consuming than developing their own Decision-making tools The majority of research on engineering change has been focused on tools to aid in the decision-making of the engineering change process. While CAD modelling, Failure Mode and Effect Analysis (FMEA) and Value Analysis are important tools that can be used in engineering change mitigation, the focus of this section will be on methods/prototypes proposed in academics. Ollinger and Stahovich [22] propose a tool called RedesignIT, a computer program that employs model-based reasoning to create and evaluate proposals for redesign plans. The program uses the relevant physical parameter of the design and the relationships between the parameters to build the model. The benefit to this tool is that it proposes modifications to the proposals to mitigate negative effects of the proposed change. However, it only provides the parameters that should be modified and does not propose how the quantities should be altered. Laurenti and Rozenfeld [23] present a modified version of FMEA that specifically covers the analysis of modifications to a system. The proposed method, Failure Mode and Effect Analysis of Modifications (FMEAM), was developed based on an integration of FMEA and Design Review Based on Failure Mode (DRBFM) and incorporates a multi-disciplinary work group to review engineering changes and the possible failure rates that may be associated with them. At this point, there has been no validation of the feasibility or utility of the proposed method. The Change Predication Model [15] is a tool for predicting how change will propagate through a design. This method uses Design Structure Matrices (DSMs) to build a product model. The product model consists of the relationships between components that increase either the likelihood or impact of engineering change propagation. By determining the possible propagation pathways, it is then possible to use the product model to create DSMs representing the predicted likelihood and risk of a change. From these DSMs it is possible to predict the possible impact of a change. This model is shown in Figure 2. Figure 2 Change Propagation Model (CPM) [15] This method has been used extensively in additional research and has been applied in numerous case studies [24 26]. A similar method has been proposed that uses DSMs to determine the secondorder relationships between requirements [27]. From these secondary relationships, they were able to successfully predict how product requirements would change as a result of an initial requirement change. By modelling the predicted change early in the design process (requirements development), it is possible to minimize the associated costs resulting from an engineering change. The method was proven to be successful in predicting the resulting changes in two industrial case studies, but more validation is needed to prove its effectiveness. Another potential negative of this method is that it requires an initial change in order to be effective. C-FAR (Change Favorable Representation) [28] is a methodology that uses existing product information to facilitate change representation, propagation, and qualitative evaluation. C-FAR decomposes a product into its basic entities and then represents these entities as vectors, with the attributes of the entity as components of the vector. The approach then uses matrices to create relationships between entity vectors, with the individual components of the matrix being referred to as linkage values. The linkage values represent the relationship between two attributes (one from each entity) and can be used to determine how change in one attribute or entity can affect other entities/attributes. The method has been used with numerous industrial case studies, but because of the high processing power required, it is only feasible when used with fairly simple products. ENGINEERING CHANGE MANAGEMENT - VV&T PLANNING TOOL DEVELOPMENT 591

6 4. VERIFICATION, VALIDATION AND TESTING METHOD One issue that is not addressed in previous engineering change mitigation methods is the propagation of change through variant and organization pathways. A seven step Validation, Verification, and Testing (VV&T) method is proposed by Shankar [1] that incorporates these propagation pathways. The proposed method is designed to guide a designer through the development of a VV&T plan that will fully evaluate the change propagation resulting from an initial engineering change. The steps of the method are shown in Figure 3. Figure 3 Seven- Step Process [1] The method has undergone extensive validation in three industrial companies and has been shown to work effectively [1]. However, some issue shave been identified in the application of the method, especially for complex physical systems. In complex systems, there is a large amount of data involved in the process that must be carried over from one step to another. Because much of the process is to be conducted manually, numerous opportunities for data entry error exist, especially when little or no documentation accompanies the process. Additionally, the process has been shown to rely heavily on the experience of the change engineers to properly execute the planning method. The computational support tool described in this paper is intended to address these issues. The development of the support tool will be discussed in the following section. 5. SUPPORT TOOL DEVELOPMENT As stated, the purpose of the computational support tool is to assist a change engineer in executing the validation, verification and testing planning method discussed in the previous section. Because the overall functionality of the tool was explicitly stated, its design and development was fairly straightforward. Initially, it was essential to fully understand the requirements that must be considered. In addition to the primary requirement (the ability to easily guide the engineer through the process) other requirements were discussed to mitigate some of the other issues that have been identified when using the 7-step planning method. One major issue is the large amount of data that must be carried between the steps, leading to the possibility for human input errors. Additionally, the tool should assist in the documentation of the change management process. From the above requirements, the researchers were able to formulate the formal design problem: Develop a computational support tool to guide a change engineer through the 7-step VV&T process while minimizing the opportunity for error and assisting in documenting the change process without the requirement for additional input. An additional requirement that was discussed was the availability of the tool. One reason that support tools developed in academia are not utilized heavily in industry is the resistance to new software or interfaces. In order to ensure easy distribution and use of the computational support tool, it was developed in Microsoft Excel using the Visual basic for Applications programming language. This allowed for simplified implementation while maximizing the functionality of the tool. When developing the support tool, conducted an analysis of what was necessary to minimize the difficulty in executing each step in the process. In addition, it was necessary to consider the limitations of the tool for a given step, specifically, which data could be automatically created and which would require manual input from the user. For example, in the first step (Identify Requirements), it was possible to have the tool import a requirements list from an outside source. However, because the source document was of unknown origin and the affected requirements lists were likely to be simple, it was decided that in the initial version, the step would be 592 Keith Phelan, Summers, Guarneri

7 manually entered. On the other hand, the creation of the DVP matrix is almost completely automated. The only manual input required for this document is the administrative data, such as team members and testing responsibility. Another issue that was encountered that prevented automation of a step was the need for engineer experience in identifying interactions. This is shown in the filtering of assembly configurations (Step 5). Determining which assembly configurations can be neglected is a highly specific process that is dependent on the system in question, as well as many other factors. As such, it would be difficult to automate the identification of which configurations could be eliminated without the need for a large amount of additional data. Once the requirements for each step in the process were identified, the software for each step was created on an individual basis and then the steps were linked together. The following section will discuss the implementation of the support tool. 6. SUPPORT TOOL IMPLEMENTATION The tool consists of a series of spreadsheets that are able to be edited by the user. Once pertinent data for a given step has been entered, an associated macro may be run to facilitate the completion of that step in the process. The steps follow those outlined in the 7- Step VV&T planning method (shown in Figure 3) and follow the VV&T plan development for an historic case study of a change to the brake drum in an automotive braking system Step 1 Requirements identification The first step in the VV&T planning method is the identification of requirements up to one system level above the component being changed. While future revisions of this tool may include the ability to automatically retrieve this information from an outside database, the current version requires manual entry and functions primarily as a documentation aid. An example of the data table is shown in Figure 4. It should be noted that sections highlighted in yellow are those intended for data entry. This step also stores the requirements for future use in the process and assigns primary keys for later identification and retrieval. Figure 4 Requirements identification table 6.2. Step 2 System analysis The next step in the process is system analysis. The purpose of this step is to identify any components that may be affected by the component being changed. This interaction can include geometric, variant, and organization propagation pathways. In this step, it is first necessary to manually enter the system design structure matrix (DSM) for the system containing the change component. It may also be beneficial to include external components that interact with the system being evaluated. A section of an example DSM is shown in Figure 5. In the example, the intersections with 1 represent interactions between the specified components. Any cells containing 2 or higher represent higher order interactions and will be discussed below. Figure 5 Section of an example DSM The spreadsheet also requires the entry of the number of components, the change component, and the desired order of interaction. The desired order of interaction is required because research has shown that higher order interactions are often useful in identifying/predicting change propagation [27]. Once all relevant data has been entered, the associated macro is run, populating the rest of the DSM with any higher order interactions (the second order of interaction was desired in the example in Figure 5) and a list of all of the components affected by the change component is created. Based on the DSM from the example in Figure 5, the following list was created, as shown in Figure 6. The identified ENGINEERING CHANGE MANAGEMENT - VV&T PLANNING TOOL DEVELOPMENT 593

8 components will then be brought forward into the next step for future use. evaluated further and assigns each a primary key identifier. The results from this are shown in Figure 8. Figure 6 List of affect components 6.3. Step 3 Assembly configuration identification The third step in the process is the identification of possible assembly configurations. Each component affected by the engineering change can have multiple variants and multiple suppliers. Therefore, when considering a VV&T plan, it is necessary to determine all of the combinations that may need to be tested. To support this, the tool first provides the components being evaluated, and allows for data entry regarding the various suppliers and variants possible for each component. Once the data is entered, the associated macro is run and each element-supplier-variant (E-S-V) combination is given a unique identifier and the total number of E-S- Vs for each component is identified. This is shown in Figure 7. Figure 8 Possible combination vectors These combinations will then be further evaluated in the following step Step 4 Assembly configuration filtering The next step involves the filtering of assembly configurations identified in Step 3. Conducting a full analysis for each assembly combination can be timeconsuming and costly. Therefore it is beneficial to identify any combinations that may be ignored. One reason a particular subset of configurations could be ignored is that one variant might perform better in all requirements than the alternate variant. As such, it is reasonable to assume that combinations featuring the first variant would perform better than combinations featuring the second variant. Therefore, the better performing combinations may be ignored in future analysis. Because the analysis required to identify which combinations may be neglected is highly specific to the system requirements, this aspect of the VV&T planning method remains manual. In this instance, the support tool provides an interface for documenting the decisions made and the reasoning behind the decisions. The interface provided to the user is shown in Figure 9. Figure 7 E-S-V combination identification The tool also allows the user to remove any combinations from being evaluated and provides an area for comments regarding the reasoning behind the removal. For the example in Figure 7, E-S-V 3.S5.V6 is not used because that specific variant of the tire and wheel trim is not used in the platform being tested. The support tool also provides a list of all of the combination vectors (possible combinations of different E-S-V combinations) that should be Figure 9 Filtering of assembly combinations interface As shown in the example in Figure 9, two of the combinations were neglected. As a result, the associated cells were highlighted and marked through for ease of visualization. The remaining combinations are stored for use in future analysis. 594 Keith Phelan, Summers, Guarneri

9 Figure 10 Example DVP matrix 6.5. Step 5 Develop Design Validation Plan matrix The next step in the VV&T planning method is to construct the Design Validation Plan (DVP) matrix. The DVP matrix consists of all of the administrative data as well as all of the requirements, associated tests, and any additional information regarding how the testing will be executed. An example DVP matrix is shown in Figure 10. At present, much of the administrative data must be entered manually, while the majority of the testing data is retrieved from other sources. In order to assist in the creation of the DVP matrix, additional data must be entered elsewhere. The support tool uses a test database to store all information regarding the tests that must be executed to evaluate the system. Additionally, two separate matrices are required to aid the creation of the DVP matrix. The matrices identify the relationships between the requirements and the system components and between the requirements and the tests. Examples of these matrices are shown in Figures 11 and 12. Figure 12 Requirements to components relationship matrix In an attempt to minimize the amount of data entered by the user, all of the components, tests and requirements are automatically retrieved from elsewhere in the support tool. Only the relationship data is required to be manually entered during this stage of the process. Figure 11 Requirements to tests relationship matrix 6.6. Step 6 Develop test strategy The sixth step in the VV&T planning method is the development of a baseline test strategy to evaluate the requirements. The purpose of this step is to identify the acceptance criteria for any tests that must be conducted, oftentimes based on the performance values for the existing design. Once again, this step ENGINEERING CHANGE MANAGEMENT - VV&T PLANNING TOOL DEVELOPMENT 595

10 is highly specific to the system being evaluated and requires the data to be entered manually. The support tool assists in this step by providing a table consisting of all of the tests identified in the DVP matrix with cells for each of the information requirements. An example of the interface is shown in Figure 13. Figure 13 Example baseline test strategy 6.7. Step 7 Trade-off analysis The final step in the process is to conduct a trade-off analysis for the tests and requirements to be conducted based on the DVP matrix. It is not always feasible to conduct every test or test every requirement due to cost or lead time restrictions. Therefore, it is essential to prioritize which tests to conduct given certain parameters. The VV&T planning method used in the development of this tool focuses on the requirements to be tested, as opposed to the tests available to be run. In order to aid in accomplishing this, the method uses the Verification complexity index (VCI) to determine the complexity of verifying an individual requirement [1]. The VCI is calculated using the following equation: (1) In order to facilitate this, the support tool provides the requirements and tests from the DVP matrix. The user is required to enter the severity of each requirement, the cost and lead times for each test and the number of tests to verify each requirement as described in the VV&T method. Once the associated macro is run, the tool calculates the VCI for each requirement and ranks them according to precedence. An example of the trade-off analysis matrix is shown in Figure 14. In the example shown in Figure 14, Requirement 1 (R1) has the highest VCI, which implies that the change engineers should focus on that requirement for further exploration of opportunities for trade-off and prioritization. The tool also allows the user to visualize how specific tests can possible verify multiple requirements. Figure 14 Example of a trade-off analysis matrix 7. EVALUATION OF TOOL In order to evaluate the tool, the authors used a previous case study that had been used during the development of the VV&T process in [1]. When manually conducting the planning method for the described example, the user would have to enter and keep track of 193 data points. With the implementation of the tool, the user was required to manually input data in 129 locations. Therefore there was a 33% reduction in the number of opportunities for human error. It should be noted that this is a fairly simple system and the results would increase as the system becomes more complex. Additionally, the support tool conducts all calculations and any analysis required for evaluating change propagation beyond just the first order of interaction. Without any additional input required from the user, the support tool provided documentation to show how the prescribed VV&T planning method was implemented. The documentation includes the requirements list, a list of the affected components, the DVP matrix, the baseline test strategy, and the trade-off analysis. Additionally, the support tool provides space to specify why individual decisions were made regarding the design of the VV&T plan. 8. CONCLUSIONS AND FUTURE WORK The VV&T planning method this research is based on [1] has been shown to effectively mitigate change propagation resulting from an in-production engineering change. However, the large amount of data entry involved and the planning method s reliance on an engineer s experience can hinder the application of the method in complex engineering systems. 596 Keith Phelan, Summers, Guarneri

11 The computational support tool described in this paper successfully addresses these issues. The support tool was shown to correctly guide an engineer through the implementation of the VV&T planning method for a historic example of an engineering change in an automotive brake assembly. The support tool also minimized the opportunities for human error by carrying data over between the process steps and provided documentation of all aspects of the planning methods implementations without any additional input from the change engineer. Additional research needs to be conducted in order to improve the trade-off analysis functionality of the computational support tool. Currently, the trade-off analysis is conducted based solely on the Verification Complexity Index (VCI), which focuses on the importance of verifying individual requirements, combined with the costs and lead times for the associated tests. However, depending on the scope and characteristics of the engineering change being made, different companies will have different goals in executing the trade-off analysis. For instance, in certain situations, a requirement associated with an engineering change may have legal ramifications and needs to be implemented immediately. As a result, the company would likely focus on tests that focus on the legal requirement and can be conducted with minimal lead time. Therefore additional trade-off metrics need to be determined that can allow companies to determine which criteria are most important and guide the testing plan in that direction. Another area of future research is to consider the level of change propagation to consider when managing the effects of engineering change. As previously discussed, the support tool allows for the consideration of change propagation beyond the first order. However, it is not clear to what level the change propagation should be considered. Further research into the field of change propagation and system interconnectivity should be conducted to clarify these issues. ACKNOWLEDGMENTS The authors would like to thank the Army Research Center for funding support for the research conducted in this paper. REFERENCES [1] Shankar P., Summers J., and Phelan K., 2014, A verification and validation planning method to address propagation effects in engineering design, International Symposium on Tools and Methods of Competitive Engineering, Istanbul, Turkey. [2] Wright I. C., 1997, A review of research into engineering change management: implications for product design, Des. Stud., 18(1), pp [3] Huang G. Q., Yee W. Y., and Mak K. L., 2003, Current practice of engineering change management in Hong Kong manufacturing industries, J. Mater. Process. Technol., 139(1-3), pp [4] Terwiesch C., and Loch C., 1999, Managing the process of engineering change orders: the case of the climate control system in automobile development, J. Prod. Innov. Manag., 6782(98). [5] Eckert C., Pulm U., and Jarratt T., 2003, Mass customisation, change and inspiration changing designs to meet new needs, International Conference on Engineering Design, pp [6] Jarratt T., Eckert C., Caldwell N., and Clarkson P., 2011, Engineering change: an overview and perspective on the literature, Res. Eng. Des., 22(2), pp [7] Jarratt T., Eckert C., and Clarkson P., 2004, Engineering Change, Design Process Improvement, Springer, New York, NY. [8] Fricke E., Gebhard B., Negele H., and Igenbergs E., 2000, Coping with changes: Causes, findings, and strategies, Syst. Eng., 3(4), pp [9] Eckert C., Clarkson P. J., and Zanker W., 2004, Change and customisation in complex engineering domains, Res. Eng. Des., 15(1), pp [10] Chua D. K. H., and Hossain M. a., 2012, Predicting Change Propagation and Impact on Design Schedule Due to External Changes, IEEE Trans. Eng. Manag., 59(3), pp [11] Ahmed S., and Kanike Y., 2007, Engineering change during a product s lifecycle, International Conference on Engineering Design, Paris, France, pp [12] Shankar P., Morkos B., and Summers J. D., 2012, Reasons for change propagation: a case study in an automotive OEM, Res. Eng. Des., 23(4), pp [13] Rouibah K., and Caskey K. R., 2003, Change management in concurrent engineering from a parameter perspective, Comput. Ind., 50(1), pp ENGINEERING CHANGE MANAGEMENT - VV&T PLANNING TOOL DEVELOPMENT 597

12 [14] Tavcar J., and Duhovnik J., 2006, Engineering Change Management in Distributed Environment with PDM/PLM Support, Manuf. Futur. Concepts- Technologies-Visions, (July), pp [15] Clarkson P., Simons C., and Eckert C., 2004, Predicting change propagation in complex design, J. Mech. Des., pp [16] Kidd M., and Thompson G., 2000, Engineering design change management, Integr. Manuf. Syst., (2000). assess change risks, Internation Conference on Engineering Design, Paris, France, pp [27] Morkos B., Shankar P., and Summers J., 2012, Predicting requirement change propagation, using higher order design structure matrices: an industry case study, J. Eng. Des., (November 2012), pp [28] Cohen T., Navathe S. B., and Fulton R. E., 2000, C- FAR, change favorable representation, Comput. Des., 32(5-6), pp [17] Huang G. Q., and Mak K. L., 1998, Computer aids for engineering change control, J. Mater. Process. Technol., 76(1), pp [18] Huang G. Q., Yee W. Y., and Mak K. L., 2001, Development of a web-based system for engineering change management, Robot. Comput. Integr. Manuf., 17(3), pp [19] Reddi K. R., and Moon Y. B., 2011, System dynamics modeling of engineering change management in a collaborative environment, Int. J. Adv. Manuf. Technol., 55(9-12), pp [20] Reddi K. R., and Moon Y. B., 2011, A framework for engineering change management in enterprise resource planning using service-oriented architecture, Int. J. Bus. Inf. Syst., 8(1), pp [21] Pikosz P., and Malmqvist J., 1998, A comparative study of engineering change management in three Swedish engineering companies, ASME Design Engineering Technical Conference, Atlanta, GA. [22] Ollinger G., and Stahovich T., 2001, Redesignit-a constraint-based tool for managing design changes, ASME Design Engineering Technical Conference, pp [23] Laurenti R., and Rozenfeld H., 2009, An Improved Method of Failure Mode Analysis for Design Changes, 19th CIRP Design Conference, pp [24] Giffin M., de Weck O., Bounova G., Keller R., Eckert C., and Clarkson P. J., 2009, Change Propagation Analysis in Complex Technical Systems, J. Mech. Des., 131(8), p [25] Giffin M., 2007, Change propagation in large technical systems, Massachusetts Institute of Technology. [26] Keller R., Alink T., and Pfeifer C., 2007, Product models in design: a combined use of two models to 598 Keith Phelan, Summers, Guarneri