Risk management in scientific research:

Risk management in scientific research: a proposal guided in Project Management Book of Knowledge and Failure Mode and Effects Analysis Pollyana Notargiacomo Mustaro Graduate Program in Electrical Engineering / School of Computing and Informatics Mackenzie Presbyterian University São Paulo, Brazil pollyana.mustaro@mackenzie.br Rogério Rossi Graduate Program in Electrical Engineering Mackenzie Presbyterian University São Paulo, Brazil rossirogerio@hotmail.com Abstract The achievement of different university degrees (from graduate to the specialization, master's and doctoral) usually is related to the development and presentation of research results conducted by an Academic Advisor. However, their finalization or the results may be adversely affected by the lack of identification of risk that may present themselves during the process of construction and development of the research. To minimize these impacts, the present study was based on the alignment and adjustment of processes present in the Project Management Body of Knowledge (PMBoK), a specific guide that consolidates the best practices of project management from the Project Management Institute (PMI), and the use of Failure Mode and Effects Analysis (FMEA). From such elements and the identification of factors relevant to the educational institution involved and the processes related to the advisory and development of scientific research, it was sought to build a proposed risk analysis in the academic, named Academic Project Risk Management Plan (APRMP). This aims to contribute to the discussion and gathering information that can support the Advisor and the student at all stages of the research (planning, development and conclusion) that downside risks are avoided or minimized and that opportunities become clear and effective possibilities. Keywords risk management; scientific research; PMBoK; advisoring. I. INTRODUCTION The development of scientific research constitutes a challenge for the students of higher education, graduate school, etc. especially because it has to be finished on time, with the depth degree required or even with the quality expected by the advisor. These issues affect not only the projects in academic area as projects in general. That is why it has been studied and mapped over the years by the Project Management Institute (PMI) [8], who also built project management guidelines (consolidating a guide named Project Management Body of Knowledge PMBoK) from the best practices identified [7]. Even one of the factors influencing this scenario involves failures in risk management. In the case of scientific research carried out for obtaining the degree, it can be point out that the risk management allows the reduction of problems throughout the investigation, and identification of factors that imply changes (whether in scope, timeline, resources, etc.), which results in increased chance of completing the search in accordance with the proposal. The conduction of risk management is related to the mapping of factors involved in Educational Institution, especially in processes relevant to the scientific research development (under supervision) as well as the analysis of the Research Advisory Breakdown Structure (RBAS) [5] and the Academic Research Project Plan (ARPP) [5]. From this context and the use of FMEA (Failure Mode and Effects Analysis) [3]-[10] is possible, through analysis and planning meetings, set up a risk management plan in academic scientific research (RMPASR). To deepen these introductory elements of risk management in academic research projects, this paper is organized as follows: section 2 describes the steps involved on management of scientific research projects and its overall structure; section 3 addresses from setting up risk as this is characterized in the academic field, and strategies to reduce, prevent or eliminate risks in this context; section 4 deals with the use of FMEA as a tool associated with risk management in academic scientific investigations; section 5 presents the proposal of a risk management plan for academic research; and section 6 provides the conclusions of the study as well as further work arising from this. II. STAGES FOR ACADEMIC MANAGEMENT OF SCIENTIFIC RESEARCH Considering the institutional needs, as well as the people involved in the management of academic research (ie, the Advisor and the Course Coordinator, regardless of the degree involved), it becomes necessary to introduce a way of managing such process. Therefore, it can make an adaptation of processes under PMBoK [7] for the mapping and development of academic investigations. In this sense, the Academic Management of Scientific Research Projects covers a variety of processes that can be adapted or inspired by similar elements present in PMBoK [7]. Among these are highlighted the development of a Project Charter focused on Research Advisory Breakdown Structure 978-1-4673-5261-1/13/$31.00 2013 IEEE

(RBAS) [1], the own establishment of a RBAS, as well as a pertinent Dictionary [5]. A Project Charter [5] formalizes the beginning of scientific research project under the supervision of an advisor (and a cosupervisor, if applicable), gathering information relevant to its early stages, in addition to being a tool for decision during the research. This deliverable includes, therefore, the project scope for academic research, the expected results (both, from the institution or the advisor), the identification of macro tasks of scientific research (for periodic control) and the deadline for finish investigation. Such information, upon approval, is broken down subsidizing the development of a Research Advisory Breakdown Structure (RBAS) [5]. This, through decomposition and hierarchy of elements present in the Project Charter on "work packages" [7], aims not only graphic visualization of parts of the research that can be delivered over its implementation, as well as configuration of its life cycle. From RBAS the Dictionary can be built [PMI, 2008]. This aims to map the integral elements of RBAS and have acceptance criteria. Thus, each work unit is identified as the expected result and the degree of acceptability [5]. In fact it is possible to compare the result of the Dictionary prepared with a kind of educational rubric, i.e. a structure consisting of specific criteria (scored and described) for review [2]. Considering each of the characteristics inherent in the balance is the decomposition of a task into smaller parts, and each applicability of this strategy is a written communication (organization / textual expression) [11]. It can be inferred that such a device is aligned due to its architecture, to the pertinent Dictionary described above. It should be highlighted also that the rubrics allow student/students in process-oriented view of the evaluation process dimensions [2], as well as contribute to the development planning of quality researches [1]. Finally, it should be noted that this proposed Dictionary is, by analogy, a kind of holistic rubric. This is because it has a single score (lower levels) for performance evaluation of student/students. The set of deliverables detailed above, therefore, provides grants that contribute to the management plan developed for an academic research project. The latter presents, through the use of scientific language, the theoretical framework adopted, the expected results and schedule (macro activities, as well as dependencies). In order to complement this deliverable, it can also provide an attachment with the quality level required by the institution, risks and opportunities, forms of communication between people involved (students, advisors, institution; synchronous, asynchronous; face, online, hybrid, etc.) and strategies for monitoring and control a scientific research project (which provides for the preparation of meeting minutes and development of research reports). Also is possible to emphasize that the Academic Research Management Plan, as well as other types of project deliverables, run in knowledge areas provided by the PMBoK [7]: Integration Management, Scope Management (both previously worked), Time Management, Quality Management, Communication Management, Cost Management, Procurement Management, Human Resources Management and Risk Management theme explored in the next section. III. THE CONCEPT OF RISK TO ITS CHARACTERIZATION IN THE ACADEMIC RESEARCH AREA According to the PMI [7], risk can be characterized as an event or uncertainty (due to one or more causes) that, if happens, resulting in positive or negative impact in the project. The origin or causal element of a specific risk may stem from a circumstantial condition or requirement of a premise of the project itself or even a restriction. From the academic point of view, it can be considered the following examples: Circumstantial condition lack of specific equipment (that can be purchased, borrowed, etc. in another time); Requirement/Premise fluency in a particular programming language present in the student academic curriculum; Restriction time required for the research, number of members in a group of researchers, etc. The existence of risk does not necessarily imply the change or abandonment of the project. It is necessary to analyze them in relation to the degree of its acceptability and tolerance and mapping its implications, whether these are negative or opportunities, that may be obtained from the risk probabilities calculation. In this sense, a project requires a proactive approach to risk management; otherwise the severity of the event may result in project failure [8]-[6]. Therefore, managing risks in a scientific investigation to achieve a particular degree, presents a series of advantages, among which the following: Construction of a reliable framework for design research and decision-making throughout their realization; Visualization of the threats/opportunities relevant to research (which allows to establish mechanisms for the reduction or mitigation of threats and exploiting opportunities, including taking into account the terms and requirements associated with research publication, if applicable); Analysis of variables that integrate the risks directly related to the issue of scientific publication selection (time, scope, research results (partial/complete), scientific publication journal or event impact factor, and other related topics involved in decision making); Reduce the number of contingencies. From the PMBoK viewpoint [7], the risk involved in the academic research can be adapted as described below: Management (setting as risk management will be performed in the development of scientific research under the supervision of an advisor);

Identification (lifting of risk to research and documentation of their specifics); Analysis (qualitative classification/prioritization of risks through assessment of occurrence versus impact in the context of the project it can be classified as low, medium or high; quantitative measuring the probability of occurrence and consequences of the estimate for the project, which depends substantially on the experience of the advisor provision by this, and a risks knowledge base relevant to the field of research area); Planning (establishing procedures to analyze the opportunities and minimize possible risks to the research, i.e., in a practical way to structure alternative paths that achieve the objectives set for the research planned in the time provided and with the resources available; for this the advisor and the student/students should hold meetings with regularity). Monitoring (involves controlling residual risks, as well as identifying other risks not mapped to this stage and, simultaneously, the institution of plans which enable the reduction of the risks impact and evaluation of the effect of risk management throughout the academic research project). Regarding the analysis process, it is also important to highlight the historic [6] due to previous advising procedures. This can contribute positively to the performance of the advisor as a leader and mentor of scientific research as well as for monitoring throughout the process of conducting research. Also in relation to this process, the quantitative characterization of the risks can be drawn from the use of tools such as SWOT analysis (Strengths, Weaknesses, Opportunities, Threats), Decision Tree, Brainstorming [4], Failure Mode and Effects Analysis (FMEA) [3], among others. Another detail to consider, with regard to planning, involves drawing up a contingency plan [6]. Such proposal must also consider effects or undesired side effects due to the implementation of these measures. Considering that the preparation of contingency plan depends on the preceding steps, especially a correct analysis, that consider the FMEA framework as a complementary strategy for quantitative analysis in the academic development of scientific research. IV. FAILURE MODE AND EFFECTS ANALYSIS IN THE CONTEXT OF ACADEMIC DEVELOPMENT OF SCIENTIFIC RESEARCH The framework named Failure Mode and Effects Analysis (FMEA) was developed by the U.S. Army in order to eliminate problems/defects/errors in equipment that could not suffer intervention after the launch into space [3]. The FMEA framework aims to prevent failures are up in products, services, processes or systems purchased by the consumer. Thus, the FMEA is a method of examining all aspects in which failures can occur [10]. In the academic context, it is essential not only to identify potential risks depending on the type of research conducted, but also the analysis of their impact and the probability of occurrence in order to establish prevention strategies and monitoring. Therefore, it is also necessary to work with the recognition of factors that may be the generators of such risks, analyzing their dynamics over time to define mechanisms of prevention, mitigation and transfer. The application of this framework begins with the following steps: "failure mode" (problem identified), "effect" (consequences) and "cause" (source/sources of failure). Although there is no single linearity and correlation between failures and causes, what makes a list an essential element for the construction of FMEA. From this, it is possible to classify the impact, the ability to recognize the failure (including his way of detection) and frequency of occurrence, and its product generates the Risk Priority Number (Risk Priority Number) [10]. However, it should be noted that although the contribution they can bring to the FMEA analysis and management problems in scientific research, it is necessary to choose a complement dynamic to complete the process of risk analysis. It is necessary because the FMEA turns to threats, i.e. to the negative aspects of the risk. However, its use associated with the PMBoK [PMI] can contribute in different ways, each in the risk rating (which includes strategies to Prevent, Mitigate and Transfer). The concatenation of the elements presented in the preceding sections enabled the construction of proposal for a Risk Management Plan tailored to the specific scenario of scientific research in order to obtain an academic degree (under the supervision of an advisor). V. PROPOSAL OF A RISK MANAGEMENT PLAN IN ACADEMIC SCIENTIFIC RESEARCH As already noted before, the development of the Risk Management Plan involves combining elements of FMEA [3] and PMBoK [7]. In this sense, the Risk Management Plan must provide information that identifies the research (theme and temporary title), the author/authors, the higher education institution where the research will be performed, course and degree of the student/students, advisor (and even co-advisor if is necessary), text versioning, and responsible for approving the plan. Such documentation also requires the enumeration of processes relevant to risk management in scientific research, as well as the construct of a Risk Breakdown Structure for Academic Research (RBSAR), as exemplified below:

Fig. 2. Integration of deliverables of an Academic Research Project Fig. 1. Example of RBSAR By the same token, the documentation also requires presentation of the dimensions of risk (qualitative, which involves the probability of occurrence and impact in scientific research, and the quantitative constructed based on FMEA framework). From this, is possible to establish a plan to response to the risks identified (Table 1). This should match (in order to obtain a numeric parameter) the probability of occurrence, impact, severity (product of the probability impact for probability), the capacity of detection and identification of risk category for academic research. The product result in a risk priority number, with its dynamics must be monitored over time to allow the taking of actions required for the maintenance of academic research. TABLE I. PLANNING RESPONSES TO RISKS INDENTIFIED Risk <Identification of the Risk to Academic Research> P R S D C NRP Description of risk response T a. Label: P Probability of Occurrence; R Research Impact; S Strictness (Impact versus Probability); D Detection (Ability to identify the risk); C Risk Category for Academic Research; NRP Number of Risk Priority; T Dynamic over time. It is also worth noting that risk management should be integrated with other deliverables relevant to the scientific research development in order to establish an overview of this. This scenario is represented in Figure 2. VI. CONCLUSIONS AND FURTHER WORKS From the analysis of elements in the area of project management and, more specifically, the PMBoK [PMI, 2008], and his combination with FMEA [3]-[10] it was possible to achieve an alignment of risk management according the specialties of scientific research development under the supervision of an advisor. This resulted in the mapping of elements that need to be included in the Risk Management Plan of Scientific Researche. It is believed that such subsidies (either in full or in part) may contribute to the reduction of uncertainties and the development and conclusion of scientific research with quality, on time schedule, and meet the established scope. Regarding further works, it is still necessary to analyze the degree of impact exerted by the elements described in view of the proposed calibration and setting weights for these indicators that compose the proposed risk management of scientific research project. Future studies also involve the construction of an expert system that can be used by the advisor, during the orientation process, to monitor their advisees, as well as for identifying and management of risks that may impair or constitute opportunities throughout development an academic research. ACKNOWLEDGMENT This work has been financially sponsored by MACKPESQUISA.

REFERENCES [1] Andrade, H. G. e Saddler, B. The writing rubric, Education Leadership, v. 62, n. 2, p. 48-52, 2004. [2] Arter, J. e McTighe, J. Scoring Rubrics in the Classroom: Using performance criteria for assessing and improving student performance, California, Corwin Press, 2001. [3] Dailey, K. W. The FMEA Pocket Handbook, USA, DW Publishing Co., 2004. [4] Daychoum, M. 40 ferramentas e Técnicas de Gerenciamento. Brasport, 2007. [5] Mustaro, P. N. e Rossi, R. Estrutura Analítica de Orientação Acadêmica: proposta para a formatação de diretrizes para o acompanhamento discente baseada em elementos de gestão de projetos [Research Advisory Breakdown Structure: Proposal for formatting guidelines for monitoring student based on elements of project management], In: Proceedings of the XII International Conference on Engineering and Technology Education (INTERTECH'2012). Santos: COPEC - Science and Education Research Council, 2012. v. 1. [6] ABNT (Associação Brasileira de normas Técnicas). NBR ISO 10006. Sistemas de gestão da qualidade Diretrizes para a gestão da qualidade em empreendimentos, RJ, 2006. [7] PMI. A guide to the project management body of knowledge (PMBOK ), 4. ed., PMI Standard Ansi, 2008. [8] PMSURVEY.ORG. Estudo de Benchmarking m Gerenciamento de Projetos Brasil 2010. Project Management Institute Brazilian Chapters, http://www.mp.go.gov.br/portalweb/ hp/33/docs/benchmarking_gp_2010_geral.pdf. Accessed: 10 jul. 2011. [9] Sotille, M. A., Menezes, L. C. M., Xavier, L. F. S. e Pereira, M. L. S. Gerenciamento do escopo em projetos. 2. ed. Rio de Janeiro: Editora FGV, 2010. [10] Stamatis, D. H. Failure Mode and Effect Analysis: FMEA from Theory to Execution. ASQ Quality Press, 2003 [11] Stevens, D. e Levin, A. J. Introduction to Rubrics: an assessment tool to save grading time, convey effective feedback, and promote student learning, Virginia, Stylus Publishing, 2005.