Functional Validation of SAP Implementation

Functional Validation of SAP Implementation Efficiently produce and maintain a SAP test repository thru modeling of business processes and business rules Geoffrey Potoczny/Smartesting Professional Services Director Smartesting for SAP solution Manager Paris, France Mohamed Bedouani/ AtosOrigin AM & TAM Offering Business Manage Paris, France Abstract-- Enterprise resource planning (ERP) in general and SAP in particular, are now cornerstones of major IT systems. As with all Entreprise IT Applications, the validation phase is a critical step to control the functional risk and ensure user satisfaction. But in the case of package applications, validation activities presents a particular nature related to the specific phases of an ERP implementation: The project phases covering the software configuration and its integration with other components of the information system require both basic business process tests and end-to-end tests to validate the application while mitigating the functional risk; The ERP upgrade type of project or ERP functional evolution projects require functional regression tests while always taking into account the level of risk associated with the concerned business processes. In this paper, we show how the business process modeling can enable the generation and maintenance of the test repository using automatic test generation and testing techniques for repository management and test automation. The goal is to implement an industrialized functional testing process (see reference [2]) dedicated to ERP. It aims at reducing costs and controlling risks in the context of validation of large ERP projects such as SAP or PeopleSoft implementations. In this paper, we are showcasing this approach on SAP, which technical and functional complexity and business criticality require the implementation of a rigorous validation process for the phases of deployment, evolution and migration of the SAP ERP. This approach takes into account the re-use of generic models of business processes that are then customized to the specific functional requirements of the project. It takes also into account the traceability of these models with the repository of requirements (Blueprint Documentation SAP) and with the elements of architecture SolMan to capitalize on the functional validation effort. This approach links together the development phases (specific configuration development, integration) with the maintenance and migration phases thru a continuous monitoring of the test repository, also useful in the regression test phases. Index Terms Accelerator, automation, business process, ERP, functional testing, model, regression, SAP, Test generation, traceability E INTRODUCTION nterprise resource planning (ERP) in general and SAP in particular, are now cornerstones of major information systems. Chosen for their rich functionalities, they have indisputable qualities of robustness, reliability and agility, and enable rapid "black box"-type implementation within the targeted business areas, without having to invest in their white-box development. As with all Enterprise IT Applications, the validation phase is a critical step to control the functional risk and ensure user satisfaction. But in the case of package applications, validation activities present a particular nature related to the specific phases of an ERP implementation: The project phases covering the software configuration and its integration with other components of the information system require both basic business process tests and end-toend tests to validate the application while mitigating the functional risk; The ERP upgrade type of project or ERP functional evolution projects require functional regression tests while always taking into account the level of risk associated with the concerned business processes. These functional testing activities are critical to secure the ERP roll out and the management of functional evolution. They are also quite complex to master in terms of coverage of business processes, and represent an important part of the ERP implementation and maintenance effort (see reference [1]). In this paper, we show how the business process modeling can enable the generation and maintenance of the test repository using automatic test generation and testing techniques for repository management and test automation. The goal is to implement an industrialized functional testing process (see reference [2]) dedicated to ERP. It aims at reducing costs and controlling risks in the context of validation of large ERP projects such as SAP or PeopleSoft implementations. ISQT Copyright 2011 18

In this paper, we are showcasing this approach on SAP, which technical and functional complexity and business criticality require the implementation of a rigorous validation process for the phases of deployment, evolution and migration of the SAP ERP. This approach takes into account the re-use of generic models of business processes that are then customized to the specific functional requirements of the project. It takes also into account the traceability of these models with the repository of requirements (Blueprint Documentation SAP) and with the elements of architecture SolMan to capitalize on the functional validation effort. This approach links together the development phases (specific configuration development, integration) with the maintenance and migration phases thru a continuous monitoring of the test repository, also useful in the regression test phases. This paper first details the challenges of SAP testing in a overall Sap project, then it presents a comprehensive approach oif production and maintenance of the test repository (business processes, end-to-end) based on business process models. The next section illustrates this approach for the test of SAP Sales & Delivery module (SD), to finally provide the results of an operational project. SAP TESTING : WHAT IS AT STAKE? There are 3 main types of SAP projects: - Implementation / SAP integration - Deploying SAP - SAP upgrade The implementation project involves the implementation of SAP ERP in the information system of the company. The deployment project aims at creating a set of processes that will be shared by the entire company (entities in the same country and subsidiaries in different countries). It therefore creates a consistent set of processes that are adapted to each subsidiary or entity to take into account the legal provisions in force in the country. An upgrade project consists in the migration of the installed version of SAP to the latest version proposed by the vendor. A maintenance or patch project (support packages or Enhancement packages) will be organized in the same manner as a upgrade but require less effort on tests. Test coverage is then focused on the impacted transactions and critical business processes. With an implementation/integration project, the tests will ensure the consistency of each process but also their consistency between them. For example, upon receipt of merchandise, we will ensure that the storekeeper acknowledges the delivery, increments the inventory and stores the items at the right place. In addition, the tester must ensure that the various accounting items have been created. TABLE I TESTING COSTS OF AN ERP IMPLEMENTATION PROJECT An SAP implementation project is necessarily strategic for the company since it integrates at the heart of the company s IT and business processes. Budgets generally observed for tests of a first implementation are in the range of 30 to 45% (they can reach 50% in some cases). Any industrialization of this phase is therefore important to have shorter delivery times and high quality during production. This industrialization will then reduce testing costs during maintenance and evolution. Defects detected in production are 7 times more expensive to correct than the same defect detected and corrected during functional testing. Add to this the potential costs arising from this disruptions on the activity of the company and you realize that this cost may be much higher. On a deployment project, in addition to the integration tests, all the adjustments made specifically for each entity or subsidiary must be tested. It is therefore important to capitalize on test assets developed for common business processes or "Core Model" and then customize them when and where required. TABLE II TESTING COSTS OF AN ERP UPGRADE PROJECT. On a version upgrade project, the main issue is around the regression tests. The SAP upgrade is often technical in nature: it installs the new version of the ERP with the least impact on the existing processes. It consists in making quick deliveries based on the new features of the ERP. On this type of project, ISQT Copyright 2011 19

80% of budget is devoted to testing. The upgrade of the production environment must be achieved within a relatively short time frame (24 to 72 hours maximum) which raises very strong constraints on the testing process. The relevance and coverage of the tests prior to production must be excellent to avoid any unpleasant operational surprises. The time available for testing on production usually comes down to a few hours. Having well established test scenarios or test performed automatically is therefore essential to ensure the return to operation mode of the system at the scheduled time. The vast majority of SAP projects have no dedicated team for testing, it is the project's functional experts who design and execute tests. This approach has the advantage of putting the functional expert on the front line. But as the test is not his primary function, the tests are often not comprehensive and usually deal only with nominal test cases. In addition, the developed tests only reflect the expectations of the functional expert but not necessarily what the business expressed during upstream phases of design and detailed specifications. Therefore, to strengthen the validation phase, big organizations users of SAP increasingly outsource their functional tests to specialized service providers. Outsourcing the functional tests to a third party organization with specialized testing teams ensures that the tests are developed in compliance with the needs and requirements expressed by the business during the design phases. Thus, as illustrated in TABLE III, the quality control and productivity of test phases is a major challenge for successful SAP projects. TABLE III WHAT IS AT STAKE that critical business processes are properly implemented and maintained over time and evolution. Modeling of business processes, in notations such as BPMN - Business Process Modeling Notation - will help formalize and structure this knowledge to the production and maintenance of the tests repository. This principle of capitalizing on the business modeling that is implemented in automatic test generation approaches. These approaches, as illustrated in TABLE IV, ensure the alignment of the tests repository with business needs by monitoring the tests production from these models (Process and business rules) and by automating the traceability link between business requirements and the tests. TABLE IV PRINCIPLES OF AUTOMATIC TEST GENERATION FROM MODELS OF THE BUSINESS NEEDS The modeling elements used for test generation (see TABLE V) are on three levels: Business Process Modeling, typically represented in BPMN - Business Process Modeling Notation; Modeling Domains and business entities, typically represented in UML - Unified Modeling Notation, through a class diagram. The modeling of business rules (decision rules to be tested), typically represented by annotations in UML OCL - Object Constraint Language. TABLE V MODELS A SAP TESTING APPROACH BASED ON THE FORMALIZATION OF BUSINESS PROCESSES AND BUSINESS RULES Business processes are at the heart of the activity of SAP testing project: the objective of the testing activity is to ensure ISQT Copyright 2011 20

These modeling elements that represent the requirements and expected behavior of the application under test are used in a test generation process (see TABLE VI) which provides the following functions: - Automated production of test cases covering the processes and business rules. The generated test cases include the description of test steps, parameters and input data and expected results for the establishment of the verdict, with environments such as Smartesting CertifyIt; - Automated traceability from requirements to tests - Integration in a tool chain made in particular of test management tools like HP Quality Center and IBM Rational Quality Manager; - Production of scripts to automate test execution. TABLE VI AUTOMATIC TEST GENERATION PROCESS recommended mode of use of transactions for an efficient process. Consistent with these descriptions, BPMN models were made. In the following pages, these models will be designated as the generic models. The first task of the test analyst is to identify the differences between the standard process and those specified by business experts. Modification of generic models then allows effective communication between the test architect and the business experts while simultaneously ensuring the production of high level test scenarios. His second task is to describe, with UML modeling intended to test purpose, the specific business rules that will be tested, and business objects involved in the verification of these rules. This whole business processes and specific UML model, will calculate a complete set of tests required for qualification of product developments, and publish these tests in the test repository for execution. A SAP TESTING CASE STUDY SD MODULE The SD module offers a set of standard processes. We will change its inter-co part to meet the demands of the business expert in charge of the sales cycle. The initial content of the generic model used is presented in the following manner, prioritized by theme. In the specific context of the SAP testing, this process starts from existing models of generic processes that comply with Base-Line SAP modules, which provide a point of entry for the test project, in line with the elements of the SAP repository Solman (Blueprints, documentation...). This capitalization on a generic model is illustrated in TABLE VII. TABLE IIX GLOBAL VIEW OF THE SD PROCESS TABLE VII RE-USE OF THE GENERIC MODELS OF SAP STANDARD BUSINESS PROCESSES TABLE IX DETAILED VIEW OF THE CROSS COMPANY SALES ORDER PROCESS In the case of SAP, for each module, a set of processes is offered by SAP in documentary form (baseline). This set is the From the generic model, a number of changes are provided in generic processes in BPMN, to match the ordering process of ISQT Copyright 2011 21

the company. These changes deal in our case with the conduct of the process itself, as well as the use of specific transactions. The company processes are much more detailed than the standard processes, we also added some depth levels in the hierarchical model. TABLE XI PARTICIPATING CLASSES TABLE X SPECIFIC PROCESS OF THE COMPANY Reading a process expressed in BPMN is simple: the rectangles with a gear in the upper left are the tasks performed by SAP, which represent steps of the tests. The rectangles fitted with a simple sign "" indicates the presence of a subprocess, allowing a hierarchical vision of the scenario. The arrows represent the sequence of tasks and lozenges containing "x" represent possible alternative scenarios depending on data and test objectives. The simplicity of reading at this level ensures a common understanding between business analyst and the test expert. Very often, an in-depth reading of a textual specification is not sufficient to avoid misinterpretation and will cause considerable loss of time afterwards if test development-test review-test correction cycles between the test expert and business analyst are necessary. Modeling does not protect those same errors of interpretation, but these errors will be identified much earlier in the test production cycle, and will therefore have a much lower cost. In addition, we often find that this modeling is a valuable asset for both the QA and for the business. Following this modeling process, the test analyst will proceed gradually to the inclusion of business rules in its behavioral model, described in UML. This model describes, in the one hand, the business classes that participate in the test, on the other hand, the rules themselves, using data contained in the instances of these classes. The SAP system under test (SUT) will be associated with a set of common objects (PurchaseOrder, SalesOrder, etc. in the figure above) that are linked together and possess some attributes which value will be defined according to the test campaign and to the the data sets entered by the tester when executing. These objects and attribute values will be used to monitor compliance with the business rules. Business rules are also represented in the UML model, using the OCL language, standard language associated with the UML notation. Writing the business rule above allows to express several things: - The conditions in which this rule applies (here, if it is a profit center closed). - The consequences of applying this rule (here, a error message is displayed) - The reference to control that this rule is actually tested, identified in the example with a requirement id (---REQ) that will be published in the test repository for requirement traceability. From these elements, processes, classes and rules, Smartesting s CertifyIt is used to calculate the necessary tests to verify each business rule, using the corresponding data values, and respecting the described process flow. ISQT Copyright 2011 22

TABLE XII AN EXAMPLE OF TEST COMPUTED BY CERTIFYIT Using only elements from the model, the publication will provide a complete test sequence (steps, data, and expected results) for a manual execution and if necessary to run automatically after creation scripts or automation components. The results of executed tests will be published and supported in a conventional manner by the test execution team. CASE STUDY RESULTS OF THE OPERATION PROJECT CertifyIt GUI is used by the test analyst to control the behaviour of his model, verify that all business rules are covered by at least one test and ensure an optimized coverage of the requirements. A business rule can be detected as unreachable, either because the rule was incorrectly coded in the model, or because the requirements are contradictory. In the latter case, the test analyst will pass the information to the domain experts for correction of the specification. In this way, costly mistakes at the development, deployment and operations phases will be avoided early. The test analyst at any time has the ability to publish the computed tests and the traceability links in the test repository. TABLE XIII REQUIREMENTS IN HP QUALITY CENTER AND TRACEABILITY LINKS As an illustration, we present results obtained for an SAP implementation project, by focusing on the SD module. In this case study, SAP was implemented by all the Company s entities around the world. This ever-changing system leads to regular roll out programs, precisely 4 roll out program per year. The main objective is the reduction of time to put these into production (deployment within each entity included), and thus imposes a short time for the functional validation of the system, including the functional regression tests. The order creation flow in the module includes requirements such as the creation of the sale contract, the project creation, the project invoicing, the linkage of the project to the invoices, etc. From these flows and on the basis of a functional analysis, BPMN models were created and imported into the Smartesting tool in which the test cases were generated automatically. After validation of the 100% coverage of the requirements, the results of the automatic generation (test cases, requirements and automation functional keywords) are then published in HP QC. For this example, 83 test cases were generated and successfully executed either manually or automatically after development of automation of keyword (Business Components). TABLE XIV TESTS STEPS IN HP QUALITY CENTER FOR MANUAL EXECUTION This Business Process Modelling-oriented approach makes it easy to share the repository of functional requirements with the project owner (Business analyst or functional consultant), to quickly analyze the impacts on test cases brought about by changes in requirements (end-to-end testing). From an economic standpoint, the return on investment has been observed from the third iteration cycle, as shown in the figure below. ISQT Copyright 2011 23

BIOGRAPHY SUMMARY AND CONCLUSIONS Geoffrey Potoczny manages the consulting team as Professional Services Director of Smartesting. He has a solid experience in software companies, thanks to his former experiences at SAP France, where he spent 8 years, at Ascential Software (where he stayed 3 years) and at LGS (3 years in France and 1 year in Quebec). Working on the consulting services side at Ascential as Project Director and Enterprise Architect and then at SAP as Key Account Service Manager gave Geoffrey a good knowledge and understanding of constraints and processes of his customers The approach described in this document is: Undeniably an Accelerator of SAP implementation and roll out projects Is to be considered for any new implementation of SAP projects or migrated version A great way to urbanize the tests repository and the implementation of traceability between business requirements and test cases for existing SAP systems maintenance Finally meets the main objectives: reducing time and costs, increased quality. REFERENCES [1] "Towards a professional testing service for corporate profitability - Dr. Bedouani, T. Lallemand, P. Cogoluègnes - IT Expert N 72, p. 16, March / April 2008. [2] "Industrializing the functional test - job requirements to the repository of automated testing" - B. Legeard, F. Bouquet, N. Pickaert - DUNOD 2009). ISQT Copyright 2011 24