The Art of EMV Testing

Transcription

1 The Art of EMV Testing Version 1.5. April 2012 Introduction!... 2 The Payment Industry!... 3 The EMV migration!... 4 Testing Basics!... 6 The Issuer! The Chip Card! The Issuer! The Acquirer! The Acquirer! The Terminal! Tests Execution! The Test Matrix! About the Author! KaSYS Canada

2 Introduction The Debit/Credit card has been my primary method of payment for as long as I can remember. Now, these cards are evolving. With technology reaching a turning point, countries worldwide have begun the upgrade from the old mag-stripe method to the newer EMV cards. With the United States on the verge of embracing this new technology, I believe that it is appropriate for me to share some of my personal experience in helping EMV become a reality. Over the course of twenty years and two continents I have been testing different methods for the card payment industry. Beginning in France, the home of the chip card, I migrated to Canada in the late nineties to assist in the successful installation of the EMV cards in that country. In my experience there are three main questions that clients always need to know: Why should I test? What should I test? How should I test? In an effort to answer these questions I have prepared the report below. The tests that we will discuss here are the tests associated with the quality assurance division. Quality assurance tests allow organization to make sure that the advances made by their contractors meet their expectations. This division includes certifications under the Schemes quality assurance program. Every organization will differ slightly from the model within; this is an overview, constructed to meet a broad array of needs. This report is neither for card manufacturers, nor for terminal manufacturers and only partially incorporates ATMs. Specifically, this report has been devised for the North-American trade zone. KaSYS Canada ! Page 2 / 26

3 The Payment Industry EMV represents the four world Schemes, Visa, MasterCard, JCB and Amex. EMV is the default for credit and debit cards based on the chip card technology. Maintained by EMVCo. ( EMV is the default standard that the industry has labeled the entire migration from the old technology. Making the payment card industry work correctly is a system in place called Schemes. Visa, MasterCard, Amex or American Express, Discover, Diners, Plus alongside any other card issuer all use a method involving Schemes. Set up in a worldwide pattern, Schemes are used by all merchants both local and global to accomplish card operations. Schemes are links between Issuer and cardholders and the Acquirer, the entity that actually acquires a payment transaction made between a Terminal and a Card that supports the same Scheme, in store or at an Automatic Teller Machine (ATM). Card Terminal Acquirer's Scheme's Network Issuer's Authorizing In most cases the card and the terminal both are tagged with several different Schemes, so as to maximize the amount of transactions that can go through at any one time. KaSYS Canada ! Page 3 / 26

4 The EMV migration Credit and debit cards have emerged, over the last sixty years, as the primary method behind payments, in all situations. During this period the Credit/Debit card has undergone several different upgrades. The beginning The Credit/Debit card system first appeared in the late forties. Initially very simple, the card itself was plastic with a card number, name and expiration date clearly emblazoned the face. When used for payment, the imprinter method was utilized to create an impression of the card for the business owner to send in; the details were written clearly on the sales slip. The mag-stripe era The late sixties ushered in a wide range of advances in processing revolution as well as the first introduction of a large-scale computer system. One of the major advances was the mag stripe that enables all of the relevant information to be captured easily at the point of sale, and then processed the Acquirer followed by the Issuer host. With this, the old imprint technology began to be gradually phased out. The mag-stripe technology has been a proven success since the seventies, working well throughout the years. With today s technology rich environment, it takes only a minimum of personnel to test and maintain these systems. KaSYS Canada ! Page 4 / 26

5 The EMV migration This revolution from the mag-stripe technology to the chip method is called the EMV migration, a major milestone in the payment card industry. During the process of introducing the chip technology, the older mag stripe methods will still be able to be used, just reduced over time as that technology is gradually phased out. One of the major changes over the mag-stripe system is the tiny computer that replaces the read only devices. With the newer technology, the ability to include onboard apps and other benefits drastically increases. For any basic transaction to occur, the card in question has to contact a terminal. Now, instead of the terminal only reading the data stored on the card, we have two complete computer systems talking via the EMV method. This is a major deviation in the way terminals work, by consequence this also changes the way the Acquirer and the Issuer host works. Even with the massive scope of the change, the details are not that complicated. The real challenge exists in all the slightly different applications across that board that needs to be addressed in the same way consistently. This will most likely cost more to test and work out the bugs, than actual installation. One of the major issues to be addressed after a successful upgrade to the EMV system is the testing or regression testing of the system. Some testing should be done during the installation, but it remains a required duty to keep on any new tests so that your system is ahead of the curve rather than behind it. Quality Assurance and the entire testing process will be the basis of any successful EMV migration process. It will take attention to detail to correctly navigate all the potential hurdles. KaSYS Canada ! Page 5 / 26

6 Testing Basics This chapter provides basic information about testing methods, an overview of the besting techniques, before getting into the real details of the separate methods over the next chapters. Specifications The very first thing that should get done is to make sure that the system specifications are clear, well written and very available. Do global specifications, not incremental. There are two kinds of specifications: incremental specifications type where changes are documented whereas global specification types where the entire system is documented, as well as there being a new document created for every change. Updating to the new EMV will not be accomplished overnight. A successful installation includes many small changes, developing additional EMV Scheme by Scheme, adding contact as well as contact-less, then credit and debt ad infinitum. In the beginning it may look to be best choice to go with incremental documentation, but in the long run, in my experience that is not the case. Global specifications will reduce misunderstanding, cut down on documentation incoherencies as well as make the management of the entire set far simpler. Identify the Changes Even if you decide not to do incremental documentation, it remains a valid point to identify changes in any specifications. Take a moment at the beginning of the document and list the entire value of changes for each version. Update the documentation before, during and after. Start writing documentation well before development begins, and don t stop. Throughout the entire process keep an up to date and relevant journal. Limit the documentation to a manageable set. Try to strike a balance between too much documentation, and not enough. When you realize that it is becoming a burden to keep up with your notes, you know that you have begun to over document the process. Be aware. Test plans The objective when creating every test is Quality Assurance. KaSYS Canada ! Page 6 / 26

7 Write test plans. It is good common sense to make use of an application over and over. Write down your tests, allowing yourself to measure progress and efficacy. Every test case must set the initial conditions, as well as detail the test scenario extensively and list the desired result. Derive test plan from the specifications. How many tests will be enough? More is not necessarily better as the more tests ran, the more expensive the application. A good rule of thumb is one test per function, negative tests when you find them required, and then rely on experience to update further tests. Positive and Negative testing The Positive testing confirms that the system will work correctly with correct input. There must be positive testing. Negative testing is created to understand what occurs if the system or the input is corrupt. Not too much negative testing. Too much negative testing can be very expensive and far more complicated. Regression testing Tests that have to be repeated over and over with each new release of the system are called Regression Tests. Identify regression tests. Regression testing is an important component of the quality assurance. These Regression tests will be performed even after a successful migration project. The ability to enable a large Regression test for every release, results in a far better guarantee. Bear in mind, each Regression test will increase the budget, so the key word is Balance in all things. Trust experience. Try to automate regression testing. At every opportunity take advantage of technology and automate any regression testing. This enables quicker and more reliable results KaSYS Canada ! Page 7 / 26

8 Automate testing means that you employ test tolls to parts of the job. Instead of a tester go through a terminal and physical insert a card, the program creates a terminal simulator to test and validate responses. Much faster, some prior setup is required. Four main points should be considered before automating: The number of tests that will be automated. How many times each test will have to be run, the overall complexity of the process to automate and the cost of maintaining the automated effort. Each of these should be carefully considered before the decision to automate is made. Typically, an only organization that budget over one million dollars/per year or more will experience the full range of benefits that test automation allows. Stress test Stress Tests, or volume testing, are used for authorization or acquiring hosts. The purpose of a stress test is to validate the load that a host can endure and ascertain that it is able to function even during peak payment days like Black Friday, Boxing Day or Mother s Day. Build a stress test environment To perform a stress test you will need to prepare beforehand: - Set up simulators for all inputs and outputs of your host to stress. - Define a data test bed to be used of rate stress test. - Be able to save and restore the data state before the stress test. Complicated endeavors stress tests are expensive to perform, especially the preparation. Strategies to reduce or avoid stress test cost: - Rely of the software/hardware manufacturer to estimate the maximum load and skip stress test. - Use a test host a fraction the size of the production host to stress test., and then extrapolate to the production host. Injector vs. Simulator As opposed to the complex simulator, the injector is based on a simple idea. The theory is the store all the intended data needed under a stress test, and then inject it at the intended speed. Much simpler and easier to accomplish, the obvious drawback is the lack of any dynamic calculations. The tradeoff being a complex simulator can accomplish any calculation or the simpler injector system will supply only limited data. KaSYS Canada ! Page 8 / 26

9 Over the next chapters I will examine the specifics of each of the actors; the issuer, the acquirer etc. Please, skip ahead to relevant chapters. KaSYS Canada ! Page 9 / 26

10 The Issuer Card Terminal Acquirer's Scheme's Network Issuer's Authorizing During the mag-stripe age, the Issuer was often little more than a marketing squad. The continuing challenge for these issuers will be the quest to identify quality contractors and the issuer s ability to monitor their results. Much less standard than the mag-stripe, the EMV card issuing process can be complicated. While not overwhelming, many IT departments are not set up to handle the transfer, this is one of the largest pitfalls. Be Aware. KaSYS Canada ! Page 10 / 26

11 The Chip Card Card Terminal Acquirer's Scheme's Network Issuer's Authorizing The EMV card is a complete computer with its own operating system, applications and data. Outside of the card manufacturers themselves, it is not common to test the cards themselves. The only real validation for the cards will be found in real world application. Set up trials and slow roll out. Already mandatory, a slow roll out will ensure that there is plenty of time to deal with any issues that arise due to the integration between the chip cards and terminals. Even if the card tests are limited, it is still imperative to test the card data, in most cases named the personalization s data. Personalization data is divided into two separate categories: The data common to a series of cards, and the data that changes with every card. Common data sets up the behavior for the card: parameters of each EMV application which will set up debit/credit, cardholder authentication, etc. These parameters make up the card profile. Specific data included is the cardholder name, the Primary Account Number (PAN), the expiration date and all relevant data. Profile Validation The first step is a validation of the card profile. The idea behind this is that every parameter is checked against the rules and regulations from the relevant Scheme. Due to the rising complexity of the validation, most Schemes have instituted a set process to assist in accomplishing this. KaSYS Canada ! Page 11 / 26

12 Use a profile validation tool. If there is a validation tool available, be certain to make use of it. If not, be sure to create your own set up plan for each of the schemes attached to the card. Personalization Test After production mode begins and cards are regularly being produced, there is a secondary validation process that needs to be set in place. In a quality control effort, be sure to check that the manufacturer has used the proper data during the personalization process. Create a validation process for the batch, using this tool to test only a few cards out of every batch. Use a personalization validation tool. Although very useful, be sure to double check that the issuer provided the correct files, and that there has not been a mismatch between different clients. KaSYS Canada ! Page 12 / 26

13 The Issuer Card Terminal Acquirer's Scheme's Network Issuer's Authorizing As the EMV updates, the authorization request is signed by the card and the authorization host sign the response. Scheme Network Certification EMV updates should always be validated. Generally the Scheme will provide you with a reliable certification process; it is always good practice to stay informed. Scheme network simulator. A Scheme network simulator is needed to validate your authorization host before it is able to be certified. Regression Testing It is always good practice to have automated certification tests ready to run after every update. Set up automated regression testing. Stress test A very good way to avoid the expensive Stress test is by contracting a fall back authorization server, or possibly using the authorization host software provider experience to correctly size the host. If you choose otherwise, a scheme network simulator with stress test capabilities will be necessary. Use an injector to stress test. KaSYS Canada ! Page 13 / 26

14 Monitoring It is essential to monitor the rejects and the authorization failures as it will allow the operator to find personalization problems as well as acquirers interactions issues. Set up monitoring. Allows you room to be lax with the test process, also allows space for problems from the acquirers to be resolved. Trials and Roll out To limit risks, it is strongly advised to set up trials and slow roll out. KaSYS Canada ! Page 14 / 26

15 The Acquirer Card Terminal Acquirer's Scheme's Network Issuer's Authorizing Typically it is the Acquirers that will feel the brunt of the EMV migration efforts; they will also be the focus of the most of the test efforts. The Acquirer acts as the entity that guarantees the payment systems against wrong doing. It is their responsibility to guarantee that the card is processed correctly at the terminal as well as smooth the entire transaction process. The Acquirer is the one that gets paid by the merchants to do this right. If issues appear with the personalized cards, or if the Issuers rules are not followed precisely, it is the Acquirer that will deal with the problem and find a solution. Once put in place, it is far easier to adjust the program and the software rather than change the card themselves. KaSYS Canada ! Page 15 / 26

16 The Acquirer Card Terminal Acquirer's Scheme's Network Issuer's Authorizing Acquiring host fulfills two main functions, the terminal configuration function, and the transaction routing function. Not every configuration will be the same, expect differences. Both of these functions will be impacted by the migration to the EMV card. A major difference is that the terminal configuration function changes with the increase of information that must pass through the terminal and then must be updated at least once a year. The routing function consists of routing card transactions from a terminal message format to the relevant Scheme format. The entire way, the EMV transaction will add EMV data in every message format; these message format modifications impact the routing function. Integrated Test Environment There must be a test environment created for the terminal software development, the approval process and the Scheme network certification. Do not create a host simulator for the terminals. Since a test host will be required in any case, it is not recommended to build a host simulator into an integrated test environment. However, while not useful in most cases, in the specific case of negative testing there is a place for host simulators. In almost every case the integrated test environment already exists, it just needs to be updated to match the incoming EMV upgrade. What it is essentialy doing is adding EMV ARQC & ARPC calculation to the transactions and adding the EMV test data to the downloaded configuration data. Provide a test host environment with Scheme network simulators. KaSYS Canada ! Page 16 / 26

17 The best method for adding the EMV response capabilities to the integrated test environment is through the use of simulations of Scheme networks. While some Schemes will provide their own equipment, that should only be used for certification, and you will continue to need simulators on a day to day basis. Identify the test cards to be supported in your test environment. Set the test card Issuer test keys into the Scheme simulators and update the downloaded EMV configuration data with the cards certificates. Note that the Scheme EMV test cards are only a portion of all the test cards to be supported. Terminal Approval Process (See Terminal) Scheme Network Certification Scheme certification will occur in the integrated test environment where the Schemes test cards are supported and the associated terminal can process them. During the certification process the host will be connected to the Scheme test environment and not connected to the network simulator. Scheme End-To-End Certification (See Terminal) Regression Testing It is always prudent practice to provide a method to automatically test your host after each update. The Acquirer host is responsible for the defense of the payment systems. It will be the Acquirer that firsts detects card anomalies or any terminal error. It is critical to maintain the Acquirer host in top rate condition. Use a Terminal Simulator. Use a terminal simulator to automate the regression tests. Used with Scheme network simulators they will together perform sandwich testing of the host. Negative Testing When the need to full proof the host from any and all-erratic terminal behaviors arise, use negative testing. KaSYS Canada ! Page 17 / 26

18 Always look at the possible downsides: an Acquirer is responsible for the transactions on a Scheme network. An underfunded third party commonly develops the terminal software itself; there can be errors at the terminal approval process; the host should not fail upon reception of an unwanted or unformatted transaction request. Use a terminal simulator. The correct and only way to conduct negative testing is by creating a terminal simulator. Stress test A stress test of your acquirer host is a long and complicated undertaking. Do not under estimate the data preparation: 300 TPS (transaction Per Seconds) during 5 hours is equal to 300 x 3600 x 5 = 5,400,000 transactions. No more than 10 transactions per card equal to 540,000 accounts. Trying to match the real Debit/Credit cards brands spreads is tricky. Use special simulators with stress test capabilities There will be a terminal and Scheme network simulator tailored for the specific stress test. The communication between the terminal and the host are MACed, and the MAC key is changed every X transactions. This process forces you not to use an injector, as a more complex simulator are required. Simplify the reality. It is not always necessary to exactly match reality. Use a slightly smaller host to validate a smaller stress. Increase the fraud thresholds to be able to apply the same card more and to limit the number of accounts. Monitoring Especially during production you will need to track and analyze the failed transactions. Set up a monitoring process. The best way to measure service quality is by monitoring the transactions going through the host. The alternative is to simply wait for the users to call in, usually in some quandary, with a payment emergency. KaSYS Canada ! Page 18 / 26

19 The Terminal Card Terminal Acquirer's Scheme's Network Issuer's Authorizing POS, ATM and integrated Merchant solutions all fall under the heading of The Terminal. When migrating to EMV there are going to be several changes in the terminal application. In most cases, the new library or EMV kernel, new dialogs for the merchant and the customer and the underlying fact that you are now using chip cards all combine to push out the old technology. What is L1, L2 and L3? L1, L2 and L3 stands for level 1, level 2 and level 3. This is the normalized structure of the card interface. L1 L2 L3 Level 1 is the hardware layer. Level 2 is the EMV kernel layer. Level 3 is the Scheme specifics. This document does not address the hardware testing issues. Using an already proven and certified EMV kernel save you a lot of time and money. It will also speed up and simplify the tests. Each Scheme will add some specifics within the EMV specifications, and a L3 certification. Hardware Certification (Level 1) As opposed to the other components, there is an hardware certification for the terminals. In most cases the terminal hardware certifications will have been obtained by the manufacturer. KaSYS Canada ! Page 19 / 26

20 This document does not cover the specifics of the many different hardware specifications. EMVCo Kernel Certification (Level 2) The manufacturer should accomplish the EMV Level 2 certification. Contact EMV kernel certification EMVCo has already setup a certification process named EMV Level 2 contact certification. Quite complex with both a large number of tests and the required lab certification, typically there will already be a certified contact EMV kernel available to use. Contactless EMV kernel The industry is in a transition to a contactless EMV kernel. EMVCo has not yet defined an EMV Level 2 contactless certification process. Terminal manufacturers and others should however already offer a contactless EMV kernel. Scheme Card Interface Certification (Level 3) Each supported Scheme will have its own set of tests for the card interface. In many cases an Acquirer test environment will be required to execute these tests even though they focus on the card interface to prepare the terminal, to download the correct configuration data, to properly handle online terminals. Test Card Plastic vs. Test Card Simulator The Scheme Card Interface Certification is done using a set of test cards. These test cards are available as a set of physical test cards or as a card simulator test tool. The card simulator option will speed up the test process, as well as enhance the accountability on the tests execution. Terminal Approval Process There is no standardization in North America for the Terminal/host interface, and therefore there is no certification. It remains the responsibility of the Acquirer to ensure that the terminal operates correctly when connected to the Acquirer s host. As a result of there being no certification, the tests can range from very good to poor depending on the circumstances read the Acquirer. There remains a major risk for the KaSYS Canada ! Page 20 / 26

21 Acquirer of persistent issues at the physical locations if no formal approval standard is adopted. Optimally, the Terminal Approval Process should validate that every function available at the Terminal is performed correctly by the terminal and also integrates easily into the Acquirers host. The larger the number of terminals, or integrated merchants to approve, the more clearly defined the approval process should be. Don t forget the user interface In North America it is generally left to the Acquirer to set up its own standards in regards to Schemes, as long as the guidelines are followed. This means that there is no formal user interface certification, except the acquirer approval process to validate the user interface. Care should be taken to thoroughly test the user interface, as it will be the focus of the client experience. Good interface and it will receive good support. Negative or lots of errors will scare clients away from using the system. Essential for any smooth, large roll out, remember that the term terminal includes the integrated merchants also. Maintain an approval test plan. Create a test plan that will grow and be easy to maintain with changes. A good test plan will include the expertise and past lessons from the terminal approval test. Use a host simulator to partially automate the terminal approval process. The best way to manage a large number of terminals approvals would be to use a host simulator in conjunction with the integrated test environment. The host simulator will be used to fast track most of the positive testing and every bit of the negative testing (see below). Be aware that maintaining such a host simulator will require a fair amount of effort, but the rewards will balance the work out nicely. Scheme End-To-End Certification End-to-End testing is the integrated tests of the terminal and the acquiring host. Every Scheme has some end-to-end certification. While this Scheme does not generally include a lot of tests, usually between 50 and 200, it is designed to protect the Scheme and the Issuer both from Acquirer's errors. KaSYS Canada ! Page 21 / 26

22 This certification can be difficult to set up, as the Acquirers host is required to connect to a Scheme authorization host, with time slot and is often hard to feed back. (See Test Card Plastic vs. Test Card Simulator in Scheme Card Interface Certification). As the scope of the end-to-end certification is so small, the Acquirer should not limit its Terminal Approval Process to just this method, but should add its own tests that validate the other transactions such as reversals and prepaid card activation, are successfully processed. Every Schemes End-To-End certification should be pre-tested in the Acquirer integrated test environment. Regression Testing There will almost always be more to validate at each terminal. Use the Terminal Approval Process as the regression test bed. The full terminal approval process should be used as the regression tests. There are very good reasons to be strict with the regression testing at this point: - Terminal vendors generally run very few tests. - Terminal software can often be bulky and hard to work with. - Terminal set the image of the Acquirer. A good option to help speed up the process is the use of a host simulator. Negative testing It is always possible to add negative testing to the Terminal Approval Process. The primary application for negative testing is to make sure that setup errors are correctly detected. Use a host simulator It is the best idea to use a host simulator for the terminal approval process for any negative testing; this will ease the entire process. Another benefit of the simulator is that it will free the approval process from the integrated test environment. Finally, it aids in the terminal approval process being consistent and faster during execution. As always, maintaining such a system will require a modest amount of continuous effort. KaSYS Canada ! Page 22 / 26

23 Tests Execution Test vs Certification A certification is typically a test plan execution that needs to be validated, or even performed by an independent party. Due to the involvement of an outside party, the costs are generally higher. Taking the cost into consideration, it is recommended that you execute the certification test plan on your own before bringing in the third party, thereby reducing overall cost. It is also a good idea to add in your own tests to assure quality assurance. For most certifications there is a re-certification process. For whatever reason, there will always be a process for re-attaining a certification. Due to the cost of the certification or re-certification, Schemes will likely limit the number of re-certifications. For the best results, it is always a good idea to run the certification test plan more than the mandatory number of re-certifications, in your own test environment, just to be safe. Laboratories Labs can be useful in that both certifications and tests can be executed that don t build an internal expertise. Tools There are three primary reasons to use test tools: A test tool can be used to reduce the cost of testing. Due to the automation of the tool, the overall cost of the test is reduced. Especially true in regression testing, cost can be reduced by over 80%. Test simulator can be used to create a test environment. There will come a point when a component simulator will be necessary. Be sure to carefully compare software before purchasing, not all simulators are created equal. Mandatory test tools. Due to the effectiveness of test tools, Schemes have pushed Acquirers, Terminal Developers and Issuers into using test tools as a compulsory habit. While not always for KaSYS Canada ! Page 23 / 26

24 the best, be sure to lay out your requirements before going to buy test software, so that the best tool will be available for your use. KaSYS Canada ! Page 24 / 26

25 The Test Matrix Card Terminal Acquirer Issuer Specifications Schemes Card Specs + Issuer Profile Schemes Guidance + Acquirer Specs Schemes Network Specs + Acquirer Specs Schemes Network Specs + Issuer Specs Mandatory Test Practice None (Rely on the card manufacturer industrial process) + Slow rollout Hardware certifications + EMVco kernel certification + Scheme card interface certification + Scheme End-To-End certification + Terminal Approval Process Scheme Network Certification + Integrated Test Environment Scheme Network Certification Better Test Practice Profile Validation Regression Testing Regression Testing + Negative Testing + Stress Test Regression Testing Best Test Practice Personalization Test Negative Testing Monitoring Monitoring + Stress Test KaSYS Canada ! Page 25 / 26

26 About the Author Jean-Christophe Derre is general manager at KaSYS Canada since He can be reached at Comments, remarks are welcome. Short Bio Born in France in the late 60s, Jean-Christophe Derré is an entrepreneur & computer analyst. He has worked 2 years in London for a bank, then 5 years in Paris for the payment industry and the EMV migration. For the past 14 years he has worked in Canada as a Quality Assurance and EMV specialist. In 1999 he founded Savoir-faire Linux inc., a technology oriented company to spread Linux business use, and resold it in Since 2002, as the majority shareholder, he has managed and developed KaSYS Canada inc., the US and Canadian distributor of KaNest test tools. During the last decade within KaSYS Canada, he has helped the Canadian EMV migration with test automation solutions and test simulators used in all major Canadian financial institutions, and has been closely working with the Toronto-Montreal Exchange to build a Quality Assurance strategy. He has also been a driving force behind innovative test tools such as ParseIt and KaProbe. In the next decade, he is looking forward to continue to bring innovations to the Quality Assurance of the payment industry and help US migrate to EMV. - jc.derre@kasys.ca Thanks. KaSYS Canada ! Page 26 / 26