TREASURY INSPECTOR GENERAL FOR TAX ADMINISTRATION

Transcription

1 TREASURY INSPECTOR GENERAL FOR TAX ADMINISTRATION Final Integration Test Planning and Preparation May 8, 2015 Reference Number: This report has cleared the Treasury Inspector General for Tax Administration disclosure review process and information determined to be restricted from public release has been redacted from this document. Phone Number / Address / TIGTACommunications@tigta.treas.gov Website /

2 HIGHLIGHTS FINAL INTEGRATION TEST PLANNING AND PREPARATION Highlights Report issued on May 8, 2015 Highlights of Reference Number: to the Internal Revenue Service Chief Technology Officer. IMPACT ON TAXPAYERS The Final Integration Test (FIT) is a critical part of the IRS s preparation for each filing season. If tax processing systems are not properly integrated to deliver filing season functionality, taxpayers may be unable to timely file returns, receive refunds, or obtain timely, accurate customer service. WHY TIGTA DID THE AUDIT The IRS is in the process of making significant changes to its tax processing system to implement legislative changes, including provisions of the Patient Protection and Affordable Care Act (ACA). These changes will result in increased workload and challenges for the FIT program. TIGTA conducted this audit to help ensure the effectiveness of the FIT program s Planning and Preparation process steps in testing the integration of an increasingly complex tax processing system. WHAT TIGTA FOUND The FIT program team effectively planned and prepared for the Processing Year 2015 FIT. The team conducted and completed all required planning and preparation activities as well as took corrective actions on several of the previous TIGTA audit report recommendations. However, key systems and programs were not sufficiently developed and tested before delivery to the FIT environment. For example, during the FIT Execution process step, problem tickets were recorded for issues that should have been identified and corrected during systems acceptability testing. A production-level build of the ACA applications was also not provided to the FIT program team at the start of FIT execution on November 3, As a result, the start of ACA test cases was delayed until a more advanced build of the ACA applications was delivered. The FIT program received eight builds of the ACA 5.0 systems between November 3, 2014, and January 15, The final build was received by the FIT program less than one week before the start of the 2015 Filing Season. The Modernized e-file system was delivered to the FIT environment with programming errors. Some of these delivery events caused FIT program analysts to open several priority one helpdesk tickets. A priority one ticket is the most critical priority level helpdesk ticket. In addition, FIT program analysts opened 182 helpdesk tickets for the Modernized e-file system during the Processing Year 2015 FIT. Errors continued to be identified in the ACA and Modernized e-file application and system builds that caused delays to the FIT schedule. Not completing the planned FIT execution on schedule prior to the 2015 Filing Season increases the risk of filing season processing errors that would have been identified and corrected during the FIT. WHAT TIGTA RECOMMENDED TIGTA recommended that the Chief Technology Officer ensure that: 1) production builds of systems are timely delivered to the FIT environment and 2) systems are sufficiently developed and tested prior to delivery to the FIT environment. The IRS acknowledged the report recommendations, but asserts that adequate procedures are in place to plan and schedule the timely delivery of systems to all testing environments. TIGTA is concerned that the IRS response is not adequate to ensure timely delivery of quality programs to the FIT environment. For example, the current procedures resulted in the FIT program receiving eight builds of the ACA 5.0 systems with the final build received by the FIT program less than one week before the start of the 2015 Filing Season. Based on the late delivery of systems to the FIT environment and the number of errors identified during FIT testing, the IRS should increase efforts to timely deliver programs and reduce programming errors.

3 DEPARTMENT OF THE TREASURY WASHINGTON, D.C TREASURY INSPECTOR GENERAL FOR TAX ADMINISTRATION May 8, 2015 MEMORANDUM FOR CHIEF TECHNOLOGY OFFICER FROM: SUBJECT: Michael E. McKenney Deputy Inspector General for Audit Final Audit Report Final Integration Test Planning and Preparation (Audit # ) This report presents the results of our review of Final Integration Test Planning and Preparation. The overall objective of this review was to evaluate the Planning and Preparation process steps of the Internal Revenue Service Processing Year 2015 Final Integration Test. This review is included in the Treasury Inspector General for Tax Administration s Fiscal Year 2015 Annual Audit Plan and addresses the major management challenge of Modernization. Management s complete response to the draft report is included as Appendix V. Copies of this report are also being sent to the Internal Revenue Service managers affected by the report recommendations. If you have any questions, please contact me or Alan R. Duncan, Assistant Inspector General for Audit (Security and Information Technology Services).

4 Table of Contents Background... Page 1 Results of Review... Page 4 The Final Integration Test Program Team Effectively Planned and Prepared for Processing Year Page 4 Key Systems and Programs Were Not Sufficiently Developed and Tested Before Delivery to the Final Integration Test Environment... Page 6 Appendices Recommendations 1 and 2:... Page 9 Appendix I Detailed Objective, Scope, and Methodology... Page 10 Appendix II Major Contributors to This Report... Page 12 Appendix III Report Distribution List... Page 13 Appendix IV Glossary of Terms... Page 14 Appendix V Management s Response to the Draft Report Page 17

5 Abbreviations ACA EST FIT IRS IT MeF PY SAT TIGTA Affordable Care Act Enterprise Systems Testing Final Integration Test Internal Revenue Service Information Technology Modernized e-file Processing Year Systems Acceptability Test Treasury Inspector General for Tax Administration

6 Background The Internal Revenue Service (IRS) defines the Final Integration Test (FIT) 1 program as the integrated end-to-end testing of multiple systems that support the high-level business requirements of the IRS. The FIT is the final step of the application software testing effort designed to ensure that revisions to IRS computer applications interoperate correctly prior to the tax return filing season. Each FIT performed consists of a series of tests designed to ensure that essential IRS applications will perform correctly when deployed. The FIT is performed from the perspective that all IRS applications are subsystems of the overall tax processing system. The tax processing system consists of hundreds of applications operating on many unique hardware and software platforms. The IRS creates a FIT environment that emulates, as closely as possible, the filing season environment. The FIT is planned, developed, scheduled, and conducted from the perspective of the end-user (i.e., the taxpayer). Using copies of filing season data, FIT personnel verify that data are transferred correctly between the applications within the tax processing system. Each year, the IRS incorporates system improvements and changes to the tax law into the tax processing system. For Processing Year (PY) 2015, the FIT program had the additional challenge of including the new Affordable Care Act 2 (ACA) Release 5.0 systems in the integration test. ACA Release 5.0 has a broad scope that affects systems in the core of the tax processing pipeline. It introduces significant use of third-party data matching using two new applications, the ACA Verification Service and the Coverage Data Repository (hereafter, referred to as ACA 5.0 systems). The ACA contains comprehensive health insurance reforms for both individuals and employers and establishes a new health insurance marketplace (exchanges) where health insurance coverage can be purchased. The Department of Health and Human Services is the lead agency implementing the health care law, while the IRS is responsible for administering nearly 50 tax provisions included in the law. Not all ACA provisions are directly associated with health insurance coverage or the health insurance marketplace. Most ACA tax provisions are in effect now, and more will be implemented during the next several years. Beginning in January 2015, the IRS began receiving individual tax returns (and information returns from health insurance exchanges, health insurance companies, and employers) that pertain to the premium tax credit and to individual and employer-shared responsibility coverage. The premium tax credit at-filing processing will be conducted on all individuals who submit a 1 See Appendix IV for a glossary of terms. 2 The Patient Protection and Affordable Care Act of 2010, Pub. L. No , 124 Stat. 119 (2010) (codified as amended in scattered sections of the U.S. Code), as amended by the Health Care and Education Reconciliation Act of 2010, Pub. L. No , 124 Stat Page 1

7 tax return. Those individuals enrolled in health care coverage at an exchange may have elected to receive the advance premium tax credit during enrollment and now need to reconcile any monies provided on their behalf with the value of the allowable premium tax credit based on current year income, family size, and family situation through the completion and attachment of Form 8962, Premium Tax Credit, to the tax return. For ACA 5.0 systems, the IRS must be able to: Receive and process information returns and tax returns with associated attachments containing ACA data. Store the new ACA data for downstream processing and analysis. Perform limited at-filing compliance checks to identify discrepancies early in the reporting and collection process rather than having to wait for post-filing activities. Handle and address ACA inquiries and balance due conditions from customers. Meet new reporting requirements for all of the ACA data being collected and stored. While the overall responsibility for the FIT program lies with the Information Technology (IT) organization s Enterprise Systems Testing (EST) Division, the FIT program requires the participation and support of several other organizations (including Applications Development, Enterprise Operations), the business units, and contractors. For example, the FIT program receives contractor support funded by the Modernized e-file (MeF), Customer Account Data Engine 2, and ACA program offices. The EST Division s mission is to work in partnership with its customers to improve the quality of IRS applications, products, and services. To accomplish this mission, the EST Division plans, develops, schedules, and conducts the FIT from the perspective of the end-user. The FIT is conducted in four process steps: Planning, Preparation, Execution, and Closeout. Each of the four process steps contains stated requirements for entry criteria, process activities, output, and exit criteria. This audit focused on the Planning and Preparation process steps for the PY 2015 annual FIT. The IRS conducted planning and preparation between March and November A prior Treasury Inspector General for Tax Administration (TIGTA) audit 3 reviewed the Execution and Closeout process steps and recommended that the Chief Technology Officer should: Ensure that an analysis of problem tickets is performed to determine if there are any root causes that can be addressed to reduce the burden on the FIT program and IT organization resources. 3 TIGTA, Ref. No , Increased Support Is Needed to Ensure the Effectiveness of the Final Integration Test (Sept. 2014). Page 2

8 Identify and provide the resources needed to ensure the necessary level of Enterprise Operations organization and Wage and Investment Division support to the FIT program. Ensure that the Integrated Customer Communications Environment is included in the FIT program s environment. Establish performance goals and metrics for the annual FIT program and establish procedures for reporting on the progress in meeting those goals and metrics. Establish a process to determine the reason FIT test tax returns go to the Error Resolution System and use those results to identify training opportunities or operational improvements. Implement the environment comparison and synchronization process between the FIT program s environment and the filing season environment. This review was performed at the IRS FIT program office in Lanham, Maryland, during the period November 2014 through February We conducted this performance audit in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objective. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objective. Detailed information on our audit objective, scope, and methodology is presented in Appendix I. Major contributors to the report are listed in Appendix II. Page 3

9 Results of Review The Final Integration Test Program Team Effectively Planned and Prepared for Processing Year 2015 Internal Revenue Manual , Software Testing Standards and Procedures, IT Test Policy, 4 and Internal Revenue Manual , IT Software Testing Process and Procedures, 5 includes requirements for the Planning and Preparation process steps that are part of software testing. The Planning and Preparation process steps are completed to execute software tests within the IRS. Planning is the first process step in the software testing process. Planning activities include assessing requirements, establishing the test environment, training the test team, and developing the test plan. The Planning process step is complete when requirements are baselined, the test environment is established, the test plan is created, and the test repository is established. Preparation is the second process step in the software testing process. Preparation activities include verifying the test environment; reviewing documentation; preparing test cases, scripts, and data; and conducting the test readiness review. The Preparation process step is complete when the test readiness review is complete, the test plan is finalized and issued, test cases are developed, and the test environments are verified. The FIT program team effectively conducted and completed all required planning and preparation activities and documents The FIT program team carried out all required planning and preparation activities to sufficiently execute testing for the PY 2015 FIT. Planning for the PY 2015 FIT began in March 2014, immediately following the conclusion of the PY 2014 FIT. All required FIT program team roles and responsibilities such as business lead, program manager, test analyst, test lead, and test manager were assigned, as instructed in Internal Revenue Manual , to effectively carry out planning and preparation activities. The FIT program team prepared and approved the master schedule, technical test plan, and test readiness review checklist and conducted the test readiness review meeting. The FIT program team also addressed and corrected outstanding issues and lessons learned from the PY 2014 FIT. Each year, the FIT program makes a significant contribution to the success of the filing season. For PY 2015, the FIT program developed a test plan including more than 2,700 test cases to test the integration of 29 tax processing systems included in the scope of the PY 2015 FIT. There are more than 450 new test cases for PY 2015 for the ACA 5.0 systems. 4 Internal Revenue Manual (April 17, 2014). 5 Internal Revenue Manual (May 16, 2014). Page 4

10 The ACA Verification Service validates that the premium tax credit or the advance premium tax credit reported on a tax return is correct and matches health care exchange information contained within the Coverage Data Repository. The ACA test cases were designed to test the validation checks that the ACA Verification Service performs. All FIT test cases were entered into the Rational Quality Manager system to record and monitor test case results. Planning activities include reviewing all work requests to determine if any information technology system changes made throughout the year will affect the FIT and require additional test cases. The FIT team tracks all work requests for every subsystem within the scope of the FIT to ensure that all changes or modifications to systems are known, tracked, and incorporated into test cases. To plan for the PY 2015 FIT, the team reviewed more than 2,900 work requests. The work requests were reviewed in April 2014 and again in September Reviewing the work requests is a significant activity in the Planning process step. For PY 2015, the review identified system changes that would affect all of the subsystems included in the scope of the PY 2015 FIT. As a result of the review, new test cases were prepared to test those system changes. For example, testers identified 145 work requests related to the ACA 5.0 systems. Of those, testers identified 100 work requests that affect the scope of the FIT. This process ensures that all system changes are identified and tested during the FIT to ensure readiness for the tax filing season. Corrective actions were taken on previous TIGTA audit report recommendations to improve the FIT IRS IT organization management has begun implementing corrective actions on recommendations that TIGTA made in a September 2014 report. 6 For example, steps have been taken to include the Integrated Customer Communications Environment in the FIT test environment. Inclusion of the Integrated Customer Communications Environment in the FIT test environment allows customer service applications such as Where s My Refund? to be tested. However, due to the late approval of funding for the Integrated Customer Communications Environment, the system was not included in the scope of the PY 2015 FIT. The system should be included in the PY 2016 FIT. The FIT program also developed and delivered Error Resolution System training sessions to address common errors. In addition, the test case tracking tool, Rational Quality Manager, has been modified to include a marker that indicates whether the Error Resolution System results were unintentional or a planned test case outcome. This allows the IRS to easily determine the test cases that were not executed properly. The FIT Standard Operating Procedures were updated to include a procedure to review the Error Resolution System results for unnecessary errors and missed testing opportunities. The FIT program also developed and delivered a 6 TIGTA, Ref. No , Increased Support Is Needed to Ensure the Effectiveness of the Final Integration Test (Sept. 2014). Page 5

11 six-week training event for FIT program personnel to help improve the quality of test cases and test case execution. To ensure that the necessary resources are provided, the FIT program extended the length of time field support resources are needed to complete the annual FIT. Additional tax examiner resources were provided to the PY 2015 FIT program to assist with the additional ACA test cases. Finally, the EST Division has finalized a memorandum of understanding with the Enterprise Operations organization that identifies the level of support required for the annual FIT. The IRS also stated that corrective actions for two prior TIGTA recommendations are in process. The recommendation to analyze problem tickets to determine if there are any root causes is expected to be performed during the test closeout activities for the PY 2015 FIT. The results of this analysis should assist the IRS in determining the root cause of many programming errors. The IRS is also in the process of implementing a plan to identify goals, measures, and a reporting process that will require additional resources to implement, manage, and support the identification and reporting of performance metrics. The FIT program team was prepared to conduct tests with or without the ACA 5.0 systems Prior to the start of the FIT execution in November 2014, the FIT program team had not received the ACA 5.0 systems and updated the FIT environment. In preparation for the possibility that the ACA 5.0 systems would not be deployed to the FIT environment for the start of the FIT execution, the FIT program team prepared testing bypass procedures. Employing the bypass procedures would have allowed most of the FIT test cases to continue processing while bypassing the ACA verification test cases. Once the ACA 5.0 systems were deployed to the FIT environment on November 3, 2014, the FIT program team requested the bypass procedure be turned off. The FIT program team has continued to test the ACA 5.0 systems during test execution. Key Systems and Programs Were Not Sufficiently Developed and Tested Before Delivery to the Final Integration Test Environment The FIT program requires a production-like environment to effectively test the interoperability of all of the tax processing systems. The EST Division has overall responsibility for FIT program execution. The EST Division also is responsible for completing functional testing of systems before the systems can be included and tested in the annual FIT. The FIT Concept of Operations document states that the FIT is a final preproduction test that occurs at the conclusion of either the IRS development and functional testing cycle or the modernization systems release integration testing. Page 6

12 The FIT PY 2015 technical test plan includes the following assumptions: All applications being tested have been unit and compatibility tested. All defects identified during these tests have been corrected before the beginning of the FIT. All applications have been through at least one cycle of the Systems Acceptability Test (SAT) level and any defects that would prevent the FIT from starting have been corrected. The FIT environment has been loaded and test environment connectivity has been verified. ACA applications were not sufficiently tested and developed before delivery to the FIT environment In order to simulate the production environment as closely as possible, the FIT program must test the integration of tax processing systems with versions or builds of production systems. The FIT program was not provided with a production build of the ACA 5.0 systems prior to the start of FIT execution on November 3, The IT organization s ACA Program Management Office release-level testing and SAT testing were still ongoing when the FIT environment received the first build of the ACA 5.0 systems for the start of test execution. At that time, there were Coverage Data Repository performance concerns that had not been resolved and the release-level testing completion date for the ACA Verification Service was pushed back to December 17, As a result, FIT program management prudently delayed the start of ACA test cases until a more advanced build of the systems was delivered to the FIT environment. On November 23, 2014, the FIT program deployed the next build of the ACA 5.0 systems and began testing. The FIT program received eight builds of the ACA 5.0 systems between November 3, 2014, and January 15, After deploying the second ACA 5.0 systems build, the FIT began experiencing connectivity issues with ACA Verification Service, causing major delays in its test cases. On December 5, 2014, a third ACA 5.0 systems build was deployed, followed by a fourth build on December 14, Errors continued to be found on the fourth build, causing the FIT program team to stop processing certain test cases on the MeF or ACA 5.0 systems. The fifth build was deployed on December 31, 2014, followed by a sixth build on January 9, Additional errors were discovered until the seventh and eighth builds were both deployed on January 15, 2015, near the end of test execution and approximately one week before the start of the 2015 Filing Season. In addition to the delays in delivery of the ACA 5.0 systems to the FIT environment, SAT testing of the ACA 5.0 systems was not sufficiently completed before the start of the FIT Execution process step. As a result, SAT testing continued to identify and correct problems with the systems that should have been corrected before delivery to the FIT environment. SAT identified critical-level problems with the ACA Verification Service and Coverage Data Repository after Page 7

13 the systems were delivered for FIT execution and continued to create critical- and high-level helpdesk tickets throughout the duration of the FIT execution. MeF programs were delivered to the FIT environment with programming errors Between November 2014, the start of test execution, and January 2015, many versions of MeF programs and patches were delivered to the FIT environment. Some of these delivery events caused work stoppages that caused FIT program analysts to open several priority one helpdesk tickets. A priority one ticket is the most critical level of helpdesk ticket. In addition, FIT program analysts opened 182 helpdesk tickets for the MeF system during the PY 2015 FIT. The ACA 5.0 systems is one of the most complex ACA system releases, and it was a significant challenge for the IT organization s ACA Program Management Office to timely deliver the systems for the 2015 Filing Season. Programming errors have occurred in the past, including the PY 2014 FIT. These errors are the result of systems delivered that are not fully tested and not ready for the FIT. The previous TIGTA audit report on the 2014 FIT test execution recommended an analysis of helpdesk tickets to identify the root cause of problems identified during the FIT. To help identify the root cause of the problems, IT organization management agreed, contingent on the availability of resources, to analyze the FIT program s helpdesk tickets as part of the PY 2015 FIT closeout activities. Because this condition was previously reported and management s planned corrective actions are pending, we will not repeat the recommendation in this audit report. Delivery of ACA 5.0 systems that are not fully tested and the errors corrected negatively affect the FIT program s ability to effectively perform the FIT execution. As of January 20, 2015, the FIT program opened 33 critical helpdesk tickets after receiving ACA and MeF systems into the FIT environment. Of the 33 critical helpdesk tickets, 23 (70 percent) are related to the ACA 5.0 and MeF systems (12 for the ACA and 11 for the MeF). Many of these tickets were caused by missing requirements and programing errors. The critical-level helpdesk tickets caused work stoppages or were problems that would have affected significant numbers of tax returns if not corrected before the start of the 2015 Filing Season. The FIT program analysts also opened 202 high-level helpdesk tickets after receiving the ACA 5.0 and MeF systems for testing. Of these 202 tickets, 123 (61 percent) were related to the MeF, indicating that the MeF system was not ready for FIT execution. Some of these helpdesk tickets caused work stoppages for the FIT program. MeF problems are significant because the majority of tax returns are filed electronically through the MeF system. During PY 2014, approximately 85 percent of individual tax returns were filed electronically. Each build of a system and system patch resulted in problems for the FIT execution. FIT program analysts must analyze the problem, document the problem by opening a helpdesk ticket, and work with programmers to resolve the problem. Interruptions to FIT program activities, system downtime, and working to resolve problems affects the effectiveness of the FIT by taking resources from FIT program activities. Not completing the planned FIT execution on schedule Page 8

14 prior to the 2015 Filing Season increases the risk of filing season processing errors that would have been identified and corrected during the FIT execution. Recommendations To ensure that the FIT program timely, effectively, and efficiently completes its duties, the Chief Technology Officer should: Recommendation 1: Ensure that production builds of systems are timely delivered to the FIT environment. Management s Response: The IRS acknowledges this recommendation, but asserts that adequate procedures are in place to plan and schedule the timely delivery of systems to all testing environments. The testing organization establishes a risk-based schedule that defines when systems must be delivered to the functional and integration test environments in preparation for filing season testing. When planned delivery schedules cannot be achieved, risk-based actions are taken to mitigate the impact, under the guidance of senior IT management. The IRS plans to reinforce this approach for the 2016 Filing Season, working with the stakeholder organizations to define an acceptable schedule for delivery of all critical applications to FIT. Recommendation 2: Ensure that systems are sufficiently developed and tested prior to delivery to the FIT environment. Management s Response: The IRS acknowledges this recommendation, but asserts that adequate procedures are in place to plan and schedule an acceptable level of testing prior to the start of FIT testing. The delivery schedule described in the IRS s corrective action plan will also define dependences with other testing activities to help ensure that an acceptable level of testing is conducted on critical systems prior to the FIT testing of those systems for the 2016 Filing Season. Office of Audit Comment: The IRS response is not adequate to ensure timely delivery of quality programs to the FIT environment. For example, the current procedures resulted in the FIT program receiving eight builds of the ACA 5.0 systems with the final build received by the FIT program less than one week before the start of the 2015 Filing Season. Based on the late delivery of systems to the FIT environment and the number of errors identified during FIT testing, the IRS should increase efforts to timely deliver programs and reduce programming errors. Page 9

15 Detailed Objective, Scope, and Methodology Appendix I Our overall objective was to evaluate the Planning and Preparation process steps of the IRS s PY FIT. To accomplish our objective, we: I. Evaluated the effectiveness of the planning and preparation activities for the PY 2015 FIT. A. Determined if required roles and responsibilities for FIT planning and preparation were assigned to effectively conduct the PY 2015 FIT. II. B. Determined if lessons learned from the previous FIT and issues from the PY 2014 End-of-Test Report were adequately addressed and corrected prior to the start of the PY 2015 FIT. C. Determined if corrective actions were taken to address recommendations from the prior TIGTA audit report. 2 D. Determined if work requests were properly evaluated to identify the risks that need to be considered and included in the scope of the PY 2015 FIT. E. Determined if ACA 5.0 systems were included in the scope of the PY 2015 FIT. F. Determined if test cases were developed to test all systems included in the scope of the PY 2015 FIT. We determined if test cases are recorded in the Rational Quality Manager system. G. Determined if the FIT environment was prepared and requirements received for all systems and programs included in the scope of the PY 2015 FIT. H. Determined if the technical test plan was finalized and the test readiness review was conducted to mark completion of the Planning and Preparation process steps of the PY 2015 FIT. Determined the impact on the PY 2015 FIT and the risks to the filing season if the production versions of the ACA 5.0 systems could not be included in the PY 2015 FIT. A. Determined the actions that would be taken by FIT program management if the ACA 5.0 systems were not available to be included in the PY 2015 FIT. 1 See Appendix IV for a glossary of terms. 2 TIGTA, Ref. No , Increased Support Is Needed to Ensure the Effectiveness of the Final Integration Test (Sept. 2014). Page 10

16 B. Evaluated the information obtained in II.A and determined the impact on the scope of the FIT and the potential risk to the IRS s tax processing system if the FIT could not perform testing on a final build of the ACA 5.0 systems. Internal controls methodology Internal controls relate to management s plans, methods, and procedures used to meet their mission, goals, and objectives. Internal controls include the processes and procedures for planning, organizing, directing, and controlling program operations. They include the systems for measuring, reporting, and monitoring program performance. We determined that the following internal controls were relevant to our audit objective: the IRS s software testing standards and procedures. We evaluated these controls by interviewing management and reviewing IRS documentation supporting the completion of exit criteria for the Planning and Preparation process steps of the PY 2015 FIT. Page 11

17 Major Contributors to This Report Appendix II Alan R. Duncan, Assistant Inspector General for Audit (Security and Information Technology Services) Danny Verneuille, Director John Ledford, Audit Manager Joan Bonomi, Lead Auditor Chinita Coates, Auditor Kasey Koontz, Auditor Page 12

18 Report Distribution List Appendix III Commissioner C Office of the Commissioner Attn: Chief of Staff C Deputy Commissioner for Operations Support OS Commissioner, Wage and Investment Division SE:W Deputy Chief Information Officer for Operations OS:CTO Associate Chief Information Officer, Applications Development OS:CTO:AD Associate Chief Information Officer, Enterprise Operations OS:CTO:EO Associate Chief Information Officer, Enterprise Services OS:CTO:ES Director, Enterprise Systems Testing, Enterprise Services OS:CTO:ES:EST Director, Office of Legislative Affairs CL:LA Director, Office of Program Evaluation and Rick Analysis RAS:O Chief Counsel CC National Taxpayer Advocate TA Office of Internal Control OS:CFO:CPOC:IC Audit Liaison: Director, Risk Management Division OS:CTO:SP:RM Page 13

19 Glossary of Terms Appendix IV Term ACA Verification Service Applications Development Organization Coverage Data Repository Definition This system integrates with the current processing environment to perform compliance checks and validate information on tax forms related to health insurance. The system identifies exceptions associated with the Patient Protection and Affordable Care Act 1 when the returns are filed. Catching exceptions such as math errors and returns that do not match corresponding third-party data will reduce the number of returns that are flagged for downstream checks, routed to the Error Resolution System, or rejected altogether. A part of the IRS IT organization responsible for building, testing, delivering, and maintaining integrated information technology applications to support modernized systems and the filing season environment. This database will support ACA provisions by using data imported from the Integration Production Model, which contains data from the National Account Profiles, the Individual Masterfile, and the Individual Return Transaction File. The Coverage Data Repository contains information from the following projects: Income and Family Size Verification, Premium Tax Credit, ACA Information Returns, and the Infrastructure Security Review. It also contains information received through the Department of Health and Human Services data hub that is retrievable for at-filing and post-filing usage. 1 The Patient Protection and Affordable Care Act of 2010, Pub. L. No , 124 Stat. 119 (2010) (codified as amended in scattered sections of the U.S. Code), as amended by the Health Care and Education Reconciliation Act of 2010, Pub. L. No , 124 Stat Page 14

20 Term Customer Account Data Engine 2 Enterprise Operations Error Resolution System Filing Season Final Integration Test Concept of Operations Information Technology Organization Integrated Customer Communications Environment IT Organization ACA Program Management Office Definition An IRS application that will replace the existing Individual Master File and Customer Account Data Engine applications. The Customer Account Data Engine 2 is designed to provide state-of-the art individual taxpayer account processing and technologies to improve customer service to taxpayers and enhance tax administration. A part of the IRS IT organization that provides server and mainframe computing services for all IRS business entities and taxpayers. An application that provides for the correction of errors associated with input submissions. The error inventory is managed on a database, and corrected documents are validated by the Generalized Mainline Framework system. The period from January through mid-april when most individual income tax returns are filed. A document that provides an overview of the methodology for planning and executing the FIT of software systems before deployment to the current filing season environment. The IRS organization responsible for delivering information technology services and solutions that drive effective tax administration to ensure public confidence. An automated self-help system that provides customer service applications through toll-free telephone service and the Internet. The toll-free telephone service provides automated self-service applications that allow taxpayers to help themselves as well as provides avenues to route taxpayers to live customer service representatives. The Internet component of the Integrated Customer Communications Environment allows taxpayers to check their refund status. The IRS office responsible for managing the strategic planning, development, implementation, and testing of new information systems in support of business requirements with regard to the Affordable Care Act. Page 15

21 Term Modernized e-file Processing Year Definition The IRS s electronic filing system that enables real-time processing of tax returns while improving error detection, standardizing business rules, and expediting acknowledgements to taxpayers. The system serves to streamline filing processes and reduce the costs associated with a paper-based process. The calendar year in which the tax return or document is processed by the IRS. Systems Acceptability Test Testing performed by the Test, Assurance, and Documentation group to independently assess the quality of the application software by testing with controlled data to determine conformance of the system to customer requirements and to aid the customer and developer in determining the system s tax return filing season readiness. Test Case Work Request The foundation of a test. A test case references specific test data and the expected results associated with specific program criteria. It is used to verify a specific process in the application software and to test system requirements. Gives the detailed business requirements for data requests so that the IRS can properly review, assign, analyze, and respond (approve/deny) to the request and can also cost and schedule the request for the implementation and delivery of any agreed upon information technology products or services. Page 16

22 Management s Response to the Draft Report Appendix V Page 17

23 Page 18

24 Page 19