Using Independent Quality Assurance (QA) Services. Presented By: Kathy Reczkiewicz, CSG Government SoluFons Eric Smith, Oregon Department of Revenue

Transcription

1 Using Independent Quality Assurance (QA) Services Presented By: Kathy Reczkiewicz, CSG Government SoluFons Eric Smith, Oregon Department of Revenue

2 Kathy Reczkiewicz Client Execu,ve, CSG Government Solu,ons Responsible for facilitafng prompt and honest communicafon between Oregon and the CSG project team, ensuring that the QA services operate smoothly and meet State expectafons Over 18 years of experience with all phases of the souware development life cycle, including analysis, planning, programming, tesfng, and training 13 years of experience as COTS souware engineer and product manager 8/ CSG Government SoluFons 2

3 About CSG Government SoluFons CSG GOVERNMENT SOLUTIONS AT- A- GLANCE BUSINESS: Government OperaFons ConsulFng HEADQUARTERS: Chicago, IL YEAR FOUNDED: 1997 EMPLOYEES: 300 CSG REALize ArNfacts and TeamCSG Tools SERVICE OFFERINGS: Strategy & Planning Enterprise architecture Requirements definifon Business process reengineering Procurement support APD development PROGRAM EXPERTISE: Government Programs Healthcare Human Services Child Support Unemployment Insurance CLIENTS: 38 state governments U.S. Department of Health and Human Services Project Assurance Services Project Management Office IV&V / QA OrganizaFonal change management SoUware tesfng support Privacy and security Revenue / Tax EducaFon TransportaFon JusFce Motor Vehicles U.S. Department of Labor Large municipal governments CSG invests in custom QA tools to achieve the highest results for our clients. IT Infrastructure Risk Assessment Model SDLC Risk Assessment Model Project Management Risk Assessment Model OperaFonal Readiness Tool Risk Assessment Model Quality Checklists Sample Quality Management Plan Sample Master Test Plan 8/ CSG Government SoluFons 3

4 Eric Smith Core Systems Replacement Project Director Oregon Department of Revenue (25 years) 8/ CSG Government SoluFons 4 Responsible for leading the CSR project, securing funding, negofafng contracts, building the project team and implemenfng new system Established DOR s Program Management Office which provides agency- wide governance and project management support for large inifafves Experienced in many revenue programs, especially Corporate and Property Tax

5 Agenda The Oregon DOR Core Systems Replacement (CSR) Project What is Independent QA/IV&V? Roles & ResponsibiliFes Value & Benefits 8/ CSG Government SoluFons 5

6 Oregon DOR Core Systems Replacement Readiness and procurement phase Business Case Reduce risk of future failure and improve performance Commercial off- the- shelf solufon (COTS) selected as most viable ExecuFve branch and legislafve approval and funding processes Engaged Quality Assurance (QA) vendor early $7M (July 2009 to June 2013) total readiness phase cost ExecuFon phase Implement GenTax integrated tax system and data warehouse Replace two- thirds of legacy systems in four rollouts; ends fall 2017 EsFmated total cost: $85M (bonds and General Fund) $67M implementafon, $11M finance, and $7M readiness Changed QA/QC vendor to align with execufon approach 8/ CSG Government SoluFons 6

7 Oregon Quality Assurance Oversight Oregon requires independent QA oversight Policy began in early 90 s auer a high profile failed IT project (DMV) Recently passed law (HB 4122, 2014 session) requiring QA for all IT projects over $5M ReporFng to state oversight bodies ExecuFve and LegislaFve Oversight ExecuFve branch State CIO and Dept. of AdministraFve Services LegislaFve branch LegislaFve Fiscal Office ContracFng for QA Select from 11 vendors with statewide price agreements Standard Statement of Work tailored to suit specific needs Find right fit for experience (business domain) and skills 8/ CSG Government SoluFons 7

8 What is Independent QA? An Independent third party provides QA services and: Avoids conflicts of interest Avoids can t see the forest for the trees syndrome Provides addifonal advice and guidance from a different perspecfve and body of experience Removes any pride of ownership issues Can help cut through drama, polifcs, and bureaucracy Eyes and Ears of the Steering Commiree 8/ CSG Government SoluFons 8

9 Why do States Hire QA Vendors? The Need for QA Sometimes Mandated by Law Complexity of system implementation projects Highly publicized budget and political scrutiny Lack of specific program management experience Benefits to the State Identifying and reporting risks early Detecting project cost and schedule impacts early Serving as a mentor and knowledge transfer agent Improving the quality of deliverables Facilitating project communication Providing risk response plans 8/ CSG Government SoluFons 9

10 Approach and QA Methodology Provide a dedicated, experienced QA project team with COTS experience Develop a well- structured Work Breakdown Structure and execute it with discipline Apply QA experience with COTS and Oregon QA processes for the DOR CSR Project Leverage well developed tool set Balance collaborafon and independence Follow no surprises approach to QA project deliverables 8/ CSG Government SoluFons 10

11 CSG Overall Approach 8/ CSG Government SoluFons 11

12 Approach Task 1: Quality Management Planning Quality Standards Operational Definitions Quality Checklists Quality Management Plan Baseline Project Plan Internal/External Presentations and Special Requests Approach Customize Oregon Generic SoUware Project Quality Standards and CSG checklists Create the CSR Project Quality Management Plan Baseline project plans by reviewing project documentafon and aligning them with expected resources and vendor deliverables Tools Risk Assessment Model Quality Checklists Sample Quality Management Plan Lessons Learned Report Project Evaluation 8/ CSG Government SoluFons 12

13 Quality Control Reviews Security Code Review and Sampling Quality Status Reporting and Tracking Monthly QA Reports Approach Approach and Tools Task 2: Quality Control Conduct quality control reviews of vendor deliverables drawing on CSG s previous implementafon methodology experience and best pracfces UFlize our QA Methodology in conducfng security code reviews on samples of the code within the system from different developers to ensure that they incorporate the accepted coding techniques that reduce security vulnerabilifes Tools Industry standard StaFsFcal Quality Control (SQC) calculator to determine the number of security code sets to sample Oregon s Generic SoUware Project Quality Standards 8/ CSG Government SoluFons 13

14 Quarterly QA Status and Improvements Reports / Presentations Semi- Annual Review and Interview Report Approach Approach and Tools Task 3: Quality Assurance Prepare the execufve- level Quarterly QA Status and Improvement Reports and PresentaFons summarizing project risks, interviews, and recommended correcfve acfons Complete the DAS OCIO Project Assessment Report and Project Budget & Schedule Variance Report Tools Risk Assessment Model DAS OCIO Templates 8/ CSG Government SoluFons 14

15 IV&V Master Test Plan Bi- Weekly Test Execution Status Report Approach and Tools Task 4: Independent VerificaFon & ValidaFon Approach UFlize our Arribute Test Sampling Methodology based on MIL- STD- 105E/ISO 2859 PrioriFze test sampling based on CSG and DOR experience to focus on the most crifcal parts of the system Code Review Sampling System Vulnerability and Security Assessment Tools Sample Master Test Plan 8/ CSG Government SoluFons 15

16 Approach and Tools Task 5: Risk Assessment Initial Risk Assessment On- Going Risk Notification Report Updated Risk Assessment Report Approach Tools Implement a proacfve, predicfve, and collaborafve IV&V/QA approach to mifgate risk factors and eliminate crifcal problems before they occur UFlize standards- based methodologies and tools that are customized for your needs to verify, validate, and mifgate risk Risk Assessment Model and Tracking Tool 8/ CSG Government SoluFons 16

17 DOR CSR Project Roles CSG QA Team System Vendor Team CSR Project DOR Team 8/ CSG Government SoluFons 17

18 DOR CSR Project Team Roles & ResponsibiliFes ExecuNve Sponsor ExecuFve decision- maker Responsible for leadership and advocacy of ongoing funding with the legislafve review process ExecuNve Steering CommiUee Enterprise- level strategic direcfon Defining resources available to execufve the CSR project Addressing significant escalated risks Budget funding/changes OperaNons and Policy Team Represent funcfonal areas of the agency, providing cross- divisional input and oversight of CSR acfvifes AllocaFng operafonal staff OperaFonal metrics 8/ CSG Government SoluFons 18

19 DOR CSR Project Team Roles & ResponsibiliFes Project Director Changes to scope or schedule ResoluFon of QA issues Project health metrics Project staffing External oversight reporfng Approval of deliverables Project Manager Development of all project management acfvifes Manages scope, reports progress IdenFfies and manages project risks Quality Assurance Coordinator Review, provide feedback, and comment on all QA deliverables Primary liaison to the CSG QA Team 8/ CSG Government SoluFons 19

20 CSG QA Project Team Roles & ResponsibiliFes Project Manager Coordinates the day- to- day acfvifes of the project and team Works directly with the CSR project team and vendors Quality Assurance Analyst Focuses on complefng reviews of the project and vendor business- related documents, processes, work products, and arffacts Provides experfse and is crifcal to ensure early idenfficafon and mifgafon of risks and issues, and analyze overall quality of system vendor deliverables Technical Analyst Provides key technical experfse as needed for specific areas within the QA and IV&V tasks e.g. code review sampling, system vulnerability tesfng Provides experfse for reviewing design and development processes and documents, security and system architecture plans, and tesfng processes and arffacts Subject MaUer Experts Delivers comprehensive knowledge in tax/revenue and experience with vendor implementafon methodology Senior Tester ParFcipates in the execufon of IV&V test sampling during the tesfng phase for each of the four rollouts 8/ CSG Government SoluFons 20

21 System Vendor Roles & ResponsibiliFes Implement the system and assist with business process changes Cooperate with the QA vendor ParFcipate in on- site interviews and QC review acfvifes Provide project deliverables and arffacts for QC review acfvifes Make project staff available as requested and needed during QC reviews Provide access to appropriate repositories and systems (e.g. test scenarios, requirements) 8/ CSG Government SoluFons 21

22 Value and Benefit of Independent QA for DOR Business Case (QC) Governance (QA) Project Management Plans (QC) Procurement (QA & QC) ExecuFon (Planning/QA/QC/IV&V/Risk Assessment) Seasoned PM experience and solid planning skills Experience with FAST s implementafon methodology and tax administrafon (good fit) Independent perspecbve that oversight authoribes trust (important) Risk and issue idenfficafon Ongoing QA of governance, decision making and communicafon IV&V tesfng Lessons learned for each rollout (Plan, Do, Check, Adjust) So far, confirmed that we are substanfally on the right path 8/ CSG Government SoluFons 22

23 QuesFons / Answers

24 Thank You! Eric Smith Project Sponsor Oregon Department of Revenue P: Kathy Reczkiewicz Client ExecuFve CSG Government SoluFons P: / CSG Government SoluFons 24