Review of Shared Services Canada s Collaborative Procurement Process

Transcription

1 Review of Shared Services Canada s Collaborative Procurement Process Presentation to the Information Technology Infrastructure Roundtable May 2015 Yves Genest, Chief Audit and Evaluation Executive Office of Audit and Evaluation

2 Agenda Overview of the Review Overview of the Collaborative Procurement Process (CPP) Summary of Findings: Interviews with Information Technology Infrastructure Roundtable (ITIR) members CPP file review CPP vendor feedback Literature review Conclusions Appendix 2

3 Overview of the Review Objective of the review: To identify opportunities for improvement Examination phase: December 2014 to March 2015 Methodologies employed: Interviews with ITIR members File review of CPP projects Analysis of vendor feedback Literature review [Additional details on these methodologies, as well as the review limitations and mitigation strategies, are provided in the Appendix.] Not an audit or evaluation 3

4 Overview of the Collaborative Procurement Process Developed at Shared Services Canada (SSC), building on Public Works and Government Services Canada s (PWGSC) National Shipbuilding Procurement Strategy and used for SSC s complex procurement projects. Refined by initial consultations with the information and communications technologies (ICT) sector (2011 to 2012). Aligned with SSC s vision to support value for money by closer collaboration and ongoing discussion with the ICT sector. Characterized by: engagement with industry early in the procurement process (pre- Requests for Proposal, or RFP) to co-develop requirements; generally shorter timeframes for the procurement process (from launch to contract award). 4

5 Overview of the Collaborative Procurement Process (Continued) The chronology of the typical CPP: 5

6 Summary: Interviews with Information Technology Infrastructure Roundtable members Key Themes: Engagement with Vendors SSC Capacity Innovation Small- and Medium-sized Enterprises Suggested Best Practices 6

7 Summary: Interviews with Information Technology Infrastructure Roundtable members (Continued) Engagement with Vendors: Overall, the CPP is viewed positively and as an improvement in engagement with vendors, compared to the traditional procurement approach. One-on-one meetings are viewed as the CPP s most helpful vendor engagement mechanism. Industry Days and Requests for Information are perceived more as platforms for SSC announcements. ITIR members indicated that there is no feedback on vendors input provided throughout CPP phases. A few ITIR members noted that the vendor community could better align with SSC s needs by having greater access to relevant SSC planning documents. 7

8 Summary: Interviews with Information Technology Infrastructure Roundtable members (Continued) SSC Capacity: ITIR members recommended that SSC increase its participation at innovation and trade events. Some members recommended keeping and developing leadingedge expertise in-house to drive procurements. ITIR members would like to see SSC explore opportunities to leverage the existing procurement capacity of other governments / departments and vendors (possibly through partnerships with them). SSC s procurement model is viewed as conservative, rigid and risk averse (as is the case with government procurement generally). 8

9 Summary: Interviews with Information Technology Infrastructure Roundtable members (Continued) Innovation: Innovation procurement was recognized as a challenge for SSC and the CPP. There are programs that test innovative technologies within government (PWGSC s Build in Canada Innovation Program). However, none are available to procure innovative services and products. As a way to support innovation, SSC could incorporate requirements for innovation in RFPs. Other opportunities could involve an innovation testing program within SSC, which could: serve as a test bed for potential CPPs prior to their launch; and focus on end-state business requirements. 9

10 Summary: Interviews with Information Technology Infrastructure Roundtable members (Continued) Small- and Medium-sized Enterprises (SME): SMEs make up an important part of the ICT sector in Canada. SMEs in Canada have demonstrated the capacity to innovate in various areas related to IT. SSC would benefit: by fostering more integration of SMEs in its procurements; and through a wider distribution of risk across project components. This could be achieved by: parcelling large procurements into integrated and layered components; and including clauses encouraging vendors to identify how SMEs will be leveraged. 10

11 Summary: Interviews with Information Technology Infrastructure Roundtable members (Continued) Suggested Best Practices: Increasing clarity about the end-state requirements for large IT procurements and possibly breaking down procurement into manageable pieces. Focusing on the overall value and merit as opposed to lowest cost. Favouring low bids does not always result in lower costs, as low bids may be a reflection of the amount of risk a vendor is willing to take. Vendors may recoup funds through other means, such as managing change orders. Incorporating proven innovative procurement solutions in other jurisdictions (provincial, U.S. government), including: set-asides for innovation pilots and SMEs; the provision of seed money to vendors for bid preparation; and the inclusion of a proof-of-solution stage prior to contract award. Measuring and reporting on procurement results. 11

12 Summary: File Review The Office of Audit and Evaluation (OAE) obtained files from SSC Procurement. Selected six CPP projects that had advanced the furthest through CPP phases. Reviewed files against Gartner s best practices. Results validated with procurement officers. CPP Project Transformation Initiative Data Centre Consolidation Services GCNet Wide Area Network (WAN) Services High-performance Computing Renewal Services Data Centre Server Storage Infrastructure Hosted Contact Centre Services Status Awarded / Implementation Awarded / Implementation Awarded RFP RFP RFP 12

13 Summary: File Review (Continued) The CPP has evolved and adapted across projects. There is variability in how CPPs are designed and managed, which reflects the flexibility and adaptability of the procurement process in relation to vendor feedback and project requirements over time. There is also variability in bid assessment/evaluation approaches (e.g. 80/20 versus 10/90 split between financial and technical criteria across CPP projects). Evaluation criteria and their weighting methodology in CPP projects are very complex. 13

14 Summary: Vendor Feedback on File Overall, vendors felt that CPP projects followed procurement steps. Some vendors noted that one-on-one meetings were effective. SSC can make improvements in notifying vendors of changes made to documents, as well as in making announcements to vendors in a timely manner. Most vendors expressed concerns about the heavy weighting on the financial components of the evaluation criteria, versus the technical components. They also had concerns about the financial evaluation methodology itself. Some vendors noted that the process was long and costly. However, they also commented on the insufficient time allocated, at certain stages in the process, to provide feedback to SSC and prepare proposals. 14

15 Summary: Literature Review Early engagement with vendors provides significant opportunities for collaboration and supports value-for-money procurements. Complex and unique solution procurements benefit from weighting technical components over financial components and an initial test to proof solution and cost. A fixed-price or price-cap approach tends to favour value for money and provides vendors with opportunities for innovation. Parcelling of large procurements into integrated components allows for more effective risk management. In this approach, the use of a contract manager has demonstrated positive outcomes. Open and flexible procurement solutions are required to support innovation and SMEs. Clear governance, ownership and accountability of procurement processes and reporting on value achieved ensure a fair, open and transparent procurement. 15

16 Conclusions: Do you agree? Overall, the CPP is viewed as a step in the right direction to promote effective collaboration with industry. The process is seen as open and inclusive. Enhancing reciprocity and communication in the relationship was identified by ITIR members and vendors as an area for improvement. The CPP s potential was perceived as not being fully realized in past CPP projects because the collaborative aspect in defining RFP requirements with the prospective vendors was, ultimately, secondary to the role played by the financial assessments of the bids. Among the suggestions for strengthening procurement capacity were: participation in industry and innovation forums, leveraging external capacity, supporting innovation, including a proof-of-solution stage in the procurement process, and focusing on best value. 16

17 Appendix: Review Methodology Methodologies employed by the OAE: Interviews with past and current, internal and external ITIR members (n=11); File review of SSC projects that have used CPP* (n=6); Analysis of vendor feedback provided in post-award debriefs and post-rrr surveys that were available on file (n=7); Literature review to identify best practices in procurement. The OAE did not conduct interviews with the CPP projects technical and procurement authorities. Meetings with procurement staff were held to validate file-review observations. *Projects that had advanced the furthest through CPP phases were selected. 17

18 Review Limitations and Mitigation Strategies Limitation CPP is a new and continually evolving process. The small number of CPP projects and the uniqueness of each CPP made comparison and generalization across projects difficult. Each ITIR member represents a different group of the ICT sector and has a different experience with CPP, bringing a unique perspective. Some ITIR members shared not only personal opinions, but also feedback received from vendors or industry associations. Some vendors chose not to provide feedback to SSC. Some vendor feedback available on file was consolidated by SSC Procurement rather than documented verbatim. Mitigation Strategy Results were analyzed for highlevel themes, which were corroborated across all methodologies for consistency. Comments were grouped into broad themes (e.g. vendor engagement, innovation) and were reported as summaries. Secondary references (i.e. vendor opinion) were identified. All vendor feedback available on file was incorporated into analysis. The OAE may conduct a vendor survey to update the findings of this review in the future. 18