Online Competency Assessment (OCA) Project

Transcription

1 Board Direction Motion 5533 (May 21, 2015) THAT a monitoring report be prepared to demonstrate how creation of a new online competency assessment system by Engineers Canada will support constituent associations in advancing the engineering profession and its selfregulation in the public interest at a cost that is justified by the results. The monitoring report will provide details including, but not limited to: the terms and conditions of the funding agreement with the Federal Government; the use of the online system by CA's as agreed to with the Federal Government; demonstration that the online system with achieve the goals of the Federal Government as announced April 13, 2015; the plan for updating the program upon completion; and a demonstration that the work does not conflict with any of: o o o EL 2 which shall not cause or allow conditions, procedures or decisions which impede productive relations with constituent associations: EL 5 which shall not permit planning that allocates resources in a way that deviates materially from Ends priorities; and, EL 7 which shall not permit the Board to be uninformed or unsupported in its work. Author Kim Allen Date Submitted September 30, 2015 Policies requested EL-2 The chief executive officer shall not cause or allow conditions, procedures or decisions which impede productive relations with constituent associations. EL-5 EL-7 The chief executive officer shall not permit planning that allocates resources in a way that deviates materially from Ends priorities, that risks fiscal jeopardy, or that is not derived from a multi-year plan. The chief executive officer shall not permit the Board to be uninformed or unsupported in its work. Other relevant policies 1 of 12

2 This report is presented in four sections to address the specific requests and each of the policies. The project is in progress, so evidence of outcomes or actual expenditures are not available. The report therefore focuses on the planned project activities, schedule and budget. Policy interpretations in this report refer solely to aspects of the. Background: The history of the competency-based assessment project started in 2008 with Alternative Methods of Licensure, a project to investigate recommendation #9 of the From Consideration to Integration project which called for Engineers Canada to study the feasibility of alternative system of evaluating an applicant s professional competency for licensure in comparison to the existing system. This recommendation arose from studying the experiences of engineering immigrants and a recognition that it was necessary to improve the licensure process and outcomes for qualified international engineering graduates (IEGs). At the time, IEGs found the licensure process to be frustrating, time-consuming, bureaucratic, complicated, inconsistent, difficult to navigate, and in some cases unfair. Through Alternative Methods of Licensure, Engineers Canada studied the licensure requirements in each jurisdiction, investigated licensure models in other countries, and concluded that moving to a competency-based assessment for the engineering work experience portion of licensure would provide the most value in terms of simplifying, clarifying and making more consistent the process by which engineers are licensed in Canada and in terms of improving the integration of foreign trained professionals into the Canadian engineering profession. The project concluded by recommending seven core competencies that all engineers would be required to demonstrate through their engineering work experience. It also recommended that a future project develop a competency assessment tool; develop training for staff, volunteers and applicants; and conduct pilot projects. These recommendations were endorsed by the Engineers Canada Board on May 28, 2010 via motion #5120. The Competency-Based Assessment of Engineering Work Experience Project began in 2010 to implement those recommendations. It successfully developed the required tools and conducted pilot projects in Ontario and Saskatchewan which led to further refinements. The project resulted in three key recommendations for the Board. 1. Adopt the proposed competencies as the definition of the work experience requirement for licensure as a professional engineer. 2. Endorse the proposed competency-based assessment process as the baseline for national harmonization of the work experience requirement. 3. Support the progressive implementation of the proposed solution by Engineers Canada to: Coordinate the design, development and deployment of an online system. Create a national steering committee to provide oversight for the development of the online tool and implementation by associations Assist each association through the adoption phase 2 of 12

3 All three recommendations were endorsed by the Engineers Canada Board on March 1, 2013 via motion #5320. Since that time, Engineers Canada has been working to secure funding from the federal government (Employment and Skills Development Canada ESDC). When ESDC indicated that they would not fund the entire project, since it benefited both foreign and domestic applicants, the Engineers Canada Board endorsed the use of $1.0 M from reserve funds for our portion of the costs. This decision was made at the May 23, 2014 Engineers Canada Board meeting via motion #5441. On April 1, 2015 ESDC informed Engineers Canada that they were willing to fund 50% of the project. A federal government press release on April 13, 2015 stated, in part: The Honourable Pierre Poilievre, Minister of Employment and Social Development, released a report today outlining new and better ways to integrate immigrants into the Canadian workforce. The Minister shared highlights from the report, authored by the Panel on Employment Challenges of New Canadians, during his speech at The Conference Board of Canada s Canadian Immigration Summit 2015, which kicked off in Ottawa today. The Minister also announced funding for two related projects that will see internationally trained doctors and engineers have their foreign credentials more quickly and effectively recognized by eliminating red tape and taking advantage of new online tools. The two projects, one led by the Medical Council of Canada and the other by Engineers Canada, will help address some of the challenges noted by the Panel that newcomers face when trying to obtain employment. These challenges include problems getting foreign qualifications recognized, a lack of Canadian work experience, inadequate pre-arrival information and a mismatch of skills to region. For additional information on the project please also see SECTION I Requests from the Board Provide details including but not limited to: the terms and conditions of the funding agreement with the Federal Government; the use of the online system by constituent associations as agreed to with the Federal Government; demonstration that the online system with achieve the goals of the Federal Government as announced April 13, 2015; the plan for updating the program upon completion; Chief Executive Officer s Interpretation, in the context of the OCA project The press release of April 2013, 2015 did not explicitly state the goals of the Federal Government. Based on the press release quoted above (the full text is available here), those goals are interpreted as: 1. Foreign credentials recognized more quickly and effectively 2. Online tools are implemented so that assessments can begin from anywhere in the world Justification of Reasonableness The press release makes reference to the recommendations from the report of the Panel on Employment 3 of 12

4 Challenges of New Canadians, but only states that the Minister will study them, not that they are goals of the Federal Government s. In the two paragraphs that speak directly to the funding of the project, only the online tool and the recognition of foreign credentials are items that are also included in the agreed-to scope of deliverables for the Online Competency Assessment project description that is part of the funding agreement with ESDC. Requirement: Terms and conditions of the funding agreement with the Federal Government Evidence: The federal government has agreed to fund 50% of the project costs over the 24 month project duration. The project objectives are to: - develop and implement an online tool necessary for the competency-based assessment of engineering work experience for professional engineering licensure. This tool will provide orientation to applicants and assessors, and incorporate an on-demand help capability at all critical points in the workflow; - develop and implement an online assessor training tool, with a learning management system critical to improving consistency and objectivity of licensure decisions. Engineers Canada must provide quarterly progress reports to ESDC including achievement of the desired outcomes: a fully standardized and automated process that is fair, and transparent, and standard in assessing the work experience of foreign-trained engineers The achievement of these outcomes will be measured through qualitative surveys of applicants, assessors and staff; and via quantitative analysis of assessments. Use of the online system by the constituent associations as agreed to with the Federal Government The project has committed to conduct pilots with two to three constituent associations of different sizes. While the budget and schedule allow for three final implementations with one to two constituent associations per implementation (i.e. between three and six CAs), there is no commitment made as to the final number of constituent associations who will use the system. Achievement of the goals of the Federal Government as announced on April 13, Development of an online tool are all online tools (assessment, training, on-line support) developed and 4 of 12

5 piloted? 2. Quicker assessments are foreign-trained applicants able to self-navigate, understand the requirements, and submit the necessary documentation in less time than they currently are? Are the constituent associations able to make decisions about the work experience portion of licensure in less time than they currently are, considering all staff and volunteer time? 3. More effective assessments is the new system better in terms of clarity, consistency and transparency? The goals will be measured through qualitative and quantitative surveys of applicants, assesors and constituent association staff. The plan for updating the program upon completion The plan for updating the program is contingent on who provides the IT solution, where it is hosted, how much hosting and maintenance cost, and the terms of the service agreements which are established with the participating constituent associations. All of these items are currently unknown, although the decision related to each one is already included as a planned project activity. Once all of the items are established, the final plan can be developed and agreed to with all constituent associations. 5 of 12

6 SECTION II EL-2 Treatment of Constituent Associations EL-2 The chief executive officer shall not cause or allow conditions, procedures or decisions which impede productive relations with constituent associations. Further, without limiting the scope of the above statement by the following list, the chief executive officer shall not: 1. Permit constituent associations to be without easy access to clear information about the privileges and responsibilities of membership. Not applicable to OCA Project. 2. Use methods of collecting, reviewing, storing or transmitting constituent association information that fail to protect against improper access to the information elicited. 3. Operate without consideration of constituent association viewpoints regarding operational means which directly impact the constituent associations. 4. Operate without a process to ensure that constituent association comments and complaints are responded to fairly, consistently, respectfully, and in a timely manner. 5. Permit constituent associations to be without timely information regarding assessment fees, or change in assessment fees. Not applicable to OCA Project. 6. Neglect to advise constituent associations in a timely manner about issues that Engineers Canada is aware of which may impact the profession and/or constituent associations. 7. Neglect to facilitate collaborative development of shared resources, including those developed by the constituent associations to achieve the Ends. Chief Executive Officer Interpretation, in the context of the OCA project EL-2 The OCA project must work with all constituent associations, and use effective communication, project management and change management techniques to ensure productive relationships with all constituent associations. EL-2.2 The online tools developed through OCA must respect privacy laws and ensure data security and integrity. EL-2.3 The OCA project must understand the requirements and limitations of each constituent association and work to develop customizations which respect those constraints. EL-2.4 The OCA project must work collaboratively with all interested constituent associations and develop tools which meet their needs and respect the goals of greater consistency, clarity and objectivity. EL-2.6 The OCA project must keep all constituent associations informed of progress and of new requirements (if any) raised by the Federal Government. EL-2.7 The OCA project must make use of existing systems and tools that have been developed by the constituent associations, where possible. Justification of Reasonableness Like all Engineers Canada initiatives, programs and tools, the OCA project must provide value for the constituent associations. The only way to do this is to maintain open communication, and develop the final product 6 of 12

7 collaboratively to make sure that it meets the needs of each constituent association while respecting the overall goals of online competency assessment project. Compliance will be achieved when: The OCA project must work with all constituent associations, and use effective communication, project management and change management techniques to ensure productive relationships with all constituent associations Evidence of compliance: The project activity breakdown includes change management and communications as an ongoing aspect of the project, as follows: Change management and communications strategy (June 2015 April 2017) - Design and integrate change management activities into the master project plan - Distribute Case for Change document to facilitate decision making at all levels - Maintain pro-active communications to the associations over the entire lifecycle of the project (e.g. project reports, webinars and demonstrations) - Identify and empower change agents within each association to facilitate deployment of the solution within their respective associations - Solicit feedback from the associations at all stages and respond appropriately The online tools developed through OCA must respect privacy laws and ensure data security and integrity The project activity breakdown includes analysis of constituent association privacy and security requirements and development of software design requirements to match, as follows: Analyze existing processes and IT Infrastructure of the 12 constituent associations (July 2015 Oct 2015) - Identify privacy and security concerns and requirements Analyse software design requirements (Jan 2016 Feb 2016) - Determine detailed requirements for applicant orientation - Develop application tracking requirements The OCA project must understand the requirements and limitations of each constituent association and work to develop customizations which respect those constraints The project activity breakdown includes several activities to understand constituent association needs, as follows: Analyze existing processes and IT Infrastructure of the 12 7 of 12

8 constituent associations (July 2015 Oct 2015) - Establish liaison with IT staff of the constituent associations - Identify workflow and document management required for tracking applicant submissions and decision making by each association. - Identify web architecture requirements and standards - Understand interface requirements to integrate with the existing databases of the associations Assess and obtain concurrence on strategy for limited licences (Oct 2015) - Seek agreement on the scope of limited licences within the competency framework - Seek agreement on a standard format for the scope definition of limited licences The OCA project must work collaboratively with all interested constituent associations and develop tools which meet their needs and respect the goals of greater consistency, clarity and objectivity The project activity breakdown includes several activities to work collaboratively with the constituent association, as follows: Readiness analysis (June 2015 Oct 2015) - Assess the implementation readiness of constituent associations to facilitate setting deployment targets - Identify key issues and address within the communications strategy Analyze existing processes and IT Infrastructure of the 12 constituent associations (July 2015 Oct 2015) - Establish liaison with IT staff of the constituent associations Assess and obtain concurrence on strategy for limited licences (Oct 2015) - Seek agreement on the scope of limited licences within the competency framework - Seek agreement on a standard format for the scope definition of limited licences Develop concept prototype and demonstrate nationally (Mar 2016 June 2016) - Develop a proof-of-concept prototype based on analysis of all design requirements 8 of 12

9 - Demonstrate the prototype nationally to reinforce buy-in of the associations and further refine or validate the design requirements - Update the design requirements based on the feedback received from the demonstration of the prototype Implement assessor recruitment and certification program (Aug 2016 Dec 2016) - Liaise with constituent associations to facilitate recruitment of assessors Implement adoption strategy (Dec 2016 Jan 2017) - In accordance with change management strategy finalize pilot locations for first deployments - Draft service standard agreements between Engineers Canada and the associations - Conduct pro-active deployment planning meetings with associations - Provide support to associations during deployment for policy, technical and business process changes Deploy pilot solution to select associations (Feb 2017 Apr 2017) - Monitor operationalization of pilots - Refine best practices and document lessons learned Progressively deploy the software solution to the next wave of associations (Apr 2017 June 2017) - Use the "Ready-Set-Go" process to deploy the solution to additional associations - Facilitate decision making and business process change - Facilitate technical support between association IT staff and the vendor as appropriate - Facilitate internal communications by supplying associations with communications tools Validations of operations and service standards model (May 2017 June 2017) - Support the associations during the first 60 to 90 days of initial operations - Concur on Service Standard agreements 9 of 12

10 - Gather feedback and refine the solution The OCA project must keep all constituent associations informed of progress and of new requirements (if any) raised by the Federal Government The project activity breakdown includes ongoing communications that will allow for progress reporting and to address any changes that may arise, as follows: Change management and communications strategy (June 2015 April 2017) - Maintain pro-active communications to the associations over the entire lifecycle of the project (e.g. project reports, webinars and demonstrations) The OCA project must make use of existing systems and tools that have been developed by the constituent associations, where possible Since the original project plan was developed in 2013, APEGBC has successfully launched their competency-based assessment tool for all applicants for engineering licensure. It appears that this platform will be able to support the Engineers Canada competency assessment model which differs from the model used at APEGBC. APEGBC is therefore the preferred vendor for the provision of the competency assessment tool. The ESDC funding agreement still requires a competitive bid process, however. Engineers Canada has held three teleconferences with APEGBC (May 11, May 21, and July 14, 2015) to discuss the project. Engineers Canada informed ESDC on June 12, 2015 that APEGBC is a potential supplier for this portion of the IT solution. 10 of 12

11 SECTION III EL-5 Planning EL-5 The chief executive officer shall not permit planning that allocates resources in a way that deviates materially from Ends priorities, that risks fiscal jeopardy, or that is not derived from a multiyear plan. Chief Executive Officer Interpretation, in the context of the OCA project The OCA project must establish a budget, and respect that budget. Deviations to the budget must be tracked and reported to ESDC and the Board. Justification of Reasonableness The OCA project must follow project management principles and track budget. Compliance will be achieved when: The OCA project has a budget which is tracked and reported to ESDC and the Board. Evidence of compliance: A high level project budget has been established and agreed to with ESDC. The project budget will continue to be refined as the project progresses and costs are known. Quarterly status reports to ESDC include progress, budget and schedule updates. These updates will be provided to the Board. Deviations to the budget will be reported to ESDC and to the Board. 11 of 12

12 SECTION IV EL-7 Communication and Support to the Board EL-7 The chief executive officer shall not permit the Board to be uninformed or unsupported in its work. Chief Executive Officer Interpretation, in the context of the OCA project The Board must be informed of the project progress. Justification of Reasonableness The Board must be informed of the progress of the project, and any changes in direction or objectives. Compliance will be achieved when: The Board will be informed of the project progress The Board receives copies of the quarterly status reports sent to the Federal Government Evidence of compliance: Updates on the project progress will be included in every Board update. Quarterly status reports will be submitted to the Board. Resources Engineers Canada / ESDC funding agreement (attached) Adobe Acrobat Document OCA project activities and schedule (attached) Adobe Acrobat Document OCA project budget for portion funded by ESDC (attached) Adobe Acrobat Document Certification I certify that the above information is accurate. Kim Allen, FEC, P.Eng., July 21, of 12