A Vision for the U.S. Credentialing Marketplace December 1, #CTIcred Follow CTI on Twitter and Facebook

Transcription

1 A Vision for the U.S. Credentialing Marketplace December 1, 2015 #CTIcred Follow CTI on Twitter and Facebook

2 CTI Project Leadership CTI is led by: With support from: Executive Committee provides strategic direction: American Association of Community Colleges (AACC) American Council on Education (ACE) (Co-chair) Business Roundtable Committee for Economic Development of the Conference Board The Manufacturing Institute University Professional and Continuing Education Association (UPCEA) U.S. Chamber of Commerce Foundation (Co-chair) Slide 2

3 CTI Vision & Goals: A Coherent, Transparent Credentialing Market The goals are transparency, clarity, and to align credentials with the needs of students, job seekers, workers, and employers: 1. Develop common terms for describing key features of: Credentials and credentialing organizations Quality Assurance (QA) bodies that accredit, endorse, or approve them 2. Create a voluntary, web-based registry and test it with roughly 100 pilot-site partners 3. Develop and test software apps that facilitate use of the registry by guidance counselors, colleges, employers, etc. Slide 3

4 Credential Transparency and Portability Credential Name and Version Credential Type Credentialing Organization Name Credentialing Organization Type Competencies/Learning Outcomes Required to Attain the Credential Credentialing Requirements Role in Occupational Regulation and Licensing Type and Scope of Primary Application Labor Market Value Education and Training Opportunities Credential Costs Credential Transfer Value Education and Career Pathway Connections Establishing a Common Language: Credential & QA Descriptors Credential Trust and Quality Competency/Learning Outcome and Assessment Development Assessments Required to Attain the Credential Credential Holder Authentication Version Management and Control Quality Assurance QA Descriptors Purpose of the QA Body Scope of the QA Body Quality Criteria Framework for Evaluation Outcomes Methodologies Validity and Reliability of the QA Process External Recognition Information on Decisions Publicly-Available Information Managing Conflicts of Interest Complaint Process Appeal Process Slide 4

5 Credential Registry & Applications Decentralized, open registry uses Web 3.0 technologies to access & aggregate data Credential Registry Degrees, certificates, industry certifications, occupational licenses, microcredentials Credential Directory Employer Signaling & Talent Pipeline Management Credential Transfer Value Competency Authoring Software Applications enable users to search for and view data according to their criteria Slide 5

6 Credential Registry & Applications Registry & Apps Description Design, Develop, & Pilot Planning Credential Registry Open metadata repository utilized by the apps X Credential Directory App Enables the search and comparison of credential information, including related competencies and learning outcomes X Credentialing Value Transfer App Enables the review and analysis of "competency-based" resumes, transcripts, and portfolios based on transfer policies and recommendations to determine and communicate the potential and actual transfer of value between all types of credentials within the marketplace X Competency Authoring App Aids colleges, certification organizations, and other stakeholders in developing more transparent and assessable competency and learning outcome statements based on employer requirements and leading competency frameworks X Employer Signaling & Talent Pipeline Management App Helps employers more clearly signal or communicate their competency and credentialing requirements X Slide 6

7 Credential Directory Mock-up This is mockup to show examples of Credential Registry app search options. Slide 7

8 Credential Directory Mock-up This is mock-up to show an example of Credential Registry app s credential search. 8

9 Credential Directory App This is an example of Credential Registry app search options. Search Options Example Slide 9

10 Credential Directory App Search Results Example This is an example of search results after selecting some search options. Slide 10

11 Credential Directory App Credential Information Example This is an example of credential information after selecting a credential from the search results. Slide 11

12 This is an example of the credential directory app embedded in a regional manufacturing website. Credential Directory App Quality Assurance Linkage Example Slide 12

13 Regional Manufacturing Website This is an example of the credential directory app embedded in a regional manufacturing website. Customized Example Slide 13

14 State Licensing Agency Customized Example This is an example of the credential directory app embedded in a state licensing agency website. Slide 14

15 State Board of Higher Education This is an example of the credential directory app embedded in a state board of higher education website. Customized Example Slide 15

16 Pilot-Site Partners Slide 16

17 Pilot-Site Partners CTI is engaging a balanced, broadly representative sample of credentialing organizations to serve as pilot-site partners to pilot-test the registry by posting info on their websites in a common language: private 4-year institutions, public 4-year colleges and universities, community colleges, and other 2-year schools organizations that award industry or professional certifications organizations that grant industry certificates organizations that issue licenses, apprenticeships, and badges quality assurance organizations Credentials in the following areas of concentration: Manufacturing, health care, information technology, & liberal arts Slide 17

18 Pilot-Site Partners Receive Services Conversion of information in the credential and QA domains to machine and human readable/comparable formats Credential and QA metadata published to the Credential Registry Use of the Credential Directory App Personalized Roadmap document showing where they are now and what steps can be taken to enable the W3C vision for linked data How Pilot-Site Partners Participate Pilot-Site Partners Provide access to technical and program staff Provide access to credential and QA domain information and data Participate with scheduled meetings throughout the project period Provide ongoing feedback Participate in evaluation Slide 18

19 How Will Information Be Gathered for the Registry? Slide 19

20 Gathering Info for the Registry The CTI Technical Advisory Committee (TAC) is developing CTI Description Language (CTI-DL), including definitions and coding schema for the descriptors, in consultation with experts and pilot-site partners CTI staff will use a Pilot-Site Partner Questionnaire to identify and review info for each descriptor based on CTI-DL The questionnaire has three main uses: 1. Introduce and explain CTI-DL definitions and coding schema 2. Identify the availability and location of information for each descriptor 3. Explore data formats (e.g., Excel, PDF) for the information and how this data can be accessed/provided to CTI staff for conversion into Linked Data formats The questionnaire is organized by three main categories that address what is being declared to the public on each credential: 1. Name and Type of Credential and Credentialing Organization the name and type of credential that is being declared to the public by the credentialing organization 2. What Is Inside the Credential the declared competencies, proficiency levels, assessments, evidence of attainment, and connection to competency frameworks 3. Other Key Characteristics and Relationships the other key characteristics and relationships being declared for the credential and the credentialing organization, such as scope of primary application and whether the credential is accredited, endorsed, or approved by an external quality assurance entity Slide 20

21 Questionnaire: Credential Information Slide 21

22 Descriptor Information: Location and Formats Tell the CTI Tech Team about the location and format of your credential information... Current Locations of Credential Information One or more web pages Multiple websites Desktop computers Database Learning Management System Content Management System Other internal or external system Other Current Format of Credential Information Database PDF Text spreadsheet html image metadata Other Slide 22

23 CTI Collaborative and Committees Slide 23

24 Credential Transparency Initiative Collaborative Executive Committee Stakeholder Advisory Council Evaluation Committee Technical Advisory Committee Credential Registry Pilot-Site Partners Credentialing Organization Descriptor Subcommittee Quality Assurance Descriptor Subcommittee Credential Directory Application Subcommittee Competency Authoring Application Subcommittee Credential Transfer Value App Subcommittee Employer Signaling & Talent Pipeline Management App Slide 24 Subcommittee

25 Collaborative Committee Roles Executive Committee Provides strategic direction for the CTI Holds decision-making authority Taking into consideration input gathered from the Evaluation Committee, determines whether the project goes to scale Stakeholder Advisory Council Composed of a broad array of stakeholder organizations Provides key stakeholder feedback on all aspects of the initiative Evaluation Committee Develops the evaluation and data collection plan for evaluating the value and effectiveness of the registry and its applications for major stakeholders Technical Advisory Develops the technical design of the registry Committee Advises on implementation and improvements during the project period. Credentialing Organization Descriptor Subcommittee Develops the final definitions and schema for each descriptor for credentialing organizations QA Descriptor Subcommittee Develops the final definitions and schema for each descriptor for QA organizations Credential Directory Application Subcommittee Employer Signaling & Talent Pipeline Management Application Subcommittee Competency Authoring Application Subcommittee Credential Transfer Value Subcommittee Develops technical design documents for the Credential Directory Application advises on how to provide an API for others to experiment with other credential guidance applications (e.g., WIOA Eligible Provider Lists, State Licensing Directories) Develops technical design documents and use cases for the Talent Pipeline Management and Employer Signaling Application Develops technical design docs for the Competency Authoring app Develops technical design docs and use cases for the Credential Transfer Value app

26 Evaluation Criteria & Metrics for Success Slide 26

27 Draft Evaluation Framework The CTI evaluation committee is developing criteria and metrics by which to measure the value, effectiveness, and success of the registry and its apps The criteria will help guide any corrective action or adjustments during the pilot, as well as any subsequent efforts to take the registry to full scale Two types of evaluation findings will be conducted: Formative evaluation findings relate to the effectiveness of the processes to achieve the goals and objectives Summative evaluation findings are to determine if the terminal goals and objectives have been met Data-based decisions from the formative evaluation findings will assist in improving the processes as the project progresses The summative evaluation findings will determine if the terminal goals and objectives for the project have been sufficiently met to move this project to a national scale Both qualitative and quantitative data will be used to make final decisions Slide 27

28 Formative Evaluation Processes Data Collected Metrics for Success Develop CTI Description Language--Descriptor Definitions and Schema (Six Step Work Plan) Development process data collection from CTI microsite Technical committee member survey Pilot-site partner survey Selected expert review Expected versus actual development time and costs Number and severity of implementation problems TC and pilot-site partners satisfaction Number and impact of stakeholder complaints Stakeholder comments/complaints Expert evaluation results Design and Implement Credential Registry Development process data from CTI microsite and SIU project team data collection Technical committee member survey Selected expert review Expected versus actual development time Costs and number and severity of implementation problems TC member satisfaction Expert evaluation results Slide 28

29 Formative Evaluation, Cont d Processes Data Collected Metrics for Success Identify, Review and Convert Pilot-site Data and Publish on Registry Design and Develop Directory Application Design and Develop Employer App Design Authoring and Transfer Value App SIU data collection and ability to convert pilot-site data and publish it Pilot-site partner feedback survey CTI staff time and project management system data collection including logged comments and complaints CTI committee survey External expert review Directory can query registry as requested Same as above Expected versus actual conversion time and costs Level of difficulty by pilot-site partners to identify data related to descriptors Number and percent of descriptors successfully converted and published Pilot-site partner satisfaction Planned versus actual development time and costs Number and severity of implementation programs CTI committee satisfaction External evaluation results Practical testing of directory to determine it identifies appropriate information as requested Same as above Same as above Same as above Slide 29

30 Summative Evaluation Outcomes Data Collection Metrics for Success Stakeholders (students, career counselors, employers, others) used the Directory Application to search, discover, and compare the full range of credentials, and provides value to the stakeholder for making decisions Registry data system Stakeholder survey Level of utilization and satisfaction Utilization level and success Stakeholder satisfaction with the Directory and its value for decision making Employers used the Employer Application to search and compare credentials again their required competency requirements Registry data system Employer survey Utilization level and success to obtain information requested Employer satisfaction and its business value Given proper written instruction, Credentialing organizations were able to publish and update their information without assistance Registry data system Pilot-site survey Pilot-site satisfaction with ease of process and the quality of the data published Slide 30

31 Summative Evaluation, Cont d Outcomes Data Collection Metrics for Success Credential Registry performed to functional and technical performance requirements Collaborative organization achieve broad-based support, utilization of the Registry and data support for transitioning to non-profit entity Major stakeholders have provided input on business models for taking the pilot to scale Registry data system Registry performance metrics (percent downtime, speed of search, error rate) Collaborative member survey CTI executive committee survey Focus groups to identify a sustaining business model with: pilot-site partners, employers, stakeholder advisory council and technical advisory committee Collaborative satisfaction with pilot Data supports moving to a CTI business model for Independent non-profit Consensus of stakeholder focus group with final decision by the CTI executive committee Slide 31

32 Project Timeline & Other Resources Slide 32

33 Begin Collaborative and Committees. Begin CR & CD Plan/ Design. Begin CR and CD Development. Project Timeline Run Pilot and Evaluation of CR and CD. Minimum Viable Product. Begin CA, ES/TPM and CVT Planning. Abbreviations: Competency Authoring (CA) Credential Directory (CD) Credential Registry (CR) Credential Value Transfer (CVT) Minimal Viable Product (MVP) Quality Assurance (QA) Employer Signaling & Talent Pipeline Management (ES/TPM) Complete CR and CD Pilot and Evaluation. Complete CA, ES/TPM, and CVT Plans Establish and Facilitate Collaborative (Executive Committee, Stakeholder group) and all Sub-Committees (Jul 1, 2015 Dec 2017) Launch Website (Jul 1 Sept 30, 2015) Descriptor and Evaluation Committee Documents (Jul 1 Dec 31, 2015) Maintain Website (Oct 2015 Dec 2017) Implement Evaluation Plan (Jan 2016 Oct 2017) Collect Data on Participation and Use for Evaluation (Jan 1, 2016 Oct 2017) Technical and Design Documents: CR and CD (Aug 15 Dec 15) Recruit Credentialing and QA Pilot Organizations (Jul 1, 2015 Dec 2015) Work with Credentialing and QA Organizations (Jan 1, 2016 Sept 2017) CR & CD Development MVP (Nov 1, 2015 July 2016) Continuously Run and Improve Functioning of CR and CD (Feb 2016 Dec 2017) Plans Developed for CA, ES/TPM, and CVT (Jul 1, 2016 Nov 1, 2017) Final Reports Slide 33

34 Resources CTI Website CTI Technical Planning Microsite Slide 34

35 Questions? Slide 35

36 Today s Committee Meetings Concurrent Committee Meetings (11:30 2:00) 1. Technical Advisory Committee and Credentialing Organization Descriptor Subcommittee 2. QA Descriptor Subcommittee 3. Credential Directory App Subcommittee 4. Employer Signaling & Talent Pipeline Management App Subcommittee Slide 36

37 Appendix Slide 37

38 Credential Directory App Stakeholders Can Use the App to: Credential Directory App Registry Identify credentials meeting userdefined criteria based on key features of credentials Compile customized lists and descriptions of credential based on search criteria Build customized, branded directories for different users based on their own criteria, without the need for continual, duplicative content development and updating Publish comparable information through public directories or through existing career and education guidance systems Slide 38

39 Employer Signaling & Talent Pipeline Management App Helps Employers: Employer Signaling & Talent Pipeline Management App Registry More clearly signal their competency and credentialing requirements Provide comparable information on competency and credentialing requirements for their jobs, and endorse or recognize credentials for these jobs Process and manage competency and credentialing information in job applicant tracking Helps employers source and manage relationships with credentialing organizations and related education and training providers in order to better manage their talent pipeline Helps Credentialing Organizations: In working with employers to validate the required competencies in their credentials Slide 39

40 Credential Transfer Value App Key Functions: Registry Credential Transfer Value App Designed to support the easy review and analysis of competency-based resumes, transcripts, and portfolios Provides transfer value profiles of credentials based on new measures of declared and actual transfer value from multiple credentialing organizations Provides transfer value directories and comparisons based on search criteria Provides comparisons between recommended and actual transfer for recommendation services and credential transfer consortia Slide 40

41 Competency Authoring App Key Functions: Registry Competency Authoring App Designed to aid colleges, certification organizations, and others in developing more transparent and assessable competency statements Provides authoring tools for writing and communicating competency statements, including grammar guides, feedback systems (e.g., rating power of competency statements), and competency frameworks and libraries Provides publishing tools for publishing competency statements on websites and registry Slide 41

42 For more information Steve Crawford Research Professor, GWIPP Bob Sheets Research Professor, GWIPP Roy Swift Executive Director, Workcred Roberts T. Jones Former Assist Secretary, U.S. Department of Labor Jeanne Kitchens Associate Director, SIU Center for Workforce Development Follow CTI: Twitter: twitter.com/cti_credential Facebook: facebook.com/credtransparency Slide 42