Enhancing Performance for Youth. Workshop Goals. Develop an understanding of the performance measures.

Transcription

1 Enhancing Performance for Youth 1 Workshop Goals Develop an understanding of the performance measures. Explore strategies to enhance youth performance. 2 Performance measures are important but they are not the bottom line What is best for the customer is what is important provide services to those who need it studies have proven we are not good predictors of participant outcomes Data shows that good services to participants will result in good outcomes 3 1

2 Key Definition: Participant USDOL Definition TEGL Participant is an individual who is determined eligible to participate in the program and receives a service funded by the program in either a physical location or remotely through electronic technologies 4 Key Definition: Participation The date of participation is actual start date of the first service on the Services page of OWCMS The date of the first youth service will be the youth participation date The date of eligibility and the date of participation do not have to be the same If an individual is participating in multiple funding streams they will be included in the performance measures for those funding streams 5 Key Definition: Participation (Youth) Youth Framework Activities will be the first required service for all Youth that will start their participation. (WIATL 33) If a Youth is required to be a part of the Youth Literacy/Numeracy Performance Measure their twelve month time period to increase their EFL will be based on the first actual start date of their first youth service (Date of first Youth Service) 6 2

3 7 Key Definition: EXIT Customers who do not receive a WIA funded or WIA partner service for 90 days and are not scheduled for future services except follow-up. Planned gaps in service over 90 days will not result in a soft exit (Will talk more about that later). The exit date is the date of last service. Case manager does not push a button to make an exit occur To Exit - Insert Appropriate Actual End Dates on Services screen, Close the case, and OWCMS will automatically Exit after 90 days. Highly recommend closing youth cases so local area can record school status at exit. OWCMS will presume school status at participation if not entered by local staff. This will also give you access to Post Exit and Wages and Youth 8 Follow-up screens 9 3

4 10 11 When Performance Starts Two of the three Youth performance measures are exit based Performance is not tracked till after an individual exits The exception is the Literacy/Numeracy Rate 12 4

5 Planned Gaps in Service Youth can be put on hold when they have a legitimate reason for not participating for a period of time, but plan to return. These holds are for: Health/medical reasons for the individual Health/medical reasons for a family member Temporary move from the area or When a service or training is not immediately available. Documenting a hold can prevent a soft exit. 13 Planned Gaps in Service A gap in service must be related to one of the circumstances previously mentioned and last no more than 180 days from the date of the most recent service to allow time to address the barriers to continued participation. Grantees may initiate a consecutive gap in service of up to an additional 180 days for the participant that follows the initial 180 day period to resolve ongoing issues The result is a total of 360 days in a planned gap in services

6 Global Exclusions Exclusions from all Performance Measures: Institutionalized/incarcerated at exit Health/medical exits Health/Medical for family member Reservists called to active duty who choose not to return to WIA Relocated to a Mandated Residential Program (Youth Only) Deceased Not a Valid SSN 16 Global Exclusions To be excluded from performance calculations, the condition(s) must be expected to last 90 days or more These reasons for excluding a participant from common measures calculations can be applied: at the time of the condition occurs or during the three-quarter measurement period following the exit quarter

7 Actual Start Date will be Participation Date. Seeker Exit Date is the Exit Date. Youth School Status At Exit impacts Youth Certificate Measure Seeker Exit Reason will indicate individual is to be excluded from performance measures if the exit reason is not soft exit. 19 Key Definition: QUARTERS Calendar quarters January to March April to June July to September October to December Keep in Mind: The exit quarter is the quarter containing the exit date. Quarters after exit (i.e. 1 st or 3 rd quarter) mean quarters after the exit quarter. 20 Key Definition: QUARTERS QUARTER QUARTER QUARTER QUARTER 21 7

8 Exit Quarter and Quarters after Exit Participant Exits on 2/11/2014 Exit Quarter Jan March st Quarter after exit April June nd Quarter after exit July-September rd Quarter after exit October-December th Quarter after exit January-March 2015 Unemployment Insurance (UI) wage data: UI wage data is required to be used to track employment and wages in the Placement in Employment and Education measure. NOTE: Gathering UI wage data is a State responsibility. 23 Unemployment Insurance (UI) wage data: Challenges Typically there is a two quarter lag before this data can be analyzed by the state for WIA performance. This means data verifying employment and wages for performance measures is not reported for many months after the performance period. This causes performance reporting to be delayed for most of the performance measures States can use Supplemental Data to verify employment for these workers. Not all employed workers show up in UI wage records. States can use Supplemental Data to verify employment for these workers. 24 8

9 Represent Quarters after Exit Employment Status after Exit is displayed here 25 Waiver for Common Measures Ohio has been approved for a waiver for the WIA program to operate under the Common Measures performance standards Prior to waiver Ohio had to report on 10 youth performance measures Due to the waiver we now only report on the 3 Common Measures All other statutory measures will not will not be reported to USDOL for the WIA program 26 Only Youth Programs There is no longer a reason to distinguish between Older and Younger Youth Programs The real distinction that needs to be considered is In-School and Out-of- School All Youth could be a part of all three of the new measures or a part of none 27 9

10 New Common Measures could result in longer enrollment for Youth Outcomes for two of the measures could result in multiple years of enrollment Have to be employed or in postsecondary education Have to receive a diploma, degree, or certificate Employment or enrollment in postsecondary education could take a long time 28 Performance Measures Question: What WIA performance measures are calculated for Youth? 29 Youth Common Measures Will include all youth, no distinction between younger/older 1) Placement in Employment, Education, or Training 2) Attainment of a Degree/Certificate 3) Literacy/Numeracy Gains 30 10

11 Youth Placement in Employment or Education (USDOL Definition TEGL 17-05) Of those who are not in post-secondary education, employment, or the military at the date of participation: The number of participants who are in employment or the military or enrolled in post-secondary education and/or advanced training/occupational skills training in the first quarter after the exit quarter the number of participants who exit during the quarter Youth Placement in Employment or Education Excludes youth in employment, the military, or postsecondary education at participation Employment, military and education status at participation are based on information from the individual Employment and military service in the 1st quarter after exit may be based on wage record data or supplemental data Youth who exit October 2013 September 2014 will be included in this measure for PY 2014 Successfully returning young people to school or alternative education is desirable, but secondary school enrollment does not qualify as a placement under this measure

12 Education status is used to determine whether a youth is in postsecondary status at participation

13 Key Factors Contributing to a Positive Outcome: Service/program completion Direct job placement Part-time employment as part of program design Employment or Post-Secondary Education Stability 37 YOUTH PLACEMENT IN EMPLOYMENT OR EDUCATION Matt was a 19 year old who was not employed or going to school at date of participation. He received Occupational Skills training with Youth funding. He became employed and exited in June He continued to be employed on a permanent basis for many years. Ray was a 17 year old student who participated in the Youth program working on his basic skills. He participated for a year and a half but dropped out of the program but was still in secondary school at exit. Leroy was a 21 year old going to college at participation. He wanted help in getting a job and received an internship with help from the local WIA program with Youth funding. He became employed and exited in August He continued to be employed on a permanent basis for many years. 38 Youth Attainment of a Degree or Certificate (USDOL Definition TEGL 17-05) Of those enrolled in education at the date of participation or at any point during the program: The number of participants who attain a diploma, GED, or certificate by the end of the 3rd quarter after the exit quarter the number of participants who exit during the quarter

14 40 Youth Attainment of a Degree or Certificate Education refers to secondary school, postsecondary school, adult education, or any organized program of study Organized program of study must lead to a degree or certificate Includes youth in secondary school at exit Outcome can be attained during participation Diploma is any credential the State Education Agency accepts as equivalent to a high school diploma Youth who exit October 2013 September 2014 will be included in this measure for PY Enrolled in Education An individual is considered Enrolled in Education for the measure from 3 fields in OWCMS: Basic Intake- Additional tab-education Status (enrolled in Secondary, Alternative, or Post- Secondary Education) Services (Tutoring, Occupational Skills Training, Alternative School) Services-Exit tab-youth School Status at Exit 42 14

15

16 Enrolled in Education status is displayed here. If this is yes, the individual is in the Youth Certificate, Diploma, or Degree measure Definition of a Certificate Defining certificate A certificate is awarded in recognition of an individual s attainment of measurable technical or occupational skills (emphasis added) necessary to gain employment or advance within an occupation. These technical or occupational skills are based on standards developed or endorsed by employers. USDOL has clarified that recognition of generic pre-employment and/or work readiness skills are not considered certificates. No exceptions will be made. 47 Definition of a Certificate Awarding Institutions include: A State educational agency Institution of higher education Professional, industry or employer organization or a product manufacturer Registered apprenticeship program Public regulatory agency A program approved by the Dept of Veterans Affairs Office of Job Corps Indian Tribe Higher Education Institution 48 16

17 49 50 Key Factors Contributing Positive Outcome: Youth complete program services. A certificate policy is clearly articulated. Training tied to viable certificates/diploma. Insure certificate is documented before exit. Effective follow up services and tracking are provided: identifying, getting and recording certificates

18 Youth Attainment of Degree/Certificate Lucy was a 19 year old who was not employed or going to school at date of participation. She received Occupational Skills training with Youth funding. She became employed and was exited in June 2014 but continued to go to school. She completed her training and received her Associates Degree in December Shirley was a 17 year old student who participated in the Youth program working on her basic skills. She participated for a year and a half but dropped out of the program but she was still in secondary school at exit. Cindy was a 21 year old going to college at participation. She wanted help in getting a summer job and participated in the Summer Work program with help from the local WIA program with Youth funding. She went back to college and exited in June She received her degree in September Youth Literacy/Numeracy (USDOL Definition TEGL 17-05) Official Definition: Of those out-of-school youth who are basic skills deficient: The number of participants who increase one or more Educational Functioning Levels (EFL) divided by the number of participants who have completed a year in the program (ie., one year from the date of program participation) plus the number of participants who exit before completing a year in the program

19 Out-of-School Out-of-School is defined as: An eligible youth who is a school dropout, or who has received a secondary school diploma or its equivalent but is basic skills deficient, unemployed, or underemployed (WIA section 101(33)). 55 Understanding Out-of-School Youth Simplified Definition 1. Enrolled in Education leading to a certificate In-School (See exception in Number 3) 2. Not attending School - Out-of-School 3. In post-secondary education & basic skills deficient - Out-of-School 56 Post Secondary Education Post-Secondary Education A program at an accredited degree-granting institution that leads to an academic degree (eg. A.A., A.S., B.A., B.S.). If in Post-Secondary education they are not available for work; therefore cannot be considered unemployed or underemployed 57 19

20 IN-SCHOOL In-School, HS or Less; In-School, Alternative School; In-School, Post HS (Basic Skills Proficient) Education Status is primary field used for Out-of-School status for Youth Literacy and Numeracy. If individual has a status of In School: Post HS you must also look at Basic Skill Deficient status OUT-OF- SCHOOL Not Attending School, HS Dropout Not Attending School, Graduate In-School, Post HS (Basic Skills Deficient) 58 The Eligibility Page will reflect Out-of-School Status 59 Basic Skills Deficient All Out-of-School Youth must be assessed in basic reading, writing and math, or language (based on the type of test given) Basic Skills Deficient - The individual computes or solves problems, reads, writes, or speaks English at or below the eighth grade level (8.9 grade level equivalent). If a youth continues to be basic skill deficient in the 2 nd and 3 rd year and don t exit they will be a part of the measure in those years! 60 20

21 Basic Skills Deficient Basic Skills Deficient field on the Basic Intake Program Data screen If the individual has one Pre-test/Baseline test score with a Educational Functional Level (EFL) less than Low Adult Secondary Education/Exit ESL the individual must be marked Basic Skills Deficient on the Basic Intake Screen!

22 Educational Functioning Levels Based on the National Reporting Systems (NRS) EFL Table Educational Functional Levels are based on scores on standardized tests (scale scores for TABE) that are approved by NRS. Educational Functioning Levels a measure of adult educational progress See Attachment C of TEGL Educational Functioning Levels and Grade Level Equivalencies for those who speak English Each level is equal to approximately 2 grade levels Beginning ABE Literacy (0-1.9) Beginning Basic Education (2-3.9) Low Intermediate Basic Education (4-5.9) High Intermediate Basic Education (6-8.9) Low Adult Secondary Education (9-10.9) High Adult Secondary Education ( ) 65 6 Educational Functioning Levels and Grade Level Equivalencies for those who do not speak English Beginning ESL Literacy Low Beginning ESL Literacy High Beginning ESL Literacy Low Intermediate ESL High Intermediate ESL Exit ESL 66 22

23 All tests for Out-of-School Youth Must Crosswalk NRS Test of Adult Basic Education (TABE forms 7,8,9,10) General Assessment of Instructional Needs (GAIN) WorkKeys Reading for Information, Writing, Applied Math for students at EFLs 4,5, or 6 Comprehensive Adult Student Assessment System (CASAS) MAPT - Massachusetts Adult Proficiency Test Basic English Skills Test (BEST) ESOL Only - Basic English Skills Test (BEST) Oral Interview Long Form and BEST Literacy Skills, BEST Plus (computerized) ESOL Only - CASAS Life Skills ESOL Only - TABE CLAS-E All Meet NRS Requirements Testing Requirements Pre-test A test used to assess a participant s basic literacy skills. The pre-test can be administered to a participant up to six months prior to the date of participation, if such pretest scores are available. Baseline Test Test that is used to compare to post-test to see if there is an Educational Functional level gain Post-test A test administered to a participant at regular intervals during the program. Posttest is compared to Pre-test or Baseline test to 68 determine EFL. Testing Requirements Programs must adhere to the following in choosing an assessment tool: The same assessment tool is administered to the participant for pre-testing and post-testing; The assessment tool and its scores must crosswalk directly to the educational functioning levels so that educational gains can be reported in terms of increase in one or more ABE or ESL levels; and Tests must be administered in a standardized manner throughout the jurisdiction (i.e., used consistently and reliably across programs and produce observable results)

24 Testing Requirements Intervals Participants must be post-tested at least once by the end of one year following the individual s date of first youth program service. If more than one assessment is administered after the initial test, the latest assessment will be used Individuals who remain basic skills deficient and continue to participate, must continue to receive basic skills remediation servicesand will remain part of the Literacy and Numeracy measure unless they exit before the end of the 2 nd or 3 rd assessment year. 70 Assessment Year The Assessment Year is based on an individual s Youth Participation Date! It is not based when the date Pretest/Baseline test is given. Participants are given twelve months to achieve a gain of an Educational Function Level based on Youth Participation date. 71 Educational Functioning Levels Educational Gain For Year 1 - Compares pre-test or baseline test and post-test, if the participant advances one or more educational functioning levels 12 months or earlier from the youth participation date For Year 2 and 3 - if the participant advances one or more educational functioning levels comparing the previous year s latest post-test (baseline test) to the current year post-test Individual has to participate for the full second year in the program to be included in the measure in the second year the same applies to the third year 72 24

25 EFL Calculation Success for year one is Educational Functioning Level > than original EFL Level, Date must be before or equal to anniversary of Date of First Youth Service Success for subsequent years In the PRESENT year, the Educational Functioning Level > the EFL for the previous year Date must be before to the anniversary of 73 Date of First Youth Service First Year Notes If a previous assessment was conducted (i.e. by a partner program) within six months prior to Date of First Youth Service, then that test score can be used. A participant is included in the measure even if they exit prior to end of the first year. Post-test can be anytime within the first year, but must be given prior to the end of the 12 month time period from participation date. Failure to test within first year or prior to exit will result in a negative outcome for that year If a participant exits for an exclusion (i.e. global exclusion), the participant is excluded from the measure in that year. It doesn t matter if they were a success or not. 74 Literacy or Numeracy Gain Second & Third Year Is Basic Skills Deficient? YES Has completed another full year based on Anniversary Date*? YES Has increased one EFL within last year based on latest post-test score? NO NO NO YES Excluded Numerator Denominator 75 25

26 Second Year and Third Year Notes Individual is part of this measure if they continue to be basic skills deficient Test can be given anytime between anniversary dates There is only one pre-test regardless of when it is given. The first test given will be considered a pre-test for the functional area. It can be given at any time. However, delay in giving the test will shorten the time period for success as the time period is based on the date of participation and not the date of the test! It is possible to have EFL gains in Second and Third Year even if no test is given in the previous year. For subsequent years in the program, if the participant exits prior to the anniversary of the participation year, 76 then the participant is excluded from the measure whether or not the participant completed the appropriate post-test. Youth Literacy/Numeracy Implications Gerald was 20 years old, Basic Skills Deficient, and Out-of- School when he came in to the program on March 14, His pre-test Educational Functional Level (EFL) for math was High Intermediate Education. He got a job working for UPS full-time and agreed to continue to work on his basic skills and raised his math EFL to Low Adult Secondary on a post-test on October 13, 2014, as the last service before exiting. Leslie was 15 years old, Basic Skills Deficient, and going to Secondary School when she enrolled on October 30, She dropped out of school and the program in January Kathleen was 19 years old, Basic Skills Deficient, and Out-of- School when she came into the program in February 13, Her pre-test EFL was Beginning Basic Education for math was 3.4. She raised her math EFL to Low Intermediate Basic Education on a post-test on February 17, She left the program in December of Key Factors Contributing to a Positive Outcome: For Older Youth who just want a job but are basic skills deficient must get their basic skills. Without basic skills the jobs they get will not likely make them self-sufficient. Assist them to get a job but set up a program to also assist with basic skills. In some situations, the employer may cooperate also. Ensure that all out-of-school youth who are basic skills deficient are in a program that provides sufficient academic advancement. Prevent drop-outs: Participants who exit before completing one year are included in the measure

27 Key Factors Contributing to a Positive Outcome: Provide supports to help participant remain engaged in educational services and advance one level. Ensure timely testing within one year of program participation. Youth with disabilities are provided with reasonable testing accommodations 79 Individual Must be a Participant (Be Enrolled) to be a Part of the Measure 80 Access the Literacy/Numeracy information through the Assess Education Menu Item 81 27

28 Basic Literacy and Numeracy Screen 82 Click on Add button to open new record to enter test and score 83 Click on Test Name drop down arrow to get list to select test used 84 28

29 Click on Functional Area drop down arrow to select functional area 85 Fill in the scale score into the Test Score field and the Test Date. Always click on Save button to save the record. 86 Clicking on Insert New Record Allows Entry of Next Test and Score. You can Enter Pre-Test Information for Other Functional Areas Before Entering Post- Test Information for the First Test Area

30 After entering Test Name, Functional Area, and Test Score, and date OWCMS will complete Educational Functional Level and Grade Level. 88 Always click on Save button to save the record. The correlation information is displayed in the Correlation Section for the current record. Records will only show up in this section if a correlation is possible. Notice Gain is now Y Because the Test Score resulted in an increase to a higher Educational Functional Level. 89 Manage Soft Exits Manage exits whenever possible helpful to manage performance outcomes: By the time it is time for the individual to exit, the youth may be well into their first quarter after exit. Providing services after employment or the beginning of post-secondary education is recommend to insure stability in those activities. Insure individuals have certificates before exit 90 30

31 If you have questions or need information regarding performance measures: Robert J. Haas 4020 East Fifth Avenue (Air Center) P. O. Box 1618 Columbus, OH