The P.S. Program: Using predictive analytics in program implementation

Transcription

1 The P.S. Prgram: Using predictive analytics in prgram implementatin

2 Pst Reunificatin Supprt (P.S.) Prgram Overview Helping Families Stay Tgether Wiscnsin s Waiver Demnstratin Prject Develped t address Wiscnsin s high re-entry rate int ut f hme care, which was 21% in Prvides enhanced case management, family-centered services, and blsters natural cmmunity-based supprts. Designed t als prduce changes in the family s shrt-term, intermediate, and lng-term utcmes.

3 Why Use a Predictive Risk Mdel? The decisin t use a predictive risk mdel arse t: Target limited resurces t the children wh are mst at risk f experiencing a re-entry int ut f hme care Systematically determine the allcatin f slts Learn mre abut what wrks t help high risk children and families

4 By targeting thse mst at risk f re entry we hpe t: Address Unmet Family Needs and Stressrs Decrease the Trauma Assciated with Reentry Make a Smart Investment in rder t Reinvest Savings and Help Mre Families

5 Risk Mdel Develpment As part f the early waiver evaluatin activities, the Children and Family Research Center (University f Illinis) began t develp a risk mdel t identify which reunified children were at highest risk f re-entering substitute care within 12 mnths. Wrk n the mdel began in July Data that was available fr use in the mdel develpment included: AFCARS elements Family demgraphics Placement characteristics Maltreatment histry Child and Adlescent Needs and Strengths (CANS)

6 Risk Mdel Develpment: Sample The sample used in the analysis cnsisted f children wh were reunified frm substitute care during state fiscal year 2012 (July 1, 2011 June 30, 2012). Children excluded frm the sample included: Reunified children in Milwaukee Cunty (BMCW) Children with juvenile justice nly cases (jint JJ/CW cases were included) This resulted in a sample f 1,844 reunified children. This sample was then split in half; the first half (n=922) was used t develp the risk mdel and the secnd half (n=922) was used t validate the sample.

7 Risk Mdel Develpment: Factr Cmbinatin All relevant variables were tested fr statistical significance in crrelatin t re-entry int OHC within 12 mnths f reunificatin. Variables tested included: Child/family: gender, race, child and caretaker age, disability, family structure Placement histry: number f placements during mst recent spell, last placement type prir t reunificatin, duratin f placement, ever placed in shelter, residential treatment, r institutin

8 Risk Mdel Develpment: Factr Selectin Variables tested in the mdel develpment prcess included: CANS dmains: adjustment t trauma, behaviral r emtinal needs, risk behavirs, family acculturatin, schl/daycare, child/yuth strengths, life functining, identified permanent resurce strengths/needs Maltreatment reprt: case type, relatinship t perpetratr, substantiatin, number f prir referrals r service reprts, reasn fr remval

9 Risk Mdel Develpment: Factr Selectin All variables that were related t re-entry at the bivariate level were tested in the mdel. Stepwise lgistic regressin was used t find the best cmbinatin f factrs that predicted re-entry int OHC within 12 mnths f reunificatin. The final mdel cntained 4 variables: Child disability Single-parent family Length f time in care prir t reunificatin Number f service reprts prir t mst recent entry int care

10 Determining Eligibility Cut Off Scre

11 Prgram Implementatin Use f the Re-entry Preventin Mdel (RPM) in the P.S. Prgram After the RPM was created, a reprt in ewisacwis was designed t allw cunties t find yuth in OHC wh were eligible. The reprt runs autmatically t determine a current RPM scre fr all kids in care, and a scre fr all kids if they discharge t reunificatin in 60 days. The reprt als allws cunties t specify any number f days until a planned reunificatin, and prvides a scre n demand.

12 Initial Cunty Cncerns: A lwer number f children were eligible fr the prgram than riginally prjected The need t run and meaningfully use a reprt n a frequent basis An understanding f what accurate data entry int ewisacwis meant fr child eligibility Variatins in cunty practice affect presence f factrs Anticipated factrs were absent frm the mdel

13 Prgram Implementatin Wiscnsin tk the fllwing steps t facilitate use f the RPM during the first six mnths f implementatin: Addressing lwer than anticipated enrllment: Lwered the eligibility threshld t allw mre children in the prgram Held ne-n-ne (cunty t state) meetings t walk thrugh the mdel, lcal cunty data prfiles, and ther questins regarding the RPM and enrllment prcess Allwed cunties t lck in eligible RPM scres n referrals fr planned reunificatin dates with up t a 30 day variance n the actual reunificatin date Allwed cunties t lck in eligible RPM scres fr children wh are in a trial reunificatin

14 Retling fr Year Tw: RPM 2.0 Develping RPM 2.0 with a new and larger slice f data, with the fllwing alteratins: Increased and varied pull f data fr dataset Tiered aspects f the RPM t allw fr chrts f eligible yuth Clear delineatin f hw case and child IDs are structured cunty practice

15 Year 1 Lessns Learned Fur categries f lessns learned: 1. Data 2. Timing 3. Outreach 4. Implementatin

16 Lessns Learned: Data 1. Think carefully abut data availability and quality Statistical significance is nt synnymus with data quality Key indicatrs f risk learned frm field experience may nt be dcumented The availability f data may nt be specific r reliable enugh t capture the nuance f emerging risk factrs 2. Have a full understanding f the internal data used in mdel Is data indicative f actual practice in the field? Has all the pssible surces f data been used?

17 Lessns Learned: Timing 3. Determine a timeline fr develpment, testing, and utreach cmmunicatin Allw enugh time fr internal understanding, as the use f predictive analytics is a dynamic prcess Create a cushin f time fr stakehlder and cunties t react and ask questins prir t finalizing the mdel and implementatin Carefully plan messaging and utreach cmmunicatin

18 Lessns Learned: Outreach 4. Cnsider variables with clleagues frm ther sectins/bureaus Ask abut any ptential variance in lcal agency practice related t each data item Understand any pssible changes in plicy r practice related t each data item Cnsider any unintended cnsequences t plicy, standards, r ther prgrams that may stem frm utilizatin f the variables 5. Cmmunicate clearly abut mdel develpment prir t implementatin Transparency n hw the mdel is created is crucial fr buy-in Predictive analytics will capture histric data patterns but may nt indicate emerging changes in practice

19 Lessns Learned: Implementing 6. Expect resistance Understanding and using predictive analytics may nt cme naturally t many scial service prfessinals Many scial service prfessinals will prefer t serve clients with high needs rather than thse wh are high risk Even an exceptinally well develped risk mdel will nt identify every case with legitimate risk 7. Be adaptable After initial implementatin unfreseen situatins r scenaris may cme t light Sme degree f flexibility will allw prgram staff t meet the needs f stakehlders and the ppulatin served

20 Six mnths int prgram implementatin: Cntinued adaptability f bth the RPM and the referral prcess have resulted in enhanced engagement with participating cunties. As the prgram has evlved, s has ur understanding f the mdel and ur understanding f the statewide and lcal data. Cunty partners have als grwn int using the mdel and are cllabrating with us fr retling RPM 2.0.

21 Cntact infrmatin Tamara Fuller, Ph.D, Children and Family Research Center, University f Illinis Urbana-Champaign (tfuller@illinis.edu) Thedre Crss, Ph.D, Children and Family Research Center, University f Illinis Urbana-Champaign (tpcrss@illinis.edu) Vaughn Brandt, Wiscnsin Department f Children and Families (vaughnx.brandt@wiscnsin.gv) Clleen McGrarty, Wiscnsin Department f Children and Families (clleen.mcgrarty@wiscnsin.gv)