Use of Evidence in Victorian Injury Prevention and Rehabilitation Regulators

Transcription

1 Use of Evidence in Victorian Injury Prevention and Rehabilitation Regulators Dr Pauline Zardo, Researcher Development and Research Impact Officer, The University of Queensland Prof. Alex Collie, CEO Institute for Safety Compensation and Recovery Research, Monash University

2 Background & Context Increasing expectations for publicly funded research to deliver impact McKeon Review recommended research funding strategy increased research use increased policy and program effectiveness better health outcomes Challenges and Opportunities Capacity: individual, organizational, structural Barriers to decision-makers use of research Barriers to engaging in research translation Translation intervention effectiveness 2

3 Public Health Policy Context WorkSafe and TAC injury prevention and rehabilitation compensation Injury Compensation policy affects health service access N=40,000 new workers compensation claims N=16,000 new transport compensation claims WorkSafe and TAC partnered with Monash to establish ISCRR to drive and support increased use of research 3

4 Inform and Support Research Use Aims Identify contextual factors affecting use of research. Identify how information is used to inform decision making. Inform the development of a research translation intervention Methods Use of Research in TAC Compensation Policies Quantitative policy content analysis; N=128 External & Organisational Factors Affecting Research Use Qualitative interview; N=33 Research Use and Individual Factors Affecting Research Use Quantitative online survey; N=405 4

5 Key Findings Summary Key factors affecting use of academic research evidence Individual Organisational External Relevance Communication Stakeholder feedback and action Skills for research use Relevance Ministerial and government input Internal prompts Resources Legal feedback and action Agency Access and awareness Main types of information used ordered by frequency of use Internal data and reports Policy, legislation and legal information Experience, expertise and advice Online information Clinical/medical evidence Academic research evidence Main purpose for use of information Information Types Instrumental Internal data and reports Policy, legislation and legal information Experience, expertise and advice Clinical/medical evidence Conceptual Academic research evidence Info online Symbolic 5

6 Key Findings Policy Content Analysis Research Evidence referred to in 30 of 128 policies (23.4%) Referred to more frequently (p<0.05) in treatment (65) compared to non-treatment policies (63) Of the 65 treatment policies, 41 (63%) made no reference to Research Evidence Main purpose: support claims decision making (1783) compared to policy development (600) 6

7 External Factors Affecting Research Use Direct Influence/Authority N Indirect Influence/Authority N Stakeholder feedback and action 26 Injured persons claiming compensation 8 Ministerial and government input 18 Media involvement 5 Legal feedback and action 12 Stakeholder engagement whenever we need to influence external stakeholders, so particularly Unions or Provider groups, that s when evidence becomes really important within the project. Because in those cases we have to justify why we re doing them. And if we can justify it through evidence that s been gathered that s believable and credible to an external group then we re more likely to have success in implementing the project. Ministerial and government input If you re a public servant, then you are at the behest of the government of the day, and our job is to implement the policy of the government of the day (Senior Manager 1) If the government says a decision we re not going to do X, Y, Z, then so be it, get on with it, don t worry about it (Manager 6) 7

8 Factors Affecting Decision-Making The way things ARE done N The way things OUGHT to be done Resources 16 Practical-implementable-KPI related N Individual Influences N 9 Decision making influencers 23 Internal info sharing 12 Quality control - proper process 8 Management and staff engagement 21 History and tradition 8 Commercial versus public interests 5 Politics-ideology 12 Relationships-networks 5 Priorities competing 11 Management and staff engagement If you get a Director onside who is not just doing it lip service, but has some sort of commitment to it, and if your policy is up there in about, I don t know, the top 10% of needs, won t be superseded and overcome by a higher priority 10 minutes after the project starts, well then you re probably on a winner. But if you re in some obscure part of the business God bless you and good luck to you. (Senior Manager 5)

9 Factors Affecting Research Use Research Push N Research Pull N Communicating research evidence Face to face Shorter faster simpler docs Tools, processes & strategies Actionable recommendations Relevance to context, implementation and outcome Resourcing Skills Supporting area or role Time Structure-process Staffing Costs-finance 28 Access and awareness 22 Credibility of the source 8 Political and policy processes 13 Relevance We need to be provided with clear examples of where the use of evidence has resulted in better and effective policy, otherwise it just adds to the number of stakeholders and people we have to involve in getting your job done Management support 10 Internally focused 7 Competing interests 7 Not my role 7 Networks-relationships 6 9

10 Communicating Research Evidence

11 Information Use Thinking about the last 12 months which types of information have you used to inform your work? Answer Response % Internal Data and Reports % Policy, Legislation and Legal Information % Medical/Clinical Evidence % Experience/Expertise/Advice % Academic Research Evidence % Information Collected Online % Statistic Value Total Responses

12 Research Use Demographics N % χ 2 sig level All Participants Use of Research Agency WS TAC Role Level Senior Manager Manager Non-Manager Role Type Programs/Projects Policy/Legal Operational Education Level High School/Certificate Undergraduate Postgraduate

13 Frequency of Information Use Frequency (%) Use of Information Type Daily Weekly Monthly Quarterly or less Internal Data & Reports Policy, Legislation & Legal Information Medical & Clinical Evidence Experience, Expertise & Advice Academic Research Evidence Information Online Pearson Chi square sig level = p=

14 Purpose for Information Use Main Purpose for Use of Information Type Internal Data and Report N (%total) Policy, Legislation and Legal Information N (%) Medical/Clinica l Evidence N (%) Experience, Expertise and Advice N (%) Academic Research Evidence N (%) Online Information N (%) Instrumental 192 (53.3) 182 (53.1) 128 (56.1) 148 (44.8) 44 (30.3) 96 (33.0) Conceptual 107 (30.8) 99 (28.9) 53 (23.2) 118 (35.8) 73 (50.3) 149 (51.2) Symbolic 48 (13.8) 62 (18.1) 47 (20.6) 64 (19.4) 28 (19.3) 46 (15.8) Total Use Confidence intervals (95%) for use of academic research evidence: Conceptual use ( ; ±7.23) Instrumental use ( ; ±8.08) Symbolic use ( ; ±7.31). Pearson Chi square sig level = p=

15 Predicting Research Use 15

16 Build capacity Implications for Intervention Training in research skills: access and quality assessment Effective communication: face to face and visual Internal prompts/drivers: support mechanisms, measures (KPIs), business planning process, etc. Work through existing processes and mechanisms Comparing research use to info use critical to effective evaluation increased engagement & interaction increased capacity increased relevance & uptake increased research use increased policy & program effectiveness better health outcomes 16