Challenges and threats to data collection and usage

Size: px

Start display at page:

Download "Challenges and threats to data collection and usage"

Phillip Matthew Manning
8 years ago
Views:

1 Challenges and threats to data collection and usage Presentation at Welfare Reform: Meeting the Policy Challenges of Change, September 2015, Australian National University, Canberra ".every major scientific revolution has been driven by one thing, and that is data. Shaw, 2014, Harvard Magazine

2 Context -- contradictory imperatives Policy u More complex, diverse society u Easy and rapid communication of 'information u 24 hour media cycle u Multiple sources of data and interpretation u Contestability of policy advice u Demands from ministers u Moral panics u Rights based and increasingly litigious community u Increased expectation of transparency in decision making Academic u ARC wont fund regular data collection, $$ per grant are low, does not cover academic salaries u Success rates for social science competitive grants low u No incentive for curation and depositing of data u Rewards are weighted to 'solo' research activity u Promotions system u Citations - rankings u Multiple jobs teach, grant raising, research/publish, administration, service, community outreach

3 Big data Challenges u Identifying what matters u Sophisticated analysis new software tools u Linkage of large administrative datasets u Managing teams u Communicating the value of data u New frontiers textual and visual data Threats u Privacy u Access u Failure to invest in research u Risk over quantification u Lack of rewards to make data 'available'

4 Ethics committees Challenges u Committee members u Process red tape u Timeliness u Managing 'consent' u Confidentiality and linkage u Committees being used as proxies for assessing legal or reputational risk to governing institution Threats u Quality of the research u Risk averse u 'Conservative' research u Controls over depositing, access and preservation

5 Right to privacy Challenges Threats u Public confidence in 'science' u Understanding confidentiality u Too many surveys u Collection of repititous data u Moving to a new discussion about the need for access to key data to inform policy making u Public trust/legitimacy u Stalking horse u Lawyers u Media

6 Risk averse public service Challenges u Dealing with the findings u Why no effects matter u $$$ -- good research costs u Skills to understand methodology poor methods training of undergraduate and PhD students u Collaboration -- has more benefits than negatives u Producer not just consumer of data Threats u Failure to collect high quality data poor policy u Failure to evaluate waste of money and resources u FOI media always looking for a headline u Inward looking reaffirms the status quo u Silos internal competitiveness crowds out linkages u Program goals unrealistic and not based on evidence u Turnover in key staff

7 Outsourcing Challenges u Managing quality control of information u Consistency across service providers u Who owns the data and what does that mean in practice? Threats u Can't access key data u No linkage across datasets u Consistency over time when providers change u Competitiveness between providers

8 Open data and archiving Challenge u Loss of 'control' u Managing 'adverse' findings u Data 'quality' u Documentation common standards u Preserve the historical record u Publish or perish u Qualitative and visual data u Intellectual property shortterm versus national benefit Threat u Gatekeepers u Privacy concerns u Ethics requirements u Cost u Volume of data u Restrictions on access u Funding for archives

9 Advantages of open data for academics from Kingsley and Brown

10 Why does this matter? u Quality of advice/research findings u Quality of training and reduce wasteful data collection u Enhance quality of underlying data u Replication u Ability to conduct evaluations linked datasets u Digital record for historical purposes u Value for money and new analysis u Build a strong system to support domestic research and its uptake u Innovation and global competitiveness serendipity and business

11 'Final report - Science as an open enterprise' by UK Royal Society u Open inquiry is at the heart of the scientific enterprise. Publication of scientific theories - and of the experimental and observational data on which they are based - permits others to identify errors, to support, reject or refine theories and to reuse data for further understanding and knowledge. Science s powerful capacity for self-correction comes from this openness to scrutiny and challenge.

Wir schaffen Wissen heute für morgen. Workshop Research Integrity at PSI 2013 Data management Tuesday June 4 2013, 13.30 17.00. Louis Tiefenauer, PSI

Wir schaffen Wissen heute für morgen Workshop Research Integrity at PSI 2013 Data management Tuesday June 4 2013, 13.30 17.00 Louis Tiefenauer, PSI PSI, 10. Juni 2013 Program Dur. End Welcome by Thierry