BIG DATA : National data linkage infrastructure. James Boyd
|
|
|
- Lily Arnold
- 10 years ago
- Views:
Transcription
1 BIG DATA : National data linkage infrastructure James Boyd
2 What defines Big Data? Data whose scale, diversity and complexity requires new architecture, techniques, algorithms and analytics to manage it and extract value and hidden knowledge from it
3 Characteristics of Big Data Volume data volumes exponential increasing over time Variety (Complexity) various formats, types and structures Velocity fast processing of data to ensure it is representative
4 Administrative Data Collections (Big-ish data?) Administrative data are usually collected by government for some administrative purpose - not primarily for research Life events can be generated across a number of different government areas (Health, Education, Criminal Justice etc.) The databases are often population based, so important population subgroups are not missed.
5 Data Sharing Ability to share the same data resource with multiple applications or users: Data and information used/reused to inform significant decisions Bring together key elements across Government Enhance the value of information gained from a single source
6 Data Sharing - Challenges Governance arrangements for sharing: Custodian requirements Control of data release Users agreements Confidentiality, Privacy and Security: Protecting confidentiality and Privacy Ensure data security throughout
7 Data Linkage - Overview To establish efficiently and accurately which records belong to the same individual. Personal identifying information makes Data Linkage Possible: Family Name First Name(s) Date of birth Postcode
8 Matching Techniques Exact matching can lead to inexact results e.g. requiring exact match on a number of fields e.g. surname, first initial, date of birth, sex - expect at least 10-15% errors because of discrepancies Probability matching more accurate Quantifies levels of agreement & disagreement 2% true links missed
9 How Does Linkage Work? Bring together the pairs of records to be compared Quantify the relative probability that the two records belong to the same person Make the linkage decision
10 Development of linkage methodology Matching Challenges?
11 Population Health Research Network PHRN: Collaborative Network developing data linkage capability within and between Australian jurisdictions 6 state/territory linkage units - 2 existing (WA, NSW/ACT) + 4 new (Qld, Vic, Tas & SA/NT) Program Office in Perth providing coordination and national client services National linkage (Centre for Data Linkage at Curtin University and AIHW Commonwealth Data Integration) Secure Unified Research Environment (SURE) and secure Data Delivery (Sax Institute)
12 Centre for Data Linkage (CDL) Building national data linkage infrastructure Facilitate linkages that span across state/territory borders Link these datasets with research datasets Secure linkage of datasets Research & Development into data linkage methods
13 Why develop a new linkage system? Address weaknesses & gaps in existing systems (complexity, scale, performance, functionality, administration) Provide an enterprise-grade platform that is reliable, easy to maintain and operate, with auditing capabilities Automate functions that traditionally require manual intervention Tackle emerging problems e.g. privacy-preserving linkage
14 What differentiates the NLS? Large data volume (linkage, data management, output, scalability) Manages multiple linkage & extraction projects Manages new, amended and deleted records (open file handling) Handles diverse linkage & DC needs e.g. enduring vs project linkage; researcher wishing to link their own data; DCs imposing restrictions on linkage Secure & auditable
15 Managing change over time How to handle change over time ( New, Amends & Deletes )? NLS handles new, amended & deleted Records (Open file handling) NLS handles deletion of Data collections, Data Providers and Linkage Projects NLS differentiates between end-dated & deleted records
16 Any point in time referencing NLS stores full history of records and groups Groups are dynamic entities Linkage structure can be recreated for any record at any (previous) point in time
17 Graph of Matching Group
18 PHRN: Proof of Concept Project Hospital-related mortality CDL created linkage keys using demographic data from WA, NSW, SA and QLD hospital morbidity and mortality data collections Linkage of around 45,000,000 event records Linkage Processing completed within 10 days Over 2 billion pair relationships
19 Contact Details James Boyd Centre for Data Linkage Curtin University
Data linkage for paediatric trauma and health services research
Data linkage for paediatric trauma and health services research Transport and Road Safety (TARS) Research, School of Aviation Rebecca Mitchell 1,2 1 University of New South Wales, Australia 2 Neuroscience
5C R I M I N A L J U S T I C E R E S O U R C E S
5C R I M I N A L J U S T I C E R E S O U R C E S Justice expenditure According to the Report on Government Services (1998), the total government expenditure on justice in 1996/97 was approximately $5.4
COMMINSURE HOME INSURANCE PREMIUM, EXCESS AND DISCOUNT GUIDE.
COMMINSURE HOME INSURANCE PREMIUM, EXCESS AND DISCOUNT GUIDE. This document provides you with information to help you understand how your total premium has been calculated, discounts that are available
Report into the Rural, Regional and Remote Areas Lawyers Survey. Prepared by the Law Council of Australia and the Law Institute of Victoria
Report into the Rural, Regional and Remote Areas Lawyers Survey Prepared by the Law Council of Australia and the Law Institute of Victoria July 2009 Acknowledgements The Law Council is grateful for the
Health expenditure Australia 2011 12: analysis by sector
Health expenditure Australia 2011 12: analysis by sector HEALTH AND WELFARE EXPENDITURE SERIES No. 51 HEALTH AND WELFARE EXPENDITURE SERIES Number 51 Health expenditure Australia 2011 12: analysis by sector
Information Bulletin AN INCREASING INDIGENOUS POPULATION? IMPLICATIONS FOR THE CRIMINAL JUSTICE SYSTEM. Justine Doherty
Information Bulletin OFFICE OF CRIME STATISTICS No 32 April 2002 AN INCREASING INDIGENOUS POPULATION? IMPLICATIONS FOR THE CRIMINAL JUSTICE SYSTEM Justine Doherty ATTORNEY-GENERAL S DEPARTMENT Introduction
Economic benefits of closing the gap in Indigenous employment outcomes. Reconciliation Australia
Economic benefits of closing the gap in Indigenous employment outcomes Reconciliation Australia January 2014 Contents Acronyms... i Glossary... ii Executive Summary... i 1 Introduction... 1 1.1 Methodology...
PRINCIPLES FOR ACCESSING AND USING PUBLICLY-FUNDED DATA FOR HEALTH RESEARCH
TARGETED CONSULTATION DRAFT National Health and Medical Research Council PRINCIPLES FOR ACCESSING AND USING PUBLICLY-FUNDED DATA FOR HEALTH RESEARCH Developed by NHMRC s Prevention and Community Health
Mesothelioma in Australia: Incidence (1982 to 2013) and Mortality (1997 to 2012)
Mesothelioma in Australia: Incidence (1982 to 213) and Mortality (1997 to 212) 215 Disclaimer The information provided in this document can only assist you in the most general way. This document does not
Tutorial- Create a cascading drop-down control
Tutorial- Create a cascading drop-down control www.nintex.com [email protected] Contents Introduction... 3 Creating the SharePoint lists... 4 Customizing the SharePoint list form using Nintex Forms...
Council of Ambulance Authorities
Council of Ambulance Authorities Patient Satisfaction Survey 2013 Prepared for: Mojca Bizjak-Mikic Manager, Data & Research The Council of Ambulance Authorities Prepared by: Natasha Kapulski Research Associate
X NSW/ACT X NT X QLD X SA X TAS X VIC X WA
1. Please complete this form USING BLACK INK and write within the boxes in CAPITAL LETTERS. Mark appropriate answer boxes with a CROSS. Start at the left of each answer space and leave a gap between words.
Information Circular
Information Circular Enquiries to: Brooke Smith Senior Policy Officer IC number: 0177/14 Phone number: 9222 0268 Date: March 2014 Supersedes: File No: F-AA-23386 Subject: Practice Code for the Use of Personal
NAMS.PLUS Asset Management Tools for Practitioners
NAMS.PLUS Asset Management Tools for Practitioners Vancouver, Oct 2009 Chris Champion Institute of Public Works Engineering Australia [email protected] Work within a na,onal framework Provide the
Family Day Care Educators
Obligations and expectations of Family Day Care educators Overview A Family Day Care (FDC) service that is approved under the family assistance law to administer Child Care Benefit (CCB) and Child Care
1. BACKGROUND Accuracy Timeliness Comparability Usability Relevance
Data Quality Policy 1. BACKGROUND WA Health, like most organisations, is becoming increasingly dependent on information for decision-making. Ensuring that this information is of the highest possible quality
THE AUSTRALIAN MESOTHELIOMA REGISTRY (AMR)
THE AUSTRALIAN MESOTHELIOMA REGISTRY (AMR) Professor Nico van Zandwijk Director Asbestos Diseases Research Institute US US UK UK COUNTRY NZ CAN COUNTRY NZ CAN AUS AUS Mesothelioma International context
AROC. Establishing and Maintaining a National Clinical Registry. Frances Simmonds, AROC Director
AROC Establishing and Maintaining a National Clinical Registry Frances Simmonds, AROC Director A R O C Australasian Rehabilitation Outcomes Centre A National Clinical Rehabilitation Registry Rehabilitation
As an aged care worker, this incentive is for you to upgrade your qualifications and build your career in aged care.
Aged Care Education and Training Incentive Programme COMMENCEMENT PAYMENT This application form is to be completed by eligible aged care workers who have enrolled and commenced studies to enhance their
Motor Vehicle. Claim Form PLEASE RETURN COMPLETED FORM TO YOUR JLT OFFICE:
Motor Vehicle Claim Form PLEASE RETURN COMPLETED FORM TO YOUR JLT OFFICE: GPO Box 1693 ADELAIDE SA 5001 Tel +61 (0)8 8235 6446 Fax +61 (0)8 8235 6448 PO Box 925 ALBURY NSW 2640 Tel +61 (0)2 6057 3333 Fax
BOOKED CAR SCHEME GUIDELINES FOR TAXI AND HIRE CAR CONTRACTORS
BOOKED CAR SCHEME GUIDELINES FOR TAXI AND HIRE CAR CONTRACTORS These Guidelines relate to the Deed of Agreement for the Provision of Passenger Transport Services for Eligible DVA Clients. July 2012 Version:
IF YOU DO NOT ATTEND THE CONFERENCE THE CASE MAY BE SENT TO COURT AND ORDERS MADE IN YOUR ABSENCE.
The Case Assessment Conference the first Court event This brochure provides general information and suggests ways to prepare for what will be the first Court event most people have at the Family Court
How To Understand The Benefits Of Big Data
Findings from the research collaboration of IBM Institute for Business Value and Saïd Business School, University of Oxford Analytics: The real-world use of big data How innovative enterprises extract
Competency Completion Online System Guide
Competency Completion Online System Guide Commonwealth of Australia July 2008 This work is copyright. It may be reproduced in whole or in part for study or training purposes, subject to the inclusion of
Professional Indemnity Insurance Proposal Form Lawyers Excess of Loss / Top-Up Insurance
Professional Indemnity Insurance Proposal Form Lawyers Excess of Loss / Top-Up Insurance IMPORTANT NOTICE Your Duty of Disclosure Before you enter into a contact of general insurance with any insurer,
Product Disclosure Statement
Foreign Exchange Product Disclosure Statement 26 March 2013 Happy Banking Important Notice This Product Disclosure Statement (PDS) is provided by Bankwest, a division of Commonwealth Bank of Australia
Data Governance in-brief
Data Governance in-brief What is data governance? Data governance is the system of decision rights and accountabilities surrounding data and the use of data. It can involve legislation, organisational
As an aged care worker, this incentive is for you to upgrade your qualifications and build your career in aged care.
Aged Care Education and Training Incentive Programme COMPLETION PAYMENT This application form is to be completed by applicants who have completed studies and have already received a commencement payment
National Disability Insurance Scheme (NDIS): Funding the Unfunded Commitment
National Disability Insurance Scheme (NDIS): Funding the Unfunded Commitment prepared for the Insurance Council of Australia April 2012 NDIS is currently a $6.5 billion per annum unfunded commitment this
Managing injury and return to work policy
Managing injury and return to work policy Description: This is a precedent workplace policy governing the management of injury and return to work. Author: Brooke Pendlebury Principal Lawyer, Pendlebury
Deloitte Reverse Mortgage Survey December 2013
Deloitte Reverse Mortgage Survey December 2013 James Hickey Financial Services Partner +61 2 9322 5009 [email protected] Contents 1. Market Volumes & Mix 2. Housing Splits 3. Borrowers 4. Summary
A quick guide to Australian discrimination laws
A quick guide to Australian discrimination laws Over the past 30 years the Commonwealth Government and the state and territory governments have introduced laws to help protect people from discrimination
Learning Guide. Broker Guide. Learn. Think. Grow
Learning Guide Learn Broker Guide Think Grow Talent and Development July 2006 Table of Contents Introduction... 3 The Online Submission Process... 4 Which supporting documents do I need to send?... 5 Decision
Summary Report. Department of Innovation, Industry, Science and Research. Industry and Small Business Policy Division
Summary Report Department of Innovation, Industry, Science and Research Industry and Small Business Policy Division Small Business Dispute Resolution June 2010 DIISR Small Business Dispute Resolution Research
Women s Health Victoria
Women s Health Victoria Termination of Pregnancy Post 20 Weeks Background Paper March 2007 Prepared by Kerrilie Rice Policy and Research Officer Published by Level 1, 123 Lonsdale Street Melbourne Victoria
The Comcare Self-Insurance Option Bruce Watson, Rod McInnes and Mark Hurst
The Comcare Self-Insurance Option Bruce Watson, Rod McInnes and Mark Hurst This presentation to be read in conjunction with the authors paper. Agenda 1. Productivity Commission report and the Comcare option
Mutual Recognition. Who can apply? Build better.
Build better. Mutual Recognition Mutual Recognition is a process whereby an individual who holds a licence in one state, territory or New Zealand is entitled to be licensed in another state, territory
11 Primary and community health
11 Primary and community health CONTENTS Indigenous data in the primary and community health chapter 236 Profile of primary and community health 237 Community health services 237 Dental services 237 Size
2. The costs of natural disasters
2. The costs of natural disasters Key Points Without action, the forecast annual cost in real terms of natural disasters (across government, business and communities) in Australia is expected to reach
REQUEST FOR QUOTE. Review & assessment of National Quality Framework Resource Kit
REQUEST FOR QUOTE Review & assessment of National Quality Framework Resource Kit November 2013 Contents 1 Lodgement of proposals... 3 2 Introduction... 4 3 Project purpose... 4 4 Project methodology...
PUBLIC & PRODUCTS LIABILITY PROPOSAL FORM IMPORTANT INFORMATION: PLEASE READ THE FOLLOWING INFORMATION BEFORE COMPLETING THIS PROPOSAL
PUBLIC & PRODUCTS LIABILITY PROPOSAL FORM IMPORTANT INFORMATION: PLEASE READ THE FOLLOWING INFORMATION BEFORE COMPLETING THIS PROPOSAL A. Obtaining a Quotation To minimise delays in obtaining a quotation
REPORT OF 2015 NAPLAN TEST INCIDENTS
REPORT OF NAPLAN TEST INCIDENTS Introduction The Assessment Program Literacy and Numeracy (NAPLAN) tests are held during May each year. In the tests were held on 12, 13 and 14 May and, in total, approximately
Professional Indemnity Insurance Application Form Marsh Accountants Insurance Program
Professional Indemnity Insurance Application Form Marsh Accountants Insurance Program Phone: 1300 093 634 Email: [email protected] tice to the Applicant This notice must be read before you complete
Australian Housing Outlook 2014-2017. By Robert Mellor, Managing Director BIS Shrapnel Pty Ltd October 2014
Australian Housing Outlook 2014-2017 By Robert Mellor, Managing Director BIS Shrapnel Pty Ltd October 2014 Recent Residential Property Market Trends Residential property demand has varied across purchaser
Data Governance Policy. Version 2.0 19 October 2015
Version 2.0 19 October 2015 Document Title: Summary: Date of Issue: Status: Contact Officer: Applies To: References: This policy provides the Cancer Institute NSW with an instrument to formally manage
Professional Indemnity Insurance Proposal Form
Professional Indemnity Insurance Proposal Form Important Notices Claims Made Insurance This is a proposal for a Claims Made policy of insurance. This means that the policy covers you for any claims made
Review of parentage laws
Review of parentage laws AUSTRALIAN HUMAN RIGHTS COMMISSION SUBMISSION TO THE FAMILY LAW COUNCIL 3 May 2013 ABN 47 996 232 602 Level 3, 175 Pitt Street, Sydney NSW 2000 GPO Box 5218, Sydney NSW 2001 General
Becoming an Electricity Retailer
Becoming an Electricity Retailer RDANI is investigating renewable energy options in our region, with the view to improving business competitiveness through lower energy costs, minimising the carbon tax
Feedback on the Inquiry into Serious Injury. Presented to the Road Safety Committee of the Parliament of Victoria. 08 May 2013
Feedback on the Inquiry into Serious Injury Presented to the Road Safety Committee of the Parliament of Victoria 08 May 2013 About the APA The Australian Physiotherapy Association (APA) is the peak body
1.17 Life expectancy at birth
1.17 Life expectancy at birth The life expectancy of Aboriginal and Torres Strait Islander males and females for a given period. Data sources Life expectancy estimates for the years 2005 2007 presented
ACCOUNTANTS PROFESSIONAL INDEMNITY INSURANCE PROPOSAL FORM IMPORTANT FACTS RELATING TO THIS PROPOSAL FORM
Australian Indemnity ACCOUNTANTS PROFESSIONAL INDEMNITY INSURANCE PROPOSAL FORM IMPORTANT FACTS RELATING TO THIS PROPOSAL FORM The Purpose of this Proposal Form is to set out all relevant information for
A Secure Data Architecture for Telehealth Trial
Funded by the Australian Government under the National Telehealth Pilots Program A Secure Architecture for Telehealth Trial Surya Nepal, Julian Jang-Jaccard, Rajiv Jayasena, Branko Cellar, Ross Sparks,
OAMPS Sports Risk Management
2012/13 OAMPS Sports Risk Management Sports Injury Insurance Cover Closer to clients Closer to Communities Sports Injury Insurance Cover We are pleased to present this quick guide to the 2013/2014 Sponsorworx
Life insurance how much cover is needed? Fact Sheet - October 2014
Life insurance how much cover is needed? Fact Sheet - October 2014 This fact sheet examines various methods of estimating the level of personal life insurance to be taken by an individual. The calculation
BSB41507 Certificate IV in Project Management Information and Enrolment Kit
Information and Enrolment Kit Course Description Units of Competency Application of all Project Management Units This unit applies to a project team member working under the direction of a project manager
PROFESSIONAL INDEMNITY INSURANCE PROPOSAL
NOTICE TO INSURED (Pursuant to the provisions of the Insurance Contracts Act 1984) Your Duty of Disclosure Before you enter into a contract of general insurance with an insurer, you have a duty, under
Marriage, families & separation
FAMILY COURT OF AUSTRALIA FEDERAL CIRCUIT COURT OF AUSTRALIA Marriage, families & separation Separation This brochure provides information for people considering, or affected by separation or divorce.
HUDSON SALARY GUIDES 2015
Australia HUDSON SALARY GUIDES 2015 Office Support about this publication This Salary Guide is a compilation of salary and market information provided by Hudson consultants, clients, candidates and other
Marriage, families & separation
FAMILY LAW COURTS Marriage, families & separation This brochure will tell you This brochure provides information for people considering, or affected by separation or divorce. This brochure includes information
MESOTHELIOMA IN AUSTRALIA INCIDENCE 1982 TO 2008 MORTALITY 1997 TO 2007
MESOTHELIOMA IN AUSTRALIA INCIDENCE 1982 TO 28 MORTALITY 1997 TO 27 August 212 Safe Work Australia Mesothelioma in Australia Incidence 1982 to 28 Deaths 1997 to 27 August 212 Acknowledgement Data on the
MESOTHELIOMA IN AUSTRALIA INCIDENCE 1982 TO 2009 MORTALITY 1997 TO 2011
MESOTHELIOMA IN AUSTRALIA INCIDENCE 1982 TO 29 MORTALITY 1997 TO 211 OCTOBER 213 Safe Work Australia Mesothelioma in Australia Incidence 1982 to 29 Mortality 1997 to 211 OCTOBER 213 Acknowledgement Disclaimer
Marriage, families & separation
FAMILY COURT OF AUSTRALIA FEDERAL CIRCUIT COURT OF AUSTRALIA This brochure will tell you This brochure provides information for people considering, or affected by separation or divorce. This brochure includes
Obligations and expectations of Family Day Care educators
Obligations and expectations of Family Day Care educators 1. Overview A Family Day Care (FDC) service that is approved under the family assistance law to administer Child Care Benefit (CCB) and Child Care
Information Management: A common approach
Information Management: A common approach July 2013 Document Control Document history Date Version No. Description Author July 2013 1.0 Final Department of Finance and Services October 2013 1.1 Updated
Information Security in Big Data using Encryption and Decryption
International Research Journal of Computer Science (IRJCS) ISSN: 2393-9842 Information Security in Big Data using Encryption and Decryption SHASHANK -PG Student II year MCA S.K.Saravanan, Assistant Professor
Private Health Insurance Australia
Private Health Insurance Australia Quarterly Statistics September 2014 Contents Snapshot of the industry..... 3 Membership and coverage.... 4 Benefits paid..... 6 Service utilisation... 9 Out-of-pocket
WILLIS ED GROUP STUDENT PERSONAL ACCIDENT CLAIM PROCEDURE FOR PARENTS
WILLIS ED GROUP STUDENT PERSONAL ACCIDENT CLAIM PROCEDURE FOR PARENTS Student Accident Claims are managed by the insurer AIG Australia Limited (formerly Chartis) Completed claim forms and supporting documentation