Joined up Government needs Joined up Data. IPA/ICS 10 th Annual Public Sector IT Conference Dublin Castle 31 st October 2014

Transcription

1 Joined up Government needs Joined up Data IPA/ICS 10 th Annual Public Sector IT Conference Dublin Castle 31 st October 2014

2 Joined up data - context Irish Statistical System: The Way Forward Joined up Government Needs Joined up Data ISS based on National Data Infrastructure (NDI) for efficient and effective public administration

3 Consider Tax policy and household income and expenditure Welfare payments as a deterrent to work Impact of raising the cost of 3 rd level education Where to build schools Child security child drops out of school Transport policy commuting patterns Effectiveness of Local Children Services Committees Decision making at a local level

4 NDI - Joined up data Enterprises/Firms Persons Buildings/Location Key : Permanent official identification

5 Broader Legal Context Data Protection Freedom of Information Official Statistics Key : 3 Legislative pillars and how they interact

6 Statistics Act 1993 Access to records of Public Authorities S30 not apply to guards, court... apply to medical records with agreement of Minister of Health Co-operation of public authorities S31...co-operate in assessing the potential of records for statistical purposes... shall consult with DG... and accept any recommendations he may reasonably make... Protection of information (Part V) S32 Restrictions on use of information S33 Prohibition on disclosure of information Statistics Act

7 CSO Data Protocols Agreed by NSB and Data Protection Commissioner On Register of Data Matching Activities

8 Considering administrative data records Transactions can be used to define Events can define start and end of Episodes can be used to define Status What about registrations?

9 Warehouse view of ADC Sys 1 Sys... Sys n Business systems tier Business Intelligence (BI) Data 1 Data 2 Micro data analysis tier Extract Transform Load (ETL) PREM DES Sources data tier

10 An example creating the P35 statistical analysis file SPP35 Analysis file for each reference year CSOPPSN CBR ID Month of birth, Sex, Nationality Activity (NACE Rev 2), Legal form Weeks, Reckonable pay Analysis Integrated and depersonalised for analysis purpose CSO Business Register EMR ID CBR ID Legal form Activity Revenue Employer Employee Tax returns EMR ID PPSN Class Weeks Pay DSP Client record System PPSN DOB Gender Nationality Source

11 Learning from the past - think about the bigger picture Nordbotten, Thygesen and the statistical archive concept from the 1960s 11

12 Do more with less Mindset - Opportunities exist with secondary data sources 12

13 Big data working definition Data that is difficult to collect, store or process within the conventional systems of statistical organizations. Either, their volume, velocity, structure or variety requires the adoption of new statistical software processing techniques and/or IT infrastructure to enable cost-effective insights to be made. 13

14 The sandbox CSO/ICHEC collaboration The hardware on which the sandbox system is based is a High Performance Computing cluster called Stoney. The cluster is hosted in the National University of Ireland, Galway since April 2009 and is composed of 60 compute nodes each of which has two 2.8GHz Intel (Nehalem EP) Xeon X5560 quad-core processors, 48GB of RAM and a 1TB local disk. Each node is connected to two networks an InfiniBand network for accessing the shared Lustre filesystem and for high performance communications as well as a Gigabit Ethernet network for management tasks. In addition, a 20TB shared filesystem is available to all nodes. ICHEC will dedicate 20 compute nodes to enable a Hadoop cluster with 160 cores almost 1TB of RAM and 20TB of HDFS distributed storage. Play is the highest form of Research - Einstein

15 The toys (data sources) o twitter data o mobile phone data o satellite imagery / aerial photography o price data/ job vacancy data via scraping o scanner data/price data sourced via large vendors o data from road traffic sensors o smart meter data on electricity/gas consumption 15

16 Some of the players 16

17 The sandbox - looking to the future o Centres for Research and Development o Centres of Excellence? o Partner organisations for collecting, processing or storing data of a less or non sensitive nature??? o Significant partner organisations enabling the collection, processing or storing data of a sensitive nature?????? 17

18 Learning from the past - do not underestimate privacy concerns The National Data Center and Personal Privacy By Arthur R Miller

19 Concluding remarks Digital world provides opportunities Think about the bigger picture Think about potential partners Be open, trustworthy and safe Don t underestimate privacy concerns Work together