Big Data Challenges: Data Management, Analytics & Security

Transcription

1 Big Data Challenges: Data Management, Analytics & Security Ivo D. Dinov Statistics Online Computational Resource University of Michigan

2 Big Data Challenges Availability, Sharing, Aggregation and Services Classical Data Science vs. Innovative Big Data Science Amateur Scientists vs. Experts Data Scientists vs. Practitioners Domain-specific vs. Trans-disciplinary knowledge Commercial vs. Open-source Resourceome Rapid Big Data Evolution Big Data IT proliferation Big Data Security risks Centralization won t work in Big Data Space Big Data is incredibly time, space, protocol, context dependent!

3 Big Data Characteristics * Mixture of quantitative & qualitative estimates Dinov et al. (2013)

4 Availability, Sharing, Aggregation & Services Cisco: "By the end of 2012, the number of mobile-connected devices [exceeded] the number of people on Earth There will be over 10 billion mobile-connected devices in 2016; i.e., there will be 1.3 mobile devices per capita Industry Sector Computer & Electronic Products Information Services Manufacturing Admin, support & waste management Transportation & Warehousing Wholesale Trade Professional Services Healthcare Providers Real Estate and Rental Finance and Insurance Utilities Retail Trade Government Accomodation & Food Arts & Enterntainment Corporate Management Other Services Construction Education Services Natural Resources Percent Growth Bubble Size ~ Relative size of GDP Big Data Value Potential Index U.S. Bureau of Labor Statistics McKinsey Global Institute

5 Amateur Scientists vs. Experts Democratization of Big Data Science Doctorate studies/certification is not mandatory nor does it guarantee appropriate Big Data expertise Lower barriers of entry Demand for constant Continuing Education and self-training Dichotomy between theoretical and empirical sciences Differences between fundamental knowledge and experimental skills (big data properties closely approximate core scientific principles)

6 Domain-specific vs. Trans-disciplinary knowledge Math/Stats Physics Biology Chemistry... Big Data Science Medical Sciences Social Sciences Environmental Sciences... Engineering Computer Science Bioinformatics Biomath/Biostats...

7 Commercial vs. Open-source Resourceome There is an explosion of open-data-science resources Spawning of a number of industries and enterprises blending proprietary and open-source data, code, documentation, expert-support, infrastructure and services Big Data to Knowledge: Google Cloud Platform (GCP) Amazon Web Services (AWS)

8 Commercial vs. Open-source Resourceome

9 Rapid Big Data Evolution Millions of Grass-Roots initiatives addressing Big Data Challenges Big Data complexities require truly innovative, collaborative, trans-disciplinary solutions Increase of Data complexity Sources Heterogeneity Datum-elements Incongruent sampling

10 Data Scientists vs. Practitioners Modelers, Engineers, (Applied) Users No one user completely understands the entire pipeline of data provenance, processing protocols, analytic strategies, or results interpretation Black-boxes. Accuracy Privacy concerns Consistency Infrastructure

11 Big Data Security Risks Big Data Fusion provides enormous opportunities and presents significant challenges Privacy, security and legal concerns, authenticity, accuracy, consistency, reliability, availability Healthcare The cloud services enable sharing big data Significant security and privacy concerns exist, Health Insurance Portability and Accountability Act (HIPAA) EMR/EHR Federal, state and local regulations/policies (IRBMED) Genetics Viral - Dual-use research of concern (DURC), /science de novo synthesis of polio virus, the Australian mousepox experiment, the Penn State aerosolization study

12 Kryder s law: Exponential Growth of Data Increase of Imaging Resolution 6E+15 4E+15 2E µm 10 µm 100 µm 1mm Gryo_Byte Cryo_Short Cryo_Color Cryo_Color Cryo_Short Gryo_Byte 1cm Neuroimaging(GB) Genomics_BP(GB) Moore s Law (1000'sTrans/CPU) Data volume Increases faster than computational power (estimated) Moore s Law (1000'sTrans/CPU) Genomics_BP(GB) Neuroimaging(GB) Dinov, et al., 2013

13 Alzheimer s Case Study: Stable-MCI vs. MCI-Converters Goals predictive-power of combinations of biomarkers and imaging derivative measures to provide reliable predictors of conversion from MCI to Alzheimer s disease Data MCI converters to AD (24-month period) and stable non-converters; matched for age, gender, handedness, education level Imaging (smri), Behavioral, Clinical, Neuropsychiatric, Biological data Approach Qualitative Exploratory Data Analysis and Quantitative Statistical Analysis (morphometric imaging correlates with clinical and genetics markers) MCI = Mild Cognitive Impairment (prelude to dementia of Alzheimer s type)

14

15 Alzheimer s Case Study: Stable-MCI vs. MCI-Converters Subject Demographics Gene -tics Clinical Neuroimaging Index Age Kg Sex APOE A1 APOE A2 NPI SCORE MMSE GD TOTAL CDR FAQ TOTAL L Gyrus Rectus BL L Superior Occipital Gyrus BL R Fusiform Gyrus BL L Caudate BL R Caudate BL L Putamen BL R Putamen BL F M N F

16 Alzheimer s Case Study: Stable-MCI vs. MCI-Converters Classification Results Using Baseline Data Hierarchical Clustering Prediction Ana (7 Regions) Metric True State (Dx at 24 month follow up) Converter Stable Total Converter TP FP TP+FP Stable FN TN FN+TN Total TP+FN FP+TN N Top 7 Regions Value Top 20 Regions Sensitivity Specificity Power to detect Converters Accuracy