Big Data for Higher Education and Research Growth

Transcription

1 Big Data for Higher Education and Research Growth Hao Wang, Ph.D. Chief Information Officer The State University of New York 8/1/2013

2 What is Big Data? 8/1/2013 Draft for Discussion 2

3 Big Data 250 Years Ago 8/1/2013 Draft for Discussion 3

6 Big Data Now 8/1/2013 Draft for Discussion 6

7 The Way People Live/Learn Has Changed 8/1/2013 Draft for Discussion 7

8 Characteristics of Big Data 8/1/2013 Draft for Discussion 8

9 The Immediate Value of Big Data 8/1/2013 Draft for Discussion 9

10 Source: 8/1/2013 Draft for Discussion 10

13 Alexandre Gerber, Deirdre W. Paul, James R. Rowland, and Christopher, "The Connected States of America maps communities" 8/1/2013 Draft for Discussion 13

14 8/1/2013 Draft for Discussion 14

16 Figure 1. The geography of talk in Great Britain. Ratti C, Sobolevsky S, Calabrese F, Andris C, et al. (2010) Redrawing the Map of Great Britain from a Network of Human Interactions. PLoS ONE 5(12): e doi: /journal.pone

17 Figure 2. Defining regions through the spectral modularity optimization of telecommunications networks. Ratti C, Sobolevsky S, Calabrese F, Andris C, et al. (2010) Redrawing the Map of Great Britain from a Network of Human Interactions. PLoS ONE 5(12): e doi: /journal.pone

18 Figure 3. The core regions of Britain. Ratti C, Sobolevsky S, Calabrese F, Andris C, et al. (2010) Redrawing the Map of Great Britain from a Network of Human Interactions. PLoS ONE 5(12): e doi: /journal.pone

19 Alexandre Gerber, Deirdre W. Paul, James R. Rowland, and Christopher, "The Connected States of America maps communities" 8/1/2013 Draft for Discussion 19

24 SUNY Scales 8/1/2013 Draft for Discussion 24

25 Six Big Ideas 8/1/2013 Draft for Discussion 25

26 SUNY Model for Shared Big Data Capability 8/1/2013 Draft for Discussion 26

27 Data Quality To protect our data and privacy, we start with Data and Information Governance Stage 1 Undisciplined Stage 2 Reactive Stage 3 Proactive Stage 4 Governed Few defined rules and policies regarding data Addresses data issues only after they occur Same data may exist in multiple applications Redundant data is often present Enterprise applications are non-integrated, disparate point solutions Implementation of a Master Data Management Plan Posses a unified view of data Unified Data and Information Governance strategy throughout the entire organization Data quality, integration and synchronization are integral to all business processes Adoption of Data and Information Governance Policies Stage 1 2 Establish objectives for Data and Information Governance Establish a baseline Stage 2 3 Processes in place to correct & consolidate data Formation of crossfunctional business rules Proactive data monitoring Adopted from The Data Governance Maturity Model, A DataFlux White Paper. Stage 3 4 Full integration of enterprise applications Creation of Center of Excellence Business Process Management 8/1/2013 Hao Wang, Ph.D., MPA, SUNY CIO 27

28 From Undisciplined to Governed Undisciplined Organization Dimension: People Success depends on the competence of a few individuals Business analysts are removed from development of data quality rules The organization relies on personnel who may follow different paths within each effort to reconcile and correct data No management input or buy-in on data quality problems Executives are unaware of data problems or blame IT entirely Governed Organization Dimension: People Data and Information Governance has executive-level sponsorship with direct CEO support Business users take an active role in data strategy and delivery A data quality or Data and Information Governance group works directly with Data and Information Stewards, application developers, and database administrators The organization has zero defect policies for data collection and management Adopted from The Data Governance Maturity Model, A DataFlux White Paper. 8/1/2013 Hao Wang, Ph.D., MPA, SUNY CIO 28

29 From Undisciplined to Governed Undisciplined Organization Dimension: Policies Data quality is non-existent or projectfocused only, with no defined data quality processes Data and data processing is siloed systems operate independently Firefighting mode problems are addressed as they occur through manually-driven processes Resources are not optimized due to redundant and outdated data Governed Organization Dimension: Policies New initiatives are only approved after careful consideration of how the initiatives will impact the existing data infrastructure Automated policies are in place to ensure that data remains consistent, accurate and reliable throughout the organization A service oriented architecture (SOA) encapsulates business rules for data quality and identity management Adopted from The Data Governance Maturity Model, A DataFlux White Paper. 8/1/2013 Hao Wang, Ph.D., MPA, SUNY CIO 29

30 From Undisciplined to Governed Undisciplined Organization Dimension: Technology Little or no data profiling, analysis or auditing Data cleansing and standardization occurs only in isolated data sources Data improvement is focused on single applications Governed Organization Dimension: Technology Data quality and data integration tools are standardized across the organization All aspects of the organization use standard business rules created and maintained by designated Data and Information Stewards Data is continuously inspected and any deviations from standards are resolved immediately Data models capture the business meaning and technical details of all corporate data elements Adopted from The Data Governance Maturity Model, A DataFlux White Paper. 8/1/2013 Hao Wang, Ph.D., MPA, SUNY CIO 30

31 From Undisciplined to Governed Undisciplined Organization Dimension: Risk and Reward Risk: Extremely High - Data problems result in lost information or improper procedures. A few scapegoats receive the blame, although it is impossible to accurately assign culpability Reward: Low Outside of the success of a single employee, organizations reap few benefits from data quality Governed Organization Dimension: Risk and Reward Risk: Low Master data tightly controlled across the organization, allowing the organization to maintain high-quality information about its patients and its services. Reward: High Organizational data practices can lead to a better understanding about the organization's current business landscape allowing management to have full confidence in all decisions based on data Adopted from The Data Governance Maturity Model, A DataFlux White Paper. 8/1/2013 Hao Wang, Ph.D., MPA, SUNY CIO 31

32 How to Start DIG at SUNY? Identify high value data (HVD) for SUNY and build a centralized repository and necessary work flows to collect, curate, protect, and disseminate these data. Identify Data and Information Stewards (the owners or custodians of the data assets in the enterprise). Establish policies that specifies who is accountable for various portions or aspects of the data and information, including its accuracy, accessibility, consistency, completeness, and maintenance. Define processes concerning how data and information are stored, archived, backed up, and protected. Develop standards and procedures that define how the data and information are used by authorized personnel. Implement a set of audit and control procedures to ensure ongoing compliance with governmental regulations and industrial standards. 8/1/2013 Hao Wang, Ph.D., MPA, SUNY CIO 32

33 Objectives 1. Improve data and information services for students, faculty, researchers, and administrators 2. Take advantage of our data to support improved operations (e.g., financial aid and student billing) as well as improved analytics / research 3. Properly manage how we collect, maintain, and interpret our data 4. Better address compliance, regulatory, and security concerns; reduce risks associated with managing data assets in higher education systems 5. Improve the tools available to us to manage data standards and practices 6. Cross-campus collaboration across functions, systems, campuses 8/1/2013 Hao Wang, Ph.D., MPA, SUNY CIO 33

34 SUNY DIG Constructs - Mission Mission & Scope Authority Structure Membership Process Mission 1. Lead the better use of High Value Data (HVD) at SUNY to support higher education and economic development 2. Improve data and information management practices and services through: a. Data and Information Stewardship b. Processes, Procedures, and Continuous Performance Improvement c. Control, Audit, and Change Management 3. Establish data and information management policies that improve efficiency and quality of service through improved data quality 4. Adopt data and information standards 5. Foster the universal adoption of data management policies 8/1/2013 Hao Wang, Ph.D., MPA, SUNY CIO 34

35 SUNY DIG Constructs - Scope Mission & Scope Authority Structure Membership Process Scope 1. Management for all Key Information Assets (such as MDM) a. Student Services, Faculty Services, Research, Financial Management, Assessment and Reporting, Operations, Administrative Services 2. Enhanced Data Quality and Data Standards a. All data systems and processes within SUNY s responsibility (e.g., student systems, billing, analytics, reporting, etc.), with the understanding that community colleges have different requirements than the rest of SUNY b. Facilitate improved data management practices with all external entities that do business with SUNY c. Conformance to National Standards 3. Manage cost of data as part of lifecycle management 8/1/2013 Hao Wang, Ph.D., MPA, SUNY CIO 35

36 SUNY DIG Constructs - Authority Mission & Scope Authority Structure Membership Process The Data and Information Governance Board is established by the authority of the Chancellor and SUNY executive team. It has the responsibility to: Establish data and information policies Define and prioritize SUNY s data and information management objectives Define the authoritative sources for data and information Develop and enforce data and information management policies and procedures 8/1/2013 Hao Wang, Ph.D., MPA, SUNY CIO 36

37 SUNY DIG Constructs - Structure Data and Information Governance Board - Defines and approve data management policies and data quality standards - Prioritizes the order in which SUNY-wide data issues are addresses - Identify the high value data (HVD) in SUNY systems and approve appropriate use of HVD Data and Information Stewards and SUNY Information Domain Workgroup Identified from and for all critical lines of business of SUNY - Conduct detailed analysis to assess data quality issues - Facilitate and manage data cleansing activities - Develop standards for data elements within their domain - Enforce policies & procedures - Develop metrics to monitor data quality - Define and interpret business rules Data and Information Governance Program Team - Administers MDM project - Ensures adequate security - Provides technical support - Incorporates Data Governance into - Implement software solutions Project & Software Lifecycles - Facilitates the adoption of Data and Information Governance policies and procedures Master Data Management SUNY Business Intelligence SUNY Information Exchange SUNY Enterprise Projects SUNY Enterprise Architecture SUNY Data Warehouse 8/1/2013 Hao Wang, Ph.D., MPA, SUNY CIO 37

38 MDM is a critical element for Data Governance Good data governance requires the involvement of key stakeholders from data owners to data stewards Data quality depends on accurate and consistent master data management Metadata management and data classification are enablers for process integration MDM is an indispensable part of enterprise architecture that encompass infrastructure, application, data, and business processes Based on the IBM Data Governance Council Framework Master Data Management (MDM) and Data and Information Governance are increasingly converging. Good MDM is a foundation for a sound Data Governance program. 8/1/2013 Draft for Discussion 38

39 System and Campus Needs Decide the Style of SUNY MDM Hub Patterns Transaction, Coexistence, Registry, Monolithic Integration Patterns Information oriented, Business process oriented Support Patterns SIS, Finance, HR, BI, DWH MDM can be Centralized Distributed Federated Multi-tenancy Data Mashup Deciding factors: Business needs Cost benefit Existing investment 8/1/2013 Draft for Discussion 39

40 8/1/2013

41 The big data movement is transforming our world. The 2013 Critical Issues in Higher Education Conference will examine how this movement could help build smarter universities. While much of the cutting edge work being done with big data is happening at colleges and universities, higher education has yet to turn the digital mirror on itself. Keynote Speaker: Harper Reed Former CTO, Obama for America

42 Featured topics: Improving the Student Experience Enhancing the Research Enterprise Data Science Lessons Learned University Infrastructure and Outreach Education Pipeline Academic Planning and Strategy Bringing Big Data to Life Transforming Higher Education REGISTER NOW at SUNY.EDU

43 Contact Hao Wang, Ph.D.,MPA Chief Information Officer The State University of New York State University Plaza - Albany, New York Tel: Fax: Be a part of Generation SUNY: Facebook - Twitter - YouTube activecarbon 8/1/2013 Draft for Discussion 43