2 Big Data and the Internet of Things Overview of NIST Internet of Things Big Data Public Working Group
4 NIST Bird s eye view The National Institute of Standards and Technology (NIST) is where Nobel Prize-winning science meets realworld engineering. Courtesy HDR Architecture, Inc./Steve Hall Hedrich Blessing With an extremely broad research portfolio, world-class facilities, national networks, and an international reach, NIST works to support industry innovation our central mission.
5 R. Rathe NIST: Basic Stats and Facts Major assets ~ 3,000 employees ~ 2,800 associates and facilities users ~ 1,300 field staff in partner organizations Two main locations: Gaithersburg, Md., and Boulder, Colo. Nobel Prize Winners: 1997, 2001, 2005, 2007, 2013
6 Working with NIST on Cyber Physical Systems Overview of NIST Internet of Things Big Data Public Working Group
7 Internet of Things If we had computers that knew everything there was to know about things using data they gathered without any help from us we would be able to track and count everything, and greatly reduce waste, loss and cost. Kevin Ashton, That 'Internet of Things' Thing, RFID Journal, July 22, 2009
8 Internet of Things What are the defining characteristics of the Internet of Things? Scale Capability Reach
9 Internet of Things - Scale Devices connected to the Web: 1970 = = = 313, = 93,000, = 5,000,000, = 31,000,000,000 Source: Intel
10 Internet of Things - Capability Intel Edison: "It's a full Pentiumclass PC in the form factor of an SD card," Intel CEO Brian Krzanich
11 Internet of Things Reach Virtual Physical
12 Internet Virtual Physical
13 Internet of Things Virtual Physical
14 Cisco Internet of Everything
15 Working with NIST on Cyber Physical Systems Overview of NIST Internet of Things Big Data Public Working Group
16 IBM Smarter Planet For five years, IBMers have been working with companies, cities, and communities around the world to build a Smarter Planet. We ve seen enormous advances, as leaders have begun using the vast supply of Big Data to transform their enterprises and institutions - IBM Source: en_us overview win_in_the_ era_of_smart_op_ad_03_2013.pdf
17 GE Industrial Internet New GE technology merges big iron with big data to create brilliant machines. This convergence of machine and intelligent data is known as the Industrial Internet, and it's changing the way we work. Source:
18 Big Data - Volume IDC April 2014: The Digital Universe of Opportunities From 2013 to 2020, the digital universe will grow by a factor of 10 from 4.4 trillion gigabytes to 44 trillion. It more than doubles every two years. In 2014, the digital universe will equal 1.7 megabytes a minute for every person on Earth. Data from embedded systems will grow from 2% of the digital universe in 2013 to 10% in In 2013, the available storage capacity could hold just 33% of the digital universe. By 2020, it will be able to store less than 15%. Source: IDC Corporation, sponsored by EMC
19 Petabytes Worldwide Big Data - Volume 1,000, , , , ,000 Information 500, , , , ,000 0 Available Storage Source: John Gantz, IDC Corporation, The Expanding Digital Universe
20 Big Data - Velocity Sloan Digital Sky Survey 140 Terabytes, year 2000 to present LSST Large Synoptic Survey Telescope Expect 140 Terabytes every 5 days Square Kilometer Array Expect 140 Terabytes every 3 sec LSST: Suspended between its vast mirrors will be a three billion-pixel sensor array, which on a clear winter night will produce 30 terabytes of data. In less than a week this remarkable telescope will map the whole night sky. And then the next week it will do the same again building up a database of billions of objects and millions of billions of bytes. Nature 440:383
21 Big Data - Variety Combining Structured and Unstructured Data
22 Big Data Potential Value Using detailed survey data on the business practices and information technology investments of 179 large publicly traded firms, we find that firms that adopt DDD [data driven decision making] have output and productivity that is 5-6% higher than what would be expected given their other investments and information technology usage. Brynjolfsson, Erik and Hitt, Lorin M. and Kim, Heekyung Hellen, Strength in Numbers: How Does Data-Driven Decision making Affect Firm Performance? (April 22, 2011).
23 Big Data - Limitations Good Data Won't Guarantee Good Decisions Shvetank Shah, Andrew Horne, and Jaime Capellá Harvard Business Review April 2012 At this very moment, there s an odds-on chance that someone in your organization is making a poor decision on the basis of information that was enormously expensive to collect. Analytical skills are concentrated in too few employees IT needs to spend more time on the I and less on the T Reliable information exists, but it s hard to locate
24 Working with NIST on Cyber Physical Systems Overview of NIST Internet of Things Big Data Public Working Group
25 Big Data - Questions What are the attributes that define Big Data solutions? How is Big Data different from traditional data environments and related applications? What are the essential characteristics of Big Data environments? How do these environments integrate with currently deployed architectures? What are the central scientific, technological, and standardization challenges needed to accelerate the deployment of robust Big Data solutions?
26 NIST Big Data Public Working Group & Standardization Activities Wo Chang, NIST Robert Marcus, ET-Strategies Chaitanya Baru, UC San Diego
27 Big Data PWG - Charter The focus of the (NBD-PWG) is to form a community of interest from industry, academia, and government, with the goal of developing consensus definitions, taxonomies, secure reference architectures, and a technology roadmap. The aim is to create vendorand technology-neutral and infrastructure-agnostic deliverables to enable big data stakeholders to select the best analytics tools for their processing and visualization requirements while enabling value-add from big data service providers and data flow among stakeholders in a cohesive and secure manner.
29 SUBGROUPS Requirements and Use Cases Technology Roadmap NBD-PWG Definitions & Taxonomies Reference Architecture Security and Privacy 29
30 Requirements & Use Cases Geoffrey Fox, U. Indiana Joe Paiva, VA Tsegereda Beyene, Cisco Scope (M0020) The focus is to form a community of interest from industry, academia, and government, with the goal of developing a consensus list of Big Data requirements across all stakeholders. This includes gathering and understanding various use cases from diversified application domains. Tasks Gather input from all stakeholders regarding Big Data requirements. Analyze/prioritize a list of challenging general requirements that may delay or prevent adoption of Big Data deployment Develop a comprehensive list of Big Data requirements 30
31 Requirements and Use Case Subgroup 51 Use Cases Received (http://bigdatawg.nist.gov/usecases.php) 1. Government Operations (4): National Archives & Records Administration, Census Bureau 2. Commercial (8): Finance in Cloud, Cloud Backup, Mendeley (Citations), Netflix, Web Search, Digital Materials, Cargo shipping (e.g. UPS) 3. Defense (3): Sensors, Image Surveillance, Situation Assessment 4. Healthcare & Life Sciences (10): Medical Records, Graph & Probabilistic Analysis, Pathology, Bio-imaging, Genomics, Epidemiology, People Activity Models, Biodiversity 5. Deep Learning & Social Media (6): Driving Car, Geolocate Images, Twitter, Crowd Sourcing, Network Science, NIST Benchmark Datasets 6. Astronomy & Physics (5): Sky Surveys, Large Hadron Collider at CERN, Belle Accelerator II (Japan) 7. Earth, Environmental & Polar Science (10): Ice Sheet Scattering, Earthquake, Ocean, Earth Radar Mapping, Climate Simulation, Atmospheric Turbulence, Subsurface Biogeochemistry, AmeriFlux &FLUXNET gas sensors 8. Energy (10): Smart Grid 31
32 Definitions & Taxonomies Nancy Grady, SAIC Natasha Balac, SDSC Eugene Luster, R2AD Scope (M0018) It is important to develop a consensus-based common language and vocabulary terms used in Big Data across stakeholders from industry, academia, and government. In addition, it is also critical to identify essential actors with roles and responsibilities Tasks For Definitions: Compile terms used from all stakeholders regarding the meaning of Big Data from various standard bodies, domain applications, and diversified operational environments. For Taxonomies: Identify key actors with their roles and responsibilities from all stakeholders, categorize them into components and subcomponents based on their similarities and differences Develop Big Data Definitions and taxonomies documents 32
33 Definitions and Taxonomies Subgroup Big Data consists of extensive datasets, primarily in the characteristics of volume, velocity and/or variety, that require a scalable architecture for efficient storage, manipulation, and analysis. Data Scientist is a practitioner who has sufficient knowledge of the overlapping regimes of expertise in business needs, domain knowledge, analytical skills and programming expertise to manage the end-to-end scientific method process through each stage in the Big Data lifecycle.
34 Reference Architecture Orit Levin, Microsoft James Ketner, AT&T Don Krapohl, Augmented Intelligence Scope (M0021) The goal is to enable Big Data stakeholders to pick-and-choose technology-agnostic analytics tools for processing and visualization in any computing platform and cluster while allowing added value from Big Data service providers and the flow of data between the stakeholders in a cohesive and secure manner. Tasks Gather and study available Big Data architectures representing various stakeholders, different data types, use cases, and document the architectures using the Big Data taxonomies model based upon the identified actors with their roles and responsibilities. Ensure that the developed Big Data reference architecture and the Security and Privacy Reference Architecture correspond and complement each other. 34
35 Reference Architecture Subgroup Key documents: M0151 White Paper M0123 Working Draft M0039 Data Processing Flow M0017 Data Transformation Flow M0047 IT Stack 35
36 Security & Privacy Arnab Roy, CSA/Fujitsu Nancy Landreville, U. MD Akhil Manchanda, GE Scope (M0019) The focus is to form a community of interest from industry, academia, and government, with the goal of developing a consensus secure reference architecture to handle security and privacy issues across all stakeholders. This includes gaining an understanding of what standards are available or under development, as well as identifies which key organizations are working on these standards. Tasks Gather input from all stakeholders regarding security and privacy concerns in Big Data processing, storage, and services. Analyze/prioritize a list of challenging security and privacy requirements that may delay or prevent adoption of Big Data deployment Develop a Security and Privacy Reference Architecture that supplements the general Big Data Reference Architecture 36
37 Security and Privacy Subgroup Requirements Scope Infrastructure Security Data Privacy Data Management Integrity & Reactive Security Requirements Use Cases Studied Retail (consumer) Healthcare Media Government Marketing Architecture & Taxonomies Privacy Provenance System Health 37
38 Technology Roadmap Carl Buffington, USDA/Vistronix Dan McClary, Oracle David Boyd, Data Tactic Scope (M0022) The goal is to develop a consensus vision with recommendations on how Big Data should move forward by performing a good gap analysis through the materials gathered from all other NBD subgroups. This includes setting standardization and adoption priorities through an understanding of what standards are available or under development as part of the recommendations. Tasks Gather input from NBD subgroups and study the taxonomies for the actors roles and responsibility, use cases and requirements, and secure reference architecture. Gain understanding of what standards are available or under development for Big Data Perform a thorough gap analysis and document the findings Identify what possible barriers may delay or prevent adoption of Big Data Document vision and recommendations 38
39 Technology Roadmap Subgroup Key document: M0087 Working Draft Inputs from other subgroups Potential Standards Group with Big Data-related activities (M0035) Capabilities & Technology Readiness Decision Framework Mapping & Gap Analysis Big Data Strategies Definitions & Taxonomies Requiremen ts & Use Cases Security & Privacy Reference Architecture Adoption Implementation Resourcing 39
40 Subgroups Working Draft Outline Contact: Website: Join NBD-PWG: Documents: Working Drafts (under editing) Big Data Definitions & Taxonomies (M0142) NIST Big Data Workshop Slides: Big Data Requirements (M0245) Big Data Security & Privacy Requirements (M0110) Big Data Architectures White Paper Survey (M0151) Big Data Reference Architectures (M0226) Big Data Security & Privacy Reference Architecture (M0110) Big Data Technology Roadmap (M0087) 40
41 The SmartAmerica Challenge Build an integrated Cyber-Physical Systems Framework that allows interconnection of test beds and interoperation through shared data and associated data analytics for easy integration and accelerated adoption of CPS applications. The Arpanet for CPS Innovation Sokwoo Rhee, Geoff Mulligan Presidential Innovation Fellows
42 SmartAmerica Participants Industry GE, IBM, Qualcomm, Intel, Schneider Electric, Philips, AT&T, UTRC, Boeing Research/Educational Institutions MIT, Harvard, UC Berkeley, Vanderbilt, U Penn, UCLA, Internet2, US Ignite, Massachusetts General Hospital Government NIST, NSF, DoT, DoD, DHS, Montgomery County
43 The SmartAmerica Summit June 11, 2014 Washington, DC See:
44 Thank you! Web Sites: bigdatawg.nist.gov Contact:
May 2011 Big data: The next frontier for innovation, competition, and productivity The McKinsey Global Institute The McKinsey Global Institute (MGI), established in 1990, is McKinsey & Company s business
For Big Data Analytics There s No Such Thing as Too Big The Compelling Economics and Technology of Big Data Computing March 2012 By: 4syth.com Emerging big data thought leaders Forsyth Communications 2012.
Global Headquarters: 5 Speen Street Framingham, MA 01701 USA P.508.872.8200 F.508.935.4015 www.idc.com W H I T E P A P E R B i g D a t a : W h a t I t I s a n d W h y Y o u S h o u l d C a r e Sponsored
HARNESSING THE POWER of DIGITAL DATA for SCIENCE AND SOCIETY Report of the Interagency Working Group on Digital Data to the Committee on Science of the National Science and Technology Council January 2009
JANUARY 2013 REPORT OF THE DEFENSE SCIENCE BOARD TASK FORCE ON Cyber Security and Reliability in a Digital Cloud JANUARY 2013 Office of the Under Secretary of Defense for Acquisition, Technology, and Logistics
Big-Data Computing: Creating revolutionary breakthroughs in commerce, science, and society Randal E. Bryant Carnegie Mellon University Randy H. Katz University of California, Berkeley Version 8: December
Emergence and Taxonomy of Big Data as a Service Benoy Bhagattjee Working Paper CISL# 2014-06 May 2014 Composite Information Systems Laboratory (CISL) Sloan School of Management, Room E62-422 Massachusetts
Is Big Data too Big for SMEs? MS&E-238: Leading Trends in Information Technology Stanford University Summer 2014 Carl Johan Rising - SUID: 006000731 Michael Kristensen - SUID: 005999170 Steffen Tjerrild-Hansen
The Industrial Internet@Work Marco Annunziata & Peter C. Evans Table of Contents Executive Summary The Industrial Internet Towards No Unplanned Downtime 3 Introduction A New Information and Collaboration
EUROPEAN COMMISSION Brussels, 2.7.2014 COM(2014) 442 final COMMUNICATION FROM THE COMMISSION TO THE EUROPEAN PARLIAMENT, THE COUNCIL, THE EUROPEAN ECONOMIC AND SOCIAL COMMITTEE AND THE COMMITTEE OF THE
National Spatial Data Infrastructure Strategic Plan 2014 2016 Federal Geographic Data Committee December 2013 Federal Geographic Data Committee Federal Geographic Data Committee, Reston, Virginia: 2013
Cover Page DEMYSTIFYING BIG DATA A Practical Guide To Transforming The Business of Government Prepared by TechAmerica Foundation s Federal Big Data Commission 1 TechAmerica Foundation: Federal Big Data
SAP BusinessObjects Business Intelligence SAP BusinessObjects Business Intelligence 4.0 Solutions Empowering the Real-Time, Mobile, Social, and Global Enterprise SAP BusinessObjects Business Intelligence
2014 www.tmforum.org GEO- $245 USD / free to TM Forum members ANALYTICS QUICK INSIGHTS ADDING VALUE TO BIG DATA Sponsored by: Report prepared for Kathleen Mitchell of TM Forum. No unauthorised sharing.
I D C I V I E W E x t r a c t i n g Value from Chaos June 2011 By John Gantz and David Reinsel Sponsored by EMC Corporation Content for this paper is excerpted directly from the IDC iview "Extracting Value
Designed-In Cyber Security for Cyber-Physical Systems Workshop Report by the Cyber Security Research Alliance 4-5 April 2013 in Gaithersburg, Maryland Co-sponsored with the National Institute of Standards
Digital Built Britain Level 3 Building Information Modelling - Strategic Plan February 2015 1 P a g e Contents 1. Ministerial Foreword 2. Industry Foreword 3. Executive Summary 4. Introduction, Context
TDWI RESE A RCH TDWI BEST PRACTICES REPORT THIRD QUARTER 2011 SELF-SERVICE BUSINESS INTELLIGENCE Empowering Users to Generate Insights By Claudia Imhoff and Colin White CO-SPONSORED BY tdwi.org Third QUARTER
Convergence of Social, Mobile and Cloud: 7 Steps to Ensure Success June, 2013 Contents Executive Overview...4 Business Innovation & Transformation...5 Roadmap for Social, Mobile and Cloud Solutions...7
IBM Software Big Data & Analytics Thought Leadership White Paper Better business outcomes with IBM Big Data & Analytics The insights to transform your business with speed and conviction 2 Better business
32 Big Data: present and future Big Data: present and future Mircea Răducu TRIFU, Mihaela Laura IVAN University of Economic Studies, Bucharest, Romania firstname.lastname@example.org, email@example.com
Does My Infrastructure Look Big In This? A review of Fujitsu s Infrastructure Services Author: Robin Bloor Published: April 2002 Bloor Research Table of Contents Contents Management Summary................................
J U L Y 2 0 1 2 OpenText Enterprise Information Management CIOs are under siege Do more with less is no longer an ideal, it s a mandate. With growing volumes and a host of information formats to manage