Annex: Concept Note. Big Data for Policy, Development and Official Statistics New York, 22 February 2013

Save this PDF as:
 WORD  PNG  TXT  JPG

Size: px
Start display at page:

Download "Annex: Concept Note. Big Data for Policy, Development and Official Statistics New York, 22 February 2013"

Transcription

1 Annex: Concept Note Friday Seminar on Emerging Issues Big Data for Policy, Development and Official Statistics New York, 22 February 2013 How is Big Data different from just very large databases? 1 Traditionally, data processing for analytic purposes followed a fairly static blueprint. Namely, create modest amounts of structured data with stable data models. Data processing analysis and integration tools are used to extract, transform and load the data from enterprises applications and administrative databases to a staging area where data quality and data normalization (hopefully) occur and the data is modeled into neat rows and tables. The modeled, cleansed data is then loaded into an enterprise data warehouse. This routine usually occurs on a scheduled basis usually daily or weekly, monthly or annually, sometimes more frequently. From there, data warehouse administrators create and schedule regular reports to run against normalized data stored in the warehouse or some other dissemination facility, which are distributed to a wide range of users in government, business, the media and the community at large.. They also create dashboards and other limited visualization tools for executives and management. Analysts, meanwhile, use data analytics tools/engines to run more advanced analytics against the warehouse or other dissemination facility, or more often against sample data migrated to a local data mart due to size limitations. Nonexpert users perform basic data visualization and limited analytics against the data warehouse via front-end business intelligence tools. Data volumes in traditional data warehouses rarely exceeded multiple terabytes (and even that much is rare) as large volumes of data strain warehouse resources and degrade performance. The changing nature of Big Data The advent of the Web, mobile devices and other technologies such as sensor networks has caused a fundamental change to the nature of data. Big Data has important, distinct qualities that differentiate it from traditional institutional data. 1 Most of the shown text comes from a Big Data Manifesto from the Wikibon Community by Jeff Kelly, see 1

2 Data are no longer centralized, highly structured and easily manageable, but are highly distributed, loosely structured (if structured at all), and increasingly large in volume. Source: Microsoft Specifically: Volume The amount of data created both inside corporations and outside the firewall via the web, mobile devices, IT infrastructure, and other sources is increasing exponentially each year. Type The variety of data types is increasing, namely unstructured text-based data and semi-structured data like social media data, location-based data, and log-file data. Speed The speed at which new data is being created and the need for real-time analytics to derive business value from it -- is increasing thanks to digitization of transactions, the emergence of sensor networks, mobile computing and the sheer number of internet and mobile device users. Broadly speaking, Big Data is generated by a range of sources, including: Mobile Devices: There are over 5 billion mobile phones in use worldwide. Each call, text and instant message is logged as data. Mobile devices, particularly smart phones 2

3 and tablets, also make it easier to use social media and use other data-generating applications. Mobile devices also collect and transmit location data. Internet Transactions: Billions of online purchases, funds transfers, stock trades and other transactions happen every day, including countless automated transactions. Each creates a number of data points collected by retailers, banks, credit card issuers, credit agencies and others. Networked Devices and Sensors: Electronic devices of all sorts including servers and other IT hardware, smart energy meters and temperature and other sensors -- all create semi-structured log data that record every action. Social Networking and Media: There are currently over 700 million Facebook users, 250 million Twitter users and 156 million public blogs. Each Facebook update, Tweet, blog post and comment creates multiple new data points, both structured, semistructured and unstructured, sometimes called Data Exhaust. Source: The Informatica Blog New approaches to Big Data processing and analytics Traditional data warehouses and other data management tools are not designed for processing and analyzing Big Data in a time- or cost-efficient manner. Namely, data 3

4 must be organized into relational tables -- neat rows and columns -- before a traditional enterprise data warehouse can ingest it. Due to the time and man-power needed, applying such structure to vast amounts of unstructured data is impractical. Further, in order to scale-up a traditional enterprise data warehouse to accommodate potentially petabytes of data would require unrealistic financial investments in new, often (depending on the vendor) proprietary hardware. Data warehouse performance would also suffer due to a single choke point for loading data. Therefore new ways of processing and analyzing Big Data are required. There are number of approaches to processing and analyzing Big Data, but most have some common characteristics. Namely, they take advantage of commodity hardware to enable scale-out, parallel processing techniques; employ non-relational data storage capabilities in order to process unstructured and semi-structured data; and apply advanced analytics and data visualization technology to Big Data to convey insights to end-users. Source: Wikibon 2012 In order to fully take advantage of Big Data, however, enterprises must take further steps. Namely, they must employ staff with the knowledge and skills to deploy advanced analytics techniques on the processed data to reveal meaningful insights. People with the knowledge and skills are often now described as Data Scientists 4

5 performing this sophisticated work in one of a handful of languages or approaches, including HADOOP, SAS and R. The results of this analysis can then be operationalized via Big Data applications, either homegrown or off-the-shelf. Other vendors are developing business intelligence-style applications to allow non-power users to interact with Big Data directly. The context of Official Statistics National Statistical Offices have started to explore how best to harness this phenomenon of Big Data in their mission to supply quality statistics for improving economic performance, social well-being and environmental sustainability. Some of the issues 2 raised are: Should NSOs expand its business operations to take on the opportunities of using Big Data for official government purposes? Should NSOs take on a new mission as a trusted 3rd party whose role would be to certify the statistical quality of many of these newly emerging private sector sources? Should NSOs become a clearing house for statistics from non-traditional sources that meet their quality standards? Should NSOs use non-traditional sources to supplement (and perhaps replace) their official series? How might NSOs acquire people with the knowledge and skills to effectively take advantage of Big Data for official statistics purposes? For example, the billion Price Project collects price information over the internet and computes a price index to estimate inflation. The index is published daily with a three day lag as opposed to the official inflation numbers which are published monthly with a an even longer lag. A quick turn-around allows for early detection of inflation trends and may allow policy makers to tailor policies in a much more timely manner. If governments wanted to, they could already let Big Data play a role in providing some information on areas that are currently under the responsibility of national statistical offices (NSOs). 2 These issues are being considered by the High-Level Group for Strategic Developments in Business Architecture in Statistics which reports to the Conference of European Statisticians. 5

6 The attraction of Big Data lies in the sheer amount of data which could be available in, or near, real time. Potentially, Big Data could be used as intelligence to better solve emergency situations. Satellite imaging or information gathered from mobile devices can be used both in developed and developing countries. Big Data presents an opportunity for the official statistical community to better meet its mission of disseminating timely and quality statistics. Building on the experiences of the private and public sector, NSOs and national statistical and international statistical systems more generally have an opportunity to expand into an area that could provide a new range of relevant information in a timely manner. The use of Big Data has a number of upsides but also many challenges related to security, privacy, analysis and interpretation. Analyses and results emerging from the use of Big Data should be properly checked and documented for their quality, validity and limitations. Practical challenges with Big Data are using commercial infrastructure (capacity and computational power) to store, mine and analyse Big Data and developing the appropriate enterprise architectures within statistical organizations. Moving from traditional data collection to procurement and use of Big Data, the statistical community will also require need to address the skill gap around Big Data administration and Big Data Analytics, or Data Science. In order for Big Data to truly gain mainstream adoption and achieve its full potential for official statistical purposes, it is critical that the statistical community does not ignore Big Data, but recognizes the use Big Data as part of their information management model, prepares an inventory of the state of play and formulates the implications for official statistics. 6

www.pwc.com/oracle Next presentation starting soon Business Analytics using Big Data to gain competitive advantage

www.pwc.com/oracle Next presentation starting soon Business Analytics using Big Data to gain competitive advantage www.pwc.com/oracle Next presentation starting soon Business Analytics using Big Data to gain competitive advantage If every image made and every word written from the earliest stirring of civilization

More information

The Big Data Manifesto

The Big Data Manifesto Wikibon.com - http://wikibon.com The Big Data Manifesto by Jeff Kelly - 12 January 2012 http://wikibon.com/big-data-manifesto/ 1 / 11 A Big Data Manifesto from the Wikibon Community Providing effective

More information

Busin i ess I n I t n e t ll l i l g i e g nce c T r T e r nds For 2013

Busin i ess I n I t n e t ll l i l g i e g nce c T r T e r nds For 2013 Business Intelligence Trends For 2013 10 Trends The last few years the change in Business Intelligence seems to accelerate under the pressure of increased business demand and technology innovations. Here

More information

Big Data. Fast Forward. Putting data to productive use

Big Data. Fast Forward. Putting data to productive use Big Data Putting data to productive use Fast Forward What is big data, and why should you care? Get familiar with big data terminology, technologies, and techniques. Getting started with big data to realize

More information

5 Keys to Unlocking the Big Data Analytics Puzzle. Anurag Tandon Director, Product Marketing March 26, 2014

5 Keys to Unlocking the Big Data Analytics Puzzle. Anurag Tandon Director, Product Marketing March 26, 2014 5 Keys to Unlocking the Big Data Analytics Puzzle Anurag Tandon Director, Product Marketing March 26, 2014 1 A Little About Us A global footprint. A proven innovator. A leader in enterprise analytics for

More information

Beyond Web Application Log Analysis using Apache TM Hadoop. A Whitepaper by Orzota, Inc.

Beyond Web Application Log Analysis using Apache TM Hadoop. A Whitepaper by Orzota, Inc. Beyond Web Application Log Analysis using Apache TM Hadoop A Whitepaper by Orzota, Inc. 1 Web Applications As more and more software moves to a Software as a Service (SaaS) model, the web application has

More information

Testing Big data is one of the biggest

Testing Big data is one of the biggest Infosys Labs Briefings VOL 11 NO 1 2013 Big Data: Testing Approach to Overcome Quality Challenges By Mahesh Gudipati, Shanthi Rao, Naju D. Mohan and Naveen Kumar Gajja Validate data quality by employing

More information

Using Tableau Software with Hortonworks Data Platform

Using Tableau Software with Hortonworks Data Platform Using Tableau Software with Hortonworks Data Platform September 2013 2013 Hortonworks Inc. http:// Modern businesses need to manage vast amounts of data, and in many cases they have accumulated this data

More information

INTELLIGENT BUSINESS STRATEGIES WHITE PAPER

INTELLIGENT BUSINESS STRATEGIES WHITE PAPER INTELLIGENT BUSINESS STRATEGIES WHITE PAPER Improving Access to Data for Successful Business Intelligence Part 2: Supporting Multiple Analytical Workloads in a Changing Analytical Landscape By Mike Ferguson

More information

ANALYTICS BUILT FOR INTERNET OF THINGS

ANALYTICS BUILT FOR INTERNET OF THINGS ANALYTICS BUILT FOR INTERNET OF THINGS Big Data Reporting is Out, Actionable Insights are In In recent years, it has become clear that data in itself has little relevance, it is the analysis of it that

More information

III Big Data Technologies

III Big Data Technologies III Big Data Technologies Today, new technologies make it possible to realize value from Big Data. Big data technologies can replace highly customized, expensive legacy systems with a standard solution

More information

BIG DATA: FIVE TACTICS TO MODERNIZE YOUR DATA WAREHOUSE

BIG DATA: FIVE TACTICS TO MODERNIZE YOUR DATA WAREHOUSE BIG DATA: FIVE TACTICS TO MODERNIZE YOUR DATA WAREHOUSE Current technology for Big Data allows organizations to dramatically improve return on investment (ROI) from their existing data warehouse environment.

More information

Managing Big Data with Hadoop & Vertica. A look at integration between the Cloudera distribution for Hadoop and the Vertica Analytic Database

Managing Big Data with Hadoop & Vertica. A look at integration between the Cloudera distribution for Hadoop and the Vertica Analytic Database Managing Big Data with Hadoop & Vertica A look at integration between the Cloudera distribution for Hadoop and the Vertica Analytic Database Copyright Vertica Systems, Inc. October 2009 Cloudera and Vertica

More information

Accelerate BI Initiatives With Self-Service Data Discovery And Integration

Accelerate BI Initiatives With Self-Service Data Discovery And Integration A Custom Technology Adoption Profile Commissioned By Attivio June 2015 Accelerate BI Initiatives With Self-Service Data Discovery And Integration Introduction The rapid advancement of technology has ushered

More information

Microsoft Big Data. Solution Brief

Microsoft Big Data. Solution Brief Microsoft Big Data Solution Brief Contents Introduction... 2 The Microsoft Big Data Solution... 3 Key Benefits... 3 Immersive Insight, Wherever You Are... 3 Connecting with the World s Data... 3 Any Data,

More information

Apache Hadoop: The Big Data Refinery

Apache Hadoop: The Big Data Refinery Architecting the Future of Big Data Whitepaper Apache Hadoop: The Big Data Refinery Introduction Big data has become an extremely popular term, due to the well-documented explosion in the amount of data

More information

Architecting for the Internet of Things & Big Data

Architecting for the Internet of Things & Big Data Architecting for the Internet of Things & Big Data Robert Stackowiak, Oracle North America, VP Information Architecture & Big Data September 29, 2014 Safe Harbor Statement The following is intended to

More information

BIG DATA-AS-A-SERVICE

BIG DATA-AS-A-SERVICE White Paper BIG DATA-AS-A-SERVICE What Big Data is about What service providers can do with Big Data What EMC can do to help EMC Solutions Group Abstract This white paper looks at what service providers

More information

Getting Started Practical Input For Your Roadmap

Getting Started Practical Input For Your Roadmap Getting Started Practical Input For Your Roadmap Mike Ferguson Managing Director, Intelligent Business Strategies BA4ALL Big Data & Analytics Insight Conference Stockholm, May 2015 About Mike Ferguson

More information

Wikibon Big Data Analytics Adoption Survey, 2014-2015 Frequency Analysis

Wikibon Big Data Analytics Adoption Survey, 2014-2015 Frequency Analysis Wikibon.com - http://wikibon.com Wikibon Big Data Analytics Adoption Survey, 2014-2015 Frequency Analysis by Jeff Kelly - 1 July 2014 http://wikibon.com/wikibon-big-data-analytics-adoption-survey-2014-2015-frequency-analysis/

More information

There s no way around it: learning about Big Data means

There s no way around it: learning about Big Data means In This Chapter Chapter 1 Introducing Big Data Beginning with Big Data Meeting MapReduce Saying hello to Hadoop Making connections between Big Data, MapReduce, and Hadoop There s no way around it: learning

More information

What happens when Big Data and Master Data come together?

What happens when Big Data and Master Data come together? What happens when Big Data and Master Data come together? Jeremy Pritchard Master Data Management fgdd 1 What is Master Data? Master data is data that is shared by multiple computer systems. The Information

More information

Management Accountants and IT Professionals providing Better Information = BI = Business Intelligence. Peter Simons peter.simons@cimaglobal.

Management Accountants and IT Professionals providing Better Information = BI = Business Intelligence. Peter Simons peter.simons@cimaglobal. Management Accountants and IT Professionals providing Better Information = BI = Business Intelligence Peter Simons peter.simons@cimaglobal.com Agenda Management Accountants? The need for Better Information

More information

P u b l i c a t i o n N u m b e r : W P 0 0 0 0 0 0 0 4 R e v. A

P u b l i c a t i o n N u m b e r : W P 0 0 0 0 0 0 0 4 R e v. A P u b l i c a t i o n N u m b e r : W P 0 0 0 0 0 0 0 4 R e v. A FileTek, Inc. 9400 Key West Avenue Rockville, MD 20850 Phone: 301.251.0600 International Headquarters: FileTek Ltd 1 Northumberland Avenue

More information

Business Analytics In a Big Data World Ted Malone Solutions Architect Data Platform and Cloud Microsoft Federal

Business Analytics In a Big Data World Ted Malone Solutions Architect Data Platform and Cloud Microsoft Federal Business Analytics In a Big Data World Ted Malone Solutions Architect Data Platform and Cloud Microsoft Federal Information has gone from scarce to super-abundant. That brings huge new benefits. The Economist

More information

INTRODUCTION TO CASSANDRA

INTRODUCTION TO CASSANDRA INTRODUCTION TO CASSANDRA This ebook provides a high level overview of Cassandra and describes some of its key strengths and applications. WHAT IS CASSANDRA? Apache Cassandra is a high performance, open

More information

Big Data Market Size and Vendor Revenues

Big Data Market Size and Vendor Revenues Analysis from The Wikibon Project February 2012 Big Data Market Size and Vendor Revenues Jeff Kelly, David Vellante, David Floyer A Wikibon Reprint The Big Data market is on the verge of a rapid growth

More information

Microsoft Analytics Platform System. Solution Brief

Microsoft Analytics Platform System. Solution Brief Microsoft Analytics Platform System Solution Brief Contents 4 Introduction 4 Microsoft Analytics Platform System 5 Enterprise-ready Big Data 7 Next-generation performance at scale 10 Engineered for optimal

More information

Information Architecture

Information Architecture The Bloor Group Actian and The Big Data Information Architecture WHITE PAPER The Actian Big Data Information Architecture Actian and The Big Data Information Architecture Originally founded in 2005 to

More information

International Journal of Advancements in Research & Technology, Volume 3, Issue 5, May-2014 18 ISSN 2278-7763. BIG DATA: A New Technology

International Journal of Advancements in Research & Technology, Volume 3, Issue 5, May-2014 18 ISSN 2278-7763. BIG DATA: A New Technology International Journal of Advancements in Research & Technology, Volume 3, Issue 5, May-2014 18 BIG DATA: A New Technology Farah DeebaHasan Student, M.Tech.(IT) Anshul Kumar Sharma Student, M.Tech.(IT)

More information

Taming Big Data. 1010data ACCELERATES INSIGHT

Taming Big Data. 1010data ACCELERATES INSIGHT Taming Big Data 1010data ACCELERATES INSIGHT Lightning-fast and transparent, 1010data analytics gives you instant access to all your data, without technical expertise or expensive infrastructure. TAMING

More information

Big Data Defined Introducing DataStack 3.0

Big Data Defined Introducing DataStack 3.0 Big Data Big Data Defined Introducing DataStack 3.0 Inside: Executive Summary... 1 Introduction... 2 Emergence of DataStack 3.0... 3 DataStack 1.0 to 2.0... 4 DataStack 2.0 Refined for Large Data & Analytics...

More information

Keywords Big Data, NoSQL, Relational Databases, Decision Making using Big Data, Hadoop

Keywords Big Data, NoSQL, Relational Databases, Decision Making using Big Data, Hadoop Volume 4, Issue 1, January 2014 ISSN: 2277 128X International Journal of Advanced Research in Computer Science and Software Engineering Research Paper Available online at: www.ijarcsse.com Transitioning

More information

BIG DATA FUNDAMENTALS

BIG DATA FUNDAMENTALS BIG DATA FUNDAMENTALS Timeframe Minimum of 30 hours Use the concepts of volume, velocity, variety, veracity and value to define big data Learning outcomes Critically evaluate the need for big data management

More information

DATA MANAGEMENT FOR THE INTERNET OF THINGS

DATA MANAGEMENT FOR THE INTERNET OF THINGS DATA MANAGEMENT FOR THE INTERNET OF THINGS February, 2015 Peter Krensky, Research Analyst, Analytics & Business Intelligence Report Highlights p2 p4 p6 p7 Data challenges Managing data at the edge Time

More information

The Definitive Guide to Data Blending. White Paper

The Definitive Guide to Data Blending. White Paper The Definitive Guide to Data Blending White Paper Leveraging Alteryx Analytics for data blending you can: Gather and blend data from virtually any data source including local, third-party, and cloud/ social

More information

From Spark to Ignition:

From Spark to Ignition: From Spark to Ignition: Fueling Your Business on Real-Time Analytics Eric Frenkiel, MemSQL CEO June 29, 2015 San Francisco, CA What s in Store For This Presentation? 1. MemSQL: A real-time database for

More information

Big Data at Cloud Scale

Big Data at Cloud Scale Big Data at Cloud Scale Pushing the limits of flexible & powerful analytics Copyright 2015 Pentaho Corporation. Redistribution permitted. All trademarks are the property of their respective owners. For

More information

We are Big Data A Sonian Whitepaper

We are Big Data A Sonian Whitepaper EXECUTIVE SUMMARY Big Data is not an uncommon term in the technology industry anymore. It s of big interest to many leading IT providers and archiving companies. But what is Big Data? While many have formed

More information

TAMING THE BIG CHALLENGE OF BIG DATA MICROSOFT HADOOP

TAMING THE BIG CHALLENGE OF BIG DATA MICROSOFT HADOOP Pythian White Paper TAMING THE BIG CHALLENGE OF BIG DATA MICROSOFT HADOOP ABSTRACT As companies increasingly rely on big data to steer decisions, they also find themselves looking for ways to simplify

More information

Mike Maxey. Senior Director Product Marketing Greenplum A Division of EMC. Copyright 2011 EMC Corporation. All rights reserved.

Mike Maxey. Senior Director Product Marketing Greenplum A Division of EMC. Copyright 2011 EMC Corporation. All rights reserved. Mike Maxey Senior Director Product Marketing Greenplum A Division of EMC 1 Greenplum Becomes the Foundation of EMC s Big Data Analytics (July 2010) E M C A C Q U I R E S G R E E N P L U M For three years,

More information

The 3 questions to ask yourself about BIG DATA

The 3 questions to ask yourself about BIG DATA The 3 questions to ask yourself about BIG DATA Do you have a big data problem? Companies looking to tackle big data problems are embarking on a journey that is full of hype, buzz, confusion, and misinformation.

More information

Big Data, Why All the Buzz? (Abridged) Anita Luthra, February 20, 2014

Big Data, Why All the Buzz? (Abridged) Anita Luthra, February 20, 2014 Big Data, Why All the Buzz? (Abridged) Anita Luthra, February 20, 2014 Defining Big Not Just Massive Data Big data refers to data sets whose size is beyond the ability of typical database software tools

More information

The Future of Data Management

The Future of Data Management The Future of Data Management with Hadoop and the Enterprise Data Hub Amr Awadallah (@awadallah) Cofounder and CTO Cloudera Snapshot Founded 2008, by former employees of Employees Today ~ 800 World Class

More information

BIRT in the World of Big Data

BIRT in the World of Big Data BIRT in the World of Big Data David Rosenbacher VP Sales Engineering Actuate Corporation 2013 Actuate Customer Days Today s Agenda and Goals Introduction to Big Data Compare with Regular Data Common Approaches

More information

What s Trending in Analytics for the Consumer Packaged Goods Industry?

What s Trending in Analytics for the Consumer Packaged Goods Industry? What s Trending in Analytics for the Consumer Packaged Goods Industry? The 2014 Accenture CPG Analytics European Survey Shows How Executives Are Using Analytics, and Where They Expect to Get the Most Value

More information

Analytics: The real-world use of big data

Analytics: The real-world use of big data Findings from the research collaboration of IBM Institute for Business Value and Saïd Business School, University of Oxford Analytics: The real-world use of big data How innovative enterprises extract

More information

Agile Business Intelligence Data Lake Architecture

Agile Business Intelligence Data Lake Architecture Agile Business Intelligence Data Lake Architecture TABLE OF CONTENTS Introduction... 2 Data Lake Architecture... 2 Step 1 Extract From Source Data... 5 Step 2 Register And Catalogue Data Sets... 5 Step

More information

Exploiting Data at Rest and Data in Motion with a Big Data Platform

Exploiting Data at Rest and Data in Motion with a Big Data Platform Exploiting Data at Rest and Data in Motion with a Big Data Platform Sarah Brader, sarah_brader@uk.ibm.com What is Big Data? Where does it come from? 12+ TBs of tweet data every day 30 billion RFID tags

More information

Big Data Primer. Andrew Mendelsohn, SVP Database April 12, 2012

Big Data Primer. Andrew Mendelsohn, SVP Database April 12, 2012 Big Data Primer Andrew Mendelsohn, SVP Database April 12, 2012 Safe Harbor Statement "Safe Harbor" Statement: Statements in this presentation relating to Oracle's future plans, expectations, beliefs, intentions

More information

IBM Data Warehousing and Analytics Portfolio Summary

IBM Data Warehousing and Analytics Portfolio Summary IBM Information Management IBM Data Warehousing and Analytics Portfolio Summary Information Management Mike McCarthy IBM Corporation mmccart1@us.ibm.com IBM Information Management Portfolio Current Data

More information

DISCOVERING AND SECURING SENSITIVE DATA IN HADOOP DATA STORES

DISCOVERING AND SECURING SENSITIVE DATA IN HADOOP DATA STORES DATAGUISE WHITE PAPER SECURING HADOOP: DISCOVERING AND SECURING SENSITIVE DATA IN HADOOP DATA STORES OVERVIEW: The rapid expansion of corporate data being transferred or collected and stored in Hadoop

More information

Alexander Nikov. 5. Database Systems and Managing Data Resources. Learning Objectives. RR Donnelley Tries to Master Its Data

Alexander Nikov. 5. Database Systems and Managing Data Resources. Learning Objectives. RR Donnelley Tries to Master Its Data INFO 1500 Introduction to IT Fundamentals 5. Database Systems and Managing Data Resources Learning Objectives 1. Describe how the problems of managing data resources in a traditional file environment are

More information

IBM AND NEXT GENERATION ARCHITECTURE FOR BIG DATA & ANALYTICS!

IBM AND NEXT GENERATION ARCHITECTURE FOR BIG DATA & ANALYTICS! The Bloor Group IBM AND NEXT GENERATION ARCHITECTURE FOR BIG DATA & ANALYTICS VENDOR PROFILE The IBM Big Data Landscape IBM can legitimately claim to have been involved in Big Data and to have a much broader

More information

Apache Hadoop Patterns of Use

Apache Hadoop Patterns of Use Community Driven Apache Hadoop Apache Hadoop Patterns of Use April 2013 2013 Hortonworks Inc. http://www.hortonworks.com Big Data: Apache Hadoop Use Distilled There certainly is no shortage of hype when

More information

W H I T E P A P E R. Deriving Intelligence from Large Data Using Hadoop and Applying Analytics. Abstract

W H I T E P A P E R. Deriving Intelligence from Large Data Using Hadoop and Applying Analytics. Abstract W H I T E P A P E R Deriving Intelligence from Large Data Using Hadoop and Applying Analytics Abstract This white paper is focused on discussing the challenges facing large scale data processing and the

More information

Three Open Blueprints For Big Data Success

Three Open Blueprints For Big Data Success White Paper: Three Open Blueprints For Big Data Success Featuring Pentaho s Open Data Integration Platform Inside: Leverage open framework and open source Kickstart your efforts with repeatable blueprints

More information

SELLING PROJECTS ON THE MICROSOFT BUSINESS ANALYTICS PLATFORM

SELLING PROJECTS ON THE MICROSOFT BUSINESS ANALYTICS PLATFORM David Chappell SELLING PROJECTS ON THE MICROSOFT BUSINESS ANALYTICS PLATFORM A PERSPECTIVE FOR SYSTEMS INTEGRATORS Sponsored by Microsoft Corporation Copyright 2014 Chappell & Associates Contents Business

More information

Tap into Big Data at the Speed of Business

Tap into Big Data at the Speed of Business SAP Brief SAP Technology SAP Sybase IQ Objectives Tap into Big Data at the Speed of Business A simpler, more affordable approach to Big Data analytics A simpler, more affordable approach to Big Data analytics

More information

Hadoop for Enterprises:

Hadoop for Enterprises: Hadoop for Enterprises: Overcoming the Major Challenges Introduction to Big Data Big Data are information assets that are high volume, velocity, and variety. Big Data demands cost-effective, innovative

More information

A New Era Of Analytic

A New Era Of Analytic Penang egovernment Seminar 2014 A New Era Of Analytic Megat Anuar Idris Head, Project Delivery, Business Analytics & Big Data Agenda Overview of Big Data Case Studies on Big Data Big Data Technology Readiness

More information

Data Refinery with Big Data Aspects

Data Refinery with Big Data Aspects International Journal of Information and Computation Technology. ISSN 0974-2239 Volume 3, Number 7 (2013), pp. 655-662 International Research Publications House http://www. irphouse.com /ijict.htm Data

More information

End to End Solution to Accelerate Data Warehouse Optimization. Franco Flore Alliance Sales Director - APJ

End to End Solution to Accelerate Data Warehouse Optimization. Franco Flore Alliance Sales Director - APJ End to End Solution to Accelerate Data Warehouse Optimization Franco Flore Alliance Sales Director - APJ Big Data Is Driving Key Business Initiatives Increase profitability, innovation, customer satisfaction,

More information

APICS INSIGHTS AND INNOVATIONS EXPLORING THE BIG DATA REVOLUTION

APICS INSIGHTS AND INNOVATIONS EXPLORING THE BIG DATA REVOLUTION APICS INSIGHTS AND INNOVATIONS EXPLORING THE BIG DATA REVOLUTION APICS INSIGHTS AND INNOVATIONS ABOUT THIS REPORT Big data is a collection of data and technology that accesses, integrates and reports all

More information

Deploying Big Data to the Cloud: Roadmap for Success

Deploying Big Data to the Cloud: Roadmap for Success Deploying Big Data to the Cloud: Roadmap for Success James Kobielus Chair, CSCC Big Data in the Cloud Working Group IBM Big Data Evangelist. IBM Data Magazine, Editor-in- Chief. IBM Senior Program Director,

More information

Eric.kavanagh@bloorgroup.com. Twitter Tag: #briefr 8/14/12

Eric.kavanagh@bloorgroup.com. Twitter Tag: #briefr 8/14/12 Eric.kavanagh@bloorgroup.com Twitter Tag: #briefr 8/14/12 ! Reveal the essential characteristics of enterprise software, good and bad! Provide a forum for detailed analysis of today s innovative technologies!

More information

Extend your analytic capabilities with SAP Predictive Analysis

Extend your analytic capabilities with SAP Predictive Analysis September 9 11, 2013 Anaheim, California Extend your analytic capabilities with SAP Predictive Analysis Charles Gadalla Learning Points Advanced analytics strategy at SAP Simplifying predictive analytics

More information

Self-Service Big Data Analytics for Line of Business

Self-Service Big Data Analytics for Line of Business I D C A N A L Y S T C O N N E C T I O N Dan Vesset Program Vice President, Business Analytics and Big Data Self-Service Big Data Analytics for Line of Business March 2015 Big data, in all its forms, is

More information

Big Data Use Cases. To Start Today. Paul Scholey Sales Director, EMEA. 2013, Pentaho. All Rights Reserved. pentaho.com. Worldwide +1 (866) 660-7555

Big Data Use Cases. To Start Today. Paul Scholey Sales Director, EMEA. 2013, Pentaho. All Rights Reserved. pentaho.com. Worldwide +1 (866) 660-7555 Big Use Cases To Start Today Paul Scholey Sales Director, EMEA 1 Exabytes of We all know the amount of data in the world is growing exponentially 40000 30000 YOU ARE HERE 20000 FROM 2010 TO 2015 77% of

More information

Understanding traffic flow

Understanding traffic flow White Paper A Real-time Data Hub For Smarter City Applications Intelligent Transportation Innovation for Real-time Traffic Flow Analytics with Dynamic Congestion Management 2 Understanding traffic flow

More information

The Future Revolution on Big Data

The Future Revolution on Big Data The Future Revolution on Big Data Abdul Raheem Syed 1, Kumar Gillela 2, Dr. C. Venugopal 3 Assistant Professor, ECM, SNIST, Hyderabad, India 1 Assistant Professor, CSE, MRITS, Hyderabad, India 2 Professor,

More information

BEYOND BI: Big Data Analytic Use Cases

BEYOND BI: Big Data Analytic Use Cases BEYOND BI: Big Data Analytic Use Cases Big Data Analytics Use Cases This white paper discusses the types and characteristics of big data analytics use cases, how they differ from traditional business intelligence

More information

Our Approach To Your Business Intelligence Challenges

Our Approach To Your Business Intelligence Challenges Our Approach To Your Business Intelligence Challenges IntelliCo Can Help You Meet Your BI Needs What can we do for your BI challenges? Tailor BI to meet the needs of your organization Provide professional

More information

InfraStruxure TM Management Software

InfraStruxure TM Management Software InfraStruxure TM Management Software End to end data center infrastructure management software for monitoring and control of power, cooling, security and energy usage from the building through IT systems

More information

Big Analytics: A Next Generation Roadmap

Big Analytics: A Next Generation Roadmap Big Analytics: A Next Generation Roadmap Cloud Developers Summit & Expo: October 1, 2014 Neil Fox, CTO: SoftServe, Inc. 2014 SoftServe, Inc. Remember Life Before The Web? 1994 Even Revolutions Take Time

More information

Real-Time Big Data Analytics + Internet of Things (IoT) = Value Creation

Real-Time Big Data Analytics + Internet of Things (IoT) = Value Creation Real-Time Big Data Analytics + Internet of Things (IoT) = Value Creation January 2015 Market Insights Report Executive Summary According to a recent customer survey by Vitria, executives across the consumer,

More information

Secure Data Transmission Solutions for the Management and Control of Big Data

Secure Data Transmission Solutions for the Management and Control of Big Data Secure Data Transmission Solutions for the Management and Control of Big Data Get the security and governance capabilities you need to solve Big Data challenges with Axway and CA Technologies. EXECUTIVE

More information

Executive Summary... 2 Introduction... 3. Defining Big Data... 3. The Importance of Big Data... 4 Building a Big Data Platform...

Executive Summary... 2 Introduction... 3. Defining Big Data... 3. The Importance of Big Data... 4 Building a Big Data Platform... Executive Summary... 2 Introduction... 3 Defining Big Data... 3 The Importance of Big Data... 4 Building a Big Data Platform... 5 Infrastructure Requirements... 5 Solution Spectrum... 6 Oracle s Big Data

More information

Accenture and Oracle: Leading the IoT Revolution

Accenture and Oracle: Leading the IoT Revolution Accenture and Oracle: Leading the IoT Revolution ACCENTURE AND ORACLE The Internet of Things (IoT) is rapidly moving from concept to reality, as companies see the value of connecting a range of sensors,

More information

Winning with an Intuitive Business Intelligence Solution for Midsize Companies

Winning with an Intuitive Business Intelligence Solution for Midsize Companies SAP Product Brief SAP s for Small Businesses and Midsize Companies SAP BusinessObjects Business Intelligence, Edge Edition Objectives Winning with an Intuitive Business Intelligence for Midsize Companies

More information

Visualizing the Customer Journey Using Analytics. White Paper. Optimizing the Customer Experience. Sponsored by

Visualizing the Customer Journey Using Analytics. White Paper. Optimizing the Customer Experience. Sponsored by Visualizing the Customer Journey Using Analytics Optimizing the Customer Experience White Paper Sponsored by Table of Contents Becoming Customer-Focused 3 Analytics Sharpens the Focus 5 Benefits of an

More information

Danny Wang, Ph.D. Vice President of Business Strategy and Risk Management Republic Bank

Danny Wang, Ph.D. Vice President of Business Strategy and Risk Management Republic Bank Danny Wang, Ph.D. Vice President of Business Strategy and Risk Management Republic Bank Agenda» Overview» What is Big Data?» Accelerates advances in computer & technologies» Revolutionizes data measurement»

More information

Firebird meets NoSQL (Apache HBase) Case Study

Firebird meets NoSQL (Apache HBase) Case Study Firebird meets NoSQL (Apache HBase) Case Study Firebird Conference 2011 Luxembourg 25.11.2011 26.11.2011 Thomas Steinmaurer DI +43 7236 3343 896 thomas.steinmaurer@scch.at www.scch.at Michael Zwick DI

More information

Datenverwaltung im Wandel - Building an Enterprise Data Hub with

Datenverwaltung im Wandel - Building an Enterprise Data Hub with Datenverwaltung im Wandel - Building an Enterprise Data Hub with Cloudera Bernard Doering Regional Director, Central EMEA, Cloudera Cloudera Your Hadoop Experts Founded 2008, by former employees of Employees

More information

SAP Solution Brief SAP HANA. Transform Your Future with Better Business Insight Using Predictive Analytics

SAP Solution Brief SAP HANA. Transform Your Future with Better Business Insight Using Predictive Analytics SAP Brief SAP HANA Objectives Transform Your Future with Better Business Insight Using Predictive Analytics Dealing with the new reality Dealing with the new reality Organizations like yours can identify

More information

Scalable Enterprise Data Integration Your business agility depends on how fast you can access your complex data

Scalable Enterprise Data Integration Your business agility depends on how fast you can access your complex data Transforming Data into Intelligence Scalable Enterprise Data Integration Your business agility depends on how fast you can access your complex data Big Data Data Warehousing Data Governance and Quality

More information

The Next Wave of Data Management. Is Big Data The New Normal?

The Next Wave of Data Management. Is Big Data The New Normal? The Next Wave of Data Management Is Big Data The New Normal? Table of Contents Introduction 3 Separating Reality and Hype 3 Why Are Firms Making IT Investments In Big Data? 4 Trends In Data Management

More information

Vehicle Manufacturer Propels Customer Engagement with Digital Marketing Solutions Insights

Vehicle Manufacturer Propels Customer Engagement with Digital Marketing Solutions Insights Customer Solution Case Study Vehicle Manufacturer Propels Customer Engagement with Digital Marketing Solutions Insights Overview Country or Region: United States Industry: Manufacturing Farm and recreational

More information

Cisco Data Preparation

Cisco Data Preparation Data Sheet Cisco Data Preparation Unleash your business analysts to develop the insights that drive better business outcomes, sooner, from all your data. As self-service business intelligence (BI) and

More information

Converged, Real-time Analytics Enabling Faster Decision Making and New Business Opportunities

Converged, Real-time Analytics Enabling Faster Decision Making and New Business Opportunities Technology Insight Paper Converged, Real-time Analytics Enabling Faster Decision Making and New Business Opportunities By John Webster February 2015 Enabling you to make the best technology decisions Enabling

More information

REAL-TIME OPERATIONAL INTELLIGENCE. Competitive advantage from unstructured, high-velocity log and machine Big Data

REAL-TIME OPERATIONAL INTELLIGENCE. Competitive advantage from unstructured, high-velocity log and machine Big Data REAL-TIME OPERATIONAL INTELLIGENCE Competitive advantage from unstructured, high-velocity log and machine Big Data 2 SQLstream: Our s-streaming products unlock the value of high-velocity unstructured log

More information

CSC590: Selected Topics BIG DATA & DATA MINING. Lecture 2 Feb 12, 2014 Dr. Esam A. Alwagait

CSC590: Selected Topics BIG DATA & DATA MINING. Lecture 2 Feb 12, 2014 Dr. Esam A. Alwagait CSC590: Selected Topics BIG DATA & DATA MINING Lecture 2 Feb 12, 2014 Dr. Esam A. Alwagait Agenda Introduction What is Big Data Why Big Data? Characteristics of Big Data Applications of Big Data Problems

More information

North Highland Data and Analytics. Data Governance Considerations for Big Data Analytics

North Highland Data and Analytics. Data Governance Considerations for Big Data Analytics North Highland and Analytics Governance Considerations for Big Analytics Agenda Traditional BI/Analytics vs. Big Analytics Types of Requiring Governance Key Considerations Information Framework Organizational

More information

TS03: Operational Excellence by Leveraging Internet of Things Technologies

TS03: Operational Excellence by Leveraging Internet of Things Technologies TS03: Operational Excellence by Leveraging Internet of Things Technologies Virendra Chaudhari- Industry Solutions Manufacturing- Microsoft Imtiaz Javeed Product Manager -Visualization & Information Software

More information

InfraStruxure TM Management Software

InfraStruxure TM Management Software InfraStruxure TM Management Software End to end data centre infrastructure management software for monitoring and control of power, cooling, security and energy usage from the building through IT systems

More information

Big Data Are You Ready? Thomas Kyte http://asktom.oracle.com

Big Data Are You Ready? Thomas Kyte http://asktom.oracle.com Big Data Are You Ready? Thomas Kyte http://asktom.oracle.com The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated

More information

Reaping the Rewards of Big Data

Reaping the Rewards of Big Data Reaping the Rewards of Big Data TABLE OF CONTENTS INTRODUCTION: 2 TABLE OF CONTENTS FINDING #1: BIG DATA PLATFORMS ARE ESSENTIAL FOR A MAJORITY OF ORGANIZATIONS TO MANAGE FUTURE BIG DATA CHALLENGES. 4

More information

ENTERPRISE BI AND DATA DISCOVERY, FINALLY

ENTERPRISE BI AND DATA DISCOVERY, FINALLY Enterprise-caliber Cloud BI ENTERPRISE BI AND DATA DISCOVERY, FINALLY Southard Jones, Vice President, Product Strategy 1 AGENDA Market Trends Cloud BI Market Surveys Visualization, Data Discovery, & Self-Service

More information

FIVE INDUSTRIES. Where Big Data Is Making a Difference

FIVE INDUSTRIES. Where Big Data Is Making a Difference FIVE INDUSTRIES Where Big Data Is Making a Difference To understand how Big Data can transform businesses, we have to understand its nature. Although there are numerous definitions of Big Data, many will

More information

W H I T E P A P E R B u s i n e s s I n t e l l i g e n c e S o lutions from the Microsoft and Teradata Partnership

W H I T E P A P E R B u s i n e s s I n t e l l i g e n c e S o lutions from the Microsoft and Teradata Partnership W H I T E P A P E R B u s i n e s s I n t e l l i g e n c e S o lutions from the Microsoft and Teradata Partnership Sponsored by: Microsoft and Teradata Dan Vesset October 2008 Brian McDonough Global Headquarters:

More information