The Next Wave of Data Management. Is Big Data The New Normal?

Size: px
Start display at page:

Download "The Next Wave of Data Management. Is Big Data The New Normal?"

Transcription

1 The Next Wave of Data Management Is Big Data The New Normal?

2 Table of Contents Introduction 3 Separating Reality and Hype 3 Why Are Firms Making IT Investments In Big Data? 4 Trends In Data Management Technologies 5 The Technology Maturity Cycle 6 IT Challenges Meeting New Data Demands 7 Managing Increasing Data Volume and Variety 7 Ensuring Data Governance and Data Quality 8 Strategies For Managing Evolving Big (and Enterprise) Data 10 A Total Data Management Approach For All Data 10 Benefits of Total Data Management for All Your Data 11 Talend Total Data Management 12 About Talend 13 Contact Us 13

3 Introduction Much has been written about the promise of big data. With the growth of corporate data, it is clear that big data may become the new normal. In analytics, for example, companies are able to make more informed decisions by analyzing bigger and more diverse sets of data. A recent Gartner report states that through 2015, organizations integrating high value, diverse, new information types and sources into a coherent information management infrastructure will outperform their industry peers financially by more than 20% (Source). However, this new data management journey will bring many instances of failed big data experiments requiring rearchitecture in a few years. Effectively incorporating big data technologies like Hadoop and NoSQL into your current information architecture is not a simple exercise. This paper reviews the business drivers that are impacting this new wave of data management, highlights the new tools companies have for consideration in their next generation data architecture, and outlines a total data management blueprint for consideration in your information strategy. Separating Reality and Hype Is there anything big data cannot solve? Is big data overhyped? Well, many would say yes as big data is being used pervasively in vendor presentations to adorn company information strategies. Big data use cases are ranging from marketing campaign analysis and recommendation engines to predictive analytics, sentiment analysis, and fraud detection. Some were even proposing the use of big data to predict the next Pope as leader of the Catholic Church. What s next you might ask? Well, we are at the tip of the big data iceberg in finding all the value big data can deliver. What is un deniable, is that the digital universe is rapidly growing. For example: The total worldwide volume of data is growing at 59% per year, with the number of files growing at 88% per year. (Source) IDC Estimates that by 2020, business transactions on the internet business to business and business to consumer will reach 450 billion per day. (Source) Akamai analyzes 75 million events per day to better target advertisements. (Source) 800 Bridge Parkway, Suite 200, Redwood City, California US Page 3

4 Walmart handles more than 1 million customer transactions every hour, which is imported into databases estimated to contain more than 2.5 petabytes of data. (Source) More than 5 billion people are calling, texting, tweeting and browsing on mobile phones worldwide. (Source) 100 terabytes of data uploaded daily to Facebook. (Source) Why Are Firms Making IT Investments In Big Data? A recent survey by Talend highlights the top business drivers for big data with increasing the accuracy and depth of predictive analytics being the primary reason. With so much unstructured data not being analyzed, firms recognize the value in using big data. What are the business drivers for big data in your organization? (multiple responses, n=95) 800 Bridge Parkway, Suite 200, Redwood City, California US Page 4

5 Trends In Data Management Technologies Whether big data is in production, in or on the horizon for your organization, there are considerations in several areas of data management that require strategy and planning: Real time Data Processing the requirement to have information on demand or in real time is increasing, as firms do not want to make decisions with last month s or even yesterday s information. Data Services to address the demand for information accessibility and availability from many consumers, data management features such as data integration and data quality are becoming available as on premise or cloud based services. Open Source Data Management to overcome the huge investment typically made in enterprise software, open source makes data management fiscally possible for smaller organizations and improves total cost of ownership for big ones. Data Governance an evolving approach to set forth policies and procedures for the use, access and management of data. Particularly important as business users play an everincreasing role in managing data and budgeting. Big Data and NoSQL providing a bridge between familiar legacy database technologies and the demands of rapidly storing, retrieving, processing and analyzing big data. 800 Bridge Parkway, Suite 200, Redwood City, California US Page 5

6 The Technology Maturity Cycle There is a typical maturity cycle as these new technologies are introduced. First it needs to integrate with existing systems, next it needs to become hardened to address scalability, reliability and security concerns, then it may incorporate related technologies including software lifecycle management features like project management and testing, or in the example of data management, capabilities to expose a function as a consumable service. The technology maturity timeline may look like: Each one of the aforementioned trends is going through the hype to productivity timeline offering greater benefits to end users. If we look at big data (Hadoop) for example, it started as an implementation of MapReduce and has been extended into a massive operating system for distributed parallel processing of huge amounts of data. Other technologies have been added including Hbase, an open source database, HCatalog for meta data, Apache Hive as a data warehouse infrastructure, and Apache Pig as a programming language. However, Hadoop by itself, although well equipped for storing and processing large amounts of information across nodes, does not yet include software lifecycle management functionality, such as project management or related data management functionality such as data integration, metadata management, governance, profiling, cleansing and matching. Data governance is particularly important for successful firm wide adoption, since it incorporates business processes, organizational roles and best practices. Is big data a replacement for a data warehouse? A European telecommunications provider needs to store all call transactions in a database for 2 years to comply with regulations. One alternative is to create a large data warehouse to house this information; however, a less expensive and faster performing alternative is to install a Hadoop cluster or a NoSQL database to store the information. Sometimes called data hoarding, many companies are using this strategy as a means to store information for subsequent processing. 800 Bridge Parkway, Suite 200, Redwood City, California US Page 6

7 IT Challenges Meeting New Data Demands To meet the new business demands and the appetite to process and analyze more information, IT looks to modernize their data management processes. Conventional data management tools fail when trying to integrate, search and analyze very large datasets. In the past, adding hardware and software was a solution to make information available and scalable, but that is no longer the case. Information needs to be available 24x7 across a variety of channels from web to mobile devices. Managing Increasing Data Volume and Variety Much has been written (including this paper so far) about the increasing volume of growing data. IT needs to be concerned since the projected growth of global data generated per year is increasing at 40%; however IT budgets are increasing only at 5% (Source). But what is of greater concern is the increasing variety of data, or the information that is not in a structured database or data warehouse, i.e. content management repositories, streaming data (audio, video), images, blogs, customer comment forums, twitter, network sensors, transactional data and more. By 2015, IDC estimates that 90% of data in the digital universe will be unstructured (Source). It is clear that firms who focus on volume alone and not data characteristics like variety will need to make additional investments. The following survey from Forrester highlights the issue in trying to perform better customer analytics a task that involves integrating many sources of structured and unstructured data. 800 Bridge Parkway, Suite 200, Redwood City, California US Page 7

8 Source: Forrester Webinar with Talend Ensuring Data Governance and Data Quality Depending on the goal of a big data project, poor data quality can have a big impact on effectiveness. It can be argued that inconsistent or invalid data could have exponential impact on analysis in the big data world. If a record enters a system as duplicate, incomplete or incorrect, many other systems are impacted. As analysis on big data grows, so too will the need for validation, standardization, enrichment and resolution of data. Even identification of linkages can be considered a data quality issue that needs to be resolved for big data. As with any corporate data management discipline big data will eventually need to comply with established corporate standards and accepted project management norms for organizational, deployment and sharing of project artifacts. Not too many years ago the terms terabyte (or bytes) and petabyte (or bytes) were seen as very large. A RFP (requirement for proposal) for a data warehouse may have been to reliably handle a petabyte of information. Now terms such as exabyte (or bytes) and zettabyte (or bytes) may soon be the norm. 800 Bridge Parkway, Suite 200, Redwood City, California US Page 8

9 In a recent Gartner report, they state that through 2015, organizations integrating high value, diverse, new information types and sources into a coherent information management infrastructure will outperform their industry peers financially by more than 20%. However, through 2015, 85% of enterprises will fail to adapt their information infrastructure to "big data," socially mediated content and new connected devices. This means that all systems, even the largest integration platform in IT (the data warehouse), will be overwhelmed in the next three to four years (Source). 800 Bridge Parkway, Suite 200, Redwood City, California US Page 9

10 Strategies For Managing Evolving Big (and Enterprise) Data Many companies have built their information architecture department by department to solve specific data management problems. Enterprise data or the data in business systems such as CRM and ERP applications are then linked together by data integration and data quality tools into a data warehouse for further processing, analysis, viewing and management. This is an example of total data management for enterprise data. With the requirement to now include vast amounts of unstructured data for analysis (2 to 10 times structured data), one could take an approach to create a separate big data architecture as another silo. Unfortunately, many of the existing data vendors prescribe this approach since their tools cannot be adapted to support total data management across structured and unstructured data. A problem is created that you are building an information architecture where: there is a lot of duplication across silos with limited reuse; there is inconsistent data governance and data quality practices across the company; and there exists high costs for operations, maintenance and software licenses. Big data should be treated like non big data, i.e. you need to store it, you need to cleanse and enrich it, and you need to use it, which could be retrieving it for a query or analyzing it in a system. The more your systems are integrated, the bigger the potential for data quality and data governance issues. Big data management is not just about deploying a Hadoop cluster, deploying data into a Hadoop file system across distributed nodes and then MapReduce processing. It is making sure your entire information environment incorporates tools to not only handle the increasing volume, velocity, variety and complexity of data, but also provides the software lifecycle management, service enablement and data quality functions as well as data governance procedures. A Total Data Management Approach For All Data Instead of managing silos of structured, unstructured and semi structured data and linking together (bottom up), a recommended approach is to think of managing data holistically (topdown). Your company needs to manage big data just as it does enterprise data, and lets not forget about small data such as spreadsheets. The data can be located on premise, in the cloud, and in a SaaS app. It can be transactional in flight data or clickstream data from your website. You need to profile all of the data, you need to deduplicate and match data, and you need to service enable it 800 Bridge Parkway, Suite 200, Redwood City, California US Page 10

11 for consumption. And often information needs to be processed and available in real time. Total data management is applying the same best practices and tools to all of your data. Fortunately for consumers today, there are good alternatives to create this total data management approach that will not break the bank. Most data management functions have become commodity items, so the buyer can cost efficiently achieve total data management for both structured and un structured data. Benefits of Total Data Management for All Your Data With a total data management approach for all your data, firms can expect to see the following benefits: Improved Data Quality by profiling, matching, deduping and monitoring all your data, you can improve the completeness, accuracy and integrity of your data and therefore the value it will provide to the organization. Improved Accountability having set data governance processes in place for all your data, firms can better measure the use of data, ensure the appropriate access, and set internal policies and procedures. Lower Costs with a decrease in the amount of silos created there will be a decrease in the amount of duplicate processes. With data services reuse, projects can be completed quicker at a lower cost. Firms can also leverage commodity data management tools for most data management functions. 800 Bridge Parkway, Suite 200, Redwood City, California US Page 11

12 Talend Total Data Management Talend provides a unified platform for managing data, no matter if data resides in text files, enterprise databases, spreadmarts or big data clusters like Hadoop. The product set incorporates tools for all your data management needs including: big data integration, data quality and data governance, master data management, business process management and application integration. Talend recognizes that data management is about supporting a wide range of data needs, regardless of scale. Users need not learn to use different environments to manage big data and enterprise data. This saves on infrastructure costs, licensing, user training, project development time and the wide range of costs associated with managing different data scales. For example to integrate big data sources, you use Talend data management graphical tools to integrate big data and NoSQL sources as well as all your enterprise data and small data, and then generate code in Hive, Pig and other native code bases. With big data quality that takes advantage of the massively parallel environment of Hadoop, you can improve the completeness, accuracy and integrity of data as well as remove duplicates. Talend provides big data governance through a simple, intuitive environment for implementing and deploying a big data program with the ability to schedule, monitor and deploy any big data job. 800 Bridge Parkway, Suite 200, Redwood City, California US Page 12

13 About Talend Talend provides integration that truly scales. From small projects to enterprise wide implementations, Talend s highly scalable data, application and business process integration platform maximizes the value of an organization s information assets and optimizes return on investment through a usage based subscription model. Ready for big data environments, Talend s flexible architecture easily adapts to future IT platforms. And a common set of easy to use tools implemented across all Talend products enable teams to scale developer skillsets, too. More than 1,800 active subscribers worldwide leverage Talend s solutions and services. The company has major offices in North America, Europe and Asia, and a global network of technical and services partners. For more information, please visit Contact Us Contact info goes here: Bridge Parkway, Suite 200, Redwood City, California US Page 13

How Big Is Big Data Adoption? Survey Results. Survey Results... 4. Big Data Company Strategy... 6

How Big Is Big Data Adoption? Survey Results. Survey Results... 4. Big Data Company Strategy... 6 Survey Results Table of Contents Survey Results... 4 Big Data Company Strategy... 6 Big Data Business Drivers and Benefits Received... 8 Big Data Integration... 10 Big Data Implementation Challenges...

More information

Talend Big Data. Delivering instant value from all your data. Talend 2014 1

Talend Big Data. Delivering instant value from all your data. Talend 2014 1 Talend Big Data Delivering instant value from all your data Talend 2014 1 I may say that this is the greatest factor: the way in which the expedition is equipped. Roald Amundsen race to the south pole,

More information

Luncheon Webinar Series May 13, 2013

Luncheon Webinar Series May 13, 2013 Luncheon Webinar Series May 13, 2013 InfoSphere DataStage is Big Data Integration Sponsored By: Presented by : Tony Curcio, InfoSphere Product Management 0 InfoSphere DataStage is Big Data Integration

More information

Big data for the Masses The Unique Challenge of Big Data Integration

Big data for the Masses The Unique Challenge of Big Data Integration Big data for the Masses The Unique Challenge of Big Data Integration White Paper Table of contents Executive Summary... 4 1. Big Data: a Big Term... 4 1.1. The Big Data... 4 1.2. The Big Technology...

More information

TAMING THE BIG CHALLENGE OF BIG DATA MICROSOFT HADOOP

TAMING THE BIG CHALLENGE OF BIG DATA MICROSOFT HADOOP Pythian White Paper TAMING THE BIG CHALLENGE OF BIG DATA MICROSOFT HADOOP ABSTRACT As companies increasingly rely on big data to steer decisions, they also find themselves looking for ways to simplify

More information

WHITE PAPER. Four Key Pillars To A Big Data Management Solution

WHITE PAPER. Four Key Pillars To A Big Data Management Solution WHITE PAPER Four Key Pillars To A Big Data Management Solution EXECUTIVE SUMMARY... 4 1. Big Data: a Big Term... 4 EVOLVING BIG DATA USE CASES... 7 Recommendation Engines... 7 Marketing Campaign Analysis...

More information

The Future of Data Management with Hadoop and the Enterprise Data Hub

The Future of Data Management with Hadoop and the Enterprise Data Hub The Future of Data Management with Hadoop and the Enterprise Data Hub Amr Awadallah Cofounder & CTO, Cloudera, Inc. Twitter: @awadallah 1 2 Cloudera Snapshot Founded 2008, by former employees of Employees

More information

Big Data, Why All the Buzz? (Abridged) Anita Luthra, February 20, 2014

Big Data, Why All the Buzz? (Abridged) Anita Luthra, February 20, 2014 Big Data, Why All the Buzz? (Abridged) Anita Luthra, February 20, 2014 Defining Big Not Just Massive Data Big data refers to data sets whose size is beyond the ability of typical database software tools

More information

5 Keys to Unlocking the Big Data Analytics Puzzle. Anurag Tandon Director, Product Marketing March 26, 2014

5 Keys to Unlocking the Big Data Analytics Puzzle. Anurag Tandon Director, Product Marketing March 26, 2014 5 Keys to Unlocking the Big Data Analytics Puzzle Anurag Tandon Director, Product Marketing March 26, 2014 1 A Little About Us A global footprint. A proven innovator. A leader in enterprise analytics for

More information

THE JOURNEY TO A DATA LAKE

THE JOURNEY TO A DATA LAKE THE JOURNEY TO A DATA LAKE 1 THE JOURNEY TO A DATA LAKE 85% OF DATA GROWTH BY 2020 WILL COME FROM NEW TYPES OF DATA ACCORDING TO IDC, AS MUCH AS 85% OF DATA GROWTH BY 2020 WILL COME FROM NEW TYPES OF DATA,

More information

Big Data Realities Hadoop in the Enterprise Architecture

Big Data Realities Hadoop in the Enterprise Architecture Big Data Realities Hadoop in the Enterprise Architecture Paul Phillips Director, EMEA, Hortonworks pphillips@hortonworks.com +44 (0)777 444 3857 Hortonworks Inc. 2012 Page 1 Agenda The Growth of Enterprise

More information

The Future of Data Management

The Future of Data Management The Future of Data Management with Hadoop and the Enterprise Data Hub Amr Awadallah (@awadallah) Cofounder and CTO Cloudera Snapshot Founded 2008, by former employees of Employees Today ~ 800 World Class

More information

Datenverwaltung im Wandel - Building an Enterprise Data Hub with

Datenverwaltung im Wandel - Building an Enterprise Data Hub with Datenverwaltung im Wandel - Building an Enterprise Data Hub with Cloudera Bernard Doering Regional Director, Central EMEA, Cloudera Cloudera Your Hadoop Experts Founded 2008, by former employees of Employees

More information

Mike Maxey. Senior Director Product Marketing Greenplum A Division of EMC. Copyright 2011 EMC Corporation. All rights reserved.

Mike Maxey. Senior Director Product Marketing Greenplum A Division of EMC. Copyright 2011 EMC Corporation. All rights reserved. Mike Maxey Senior Director Product Marketing Greenplum A Division of EMC 1 Greenplum Becomes the Foundation of EMC s Big Data Analytics (July 2010) E M C A C Q U I R E S G R E E N P L U M For three years,

More information

Bringing Together ESB and Big Data

Bringing Together ESB and Big Data Bringing Together ESB and Big Data Bringing Together ESB and Big Data Table of Contents Why ESB and Big Data?...3 Exploring the Promise of Big Data and ESB... 4 Moving Forward With ESB and Big Data...5

More information

III Big Data Technologies

III Big Data Technologies III Big Data Technologies Today, new technologies make it possible to realize value from Big Data. Big data technologies can replace highly customized, expensive legacy systems with a standard solution

More information

HDP Hadoop From concept to deployment.

HDP Hadoop From concept to deployment. HDP Hadoop From concept to deployment. Ankur Gupta Senior Solutions Engineer Rackspace: Page 41 27 th Jan 2015 Where are you in your Hadoop Journey? A. Researching our options B. Currently evaluating some

More information

A Modern Data Architecture with Apache Hadoop

A Modern Data Architecture with Apache Hadoop Modern Data Architecture with Apache Hadoop Talend Big Data Presented by Hortonworks and Talend Executive Summary Apache Hadoop didn t disrupt the datacenter, the data did. Shortly after Corporate IT functions

More information

HDP Enabling the Modern Data Architecture

HDP Enabling the Modern Data Architecture HDP Enabling the Modern Data Architecture Herb Cunitz President, Hortonworks Page 1 Hortonworks enables adoption of Apache Hadoop through HDP (Hortonworks Data Platform) Founded in 2011 Original 24 architects,

More information

Big Data on Microsoft Platform

Big Data on Microsoft Platform Big Data on Microsoft Platform Prepared by GJ Srinivas Corporate TEG - Microsoft Page 1 Contents 1. What is Big Data?...3 2. Characteristics of Big Data...3 3. Enter Hadoop...3 4. Microsoft Big Data Solutions...4

More information

Aligning Your Strategic Initiatives with a Realistic Big Data Analytics Roadmap

Aligning Your Strategic Initiatives with a Realistic Big Data Analytics Roadmap Aligning Your Strategic Initiatives with a Realistic Big Data Analytics Roadmap 3 key strategic advantages, and a realistic roadmap for what you really need, and when 2012, Cognizant Topics to be discussed

More information

Data Integration Checklist

Data Integration Checklist The need for data integration tools exists in every company, small to large. Whether it is extracting data that exists in spreadsheets, packaged applications, databases, sensor networks or social media

More information

HP Vertica at MIT Sloan Sports Analytics Conference March 1, 2013 Will Cairns, Senior Data Scientist, HP Vertica

HP Vertica at MIT Sloan Sports Analytics Conference March 1, 2013 Will Cairns, Senior Data Scientist, HP Vertica HP Vertica at MIT Sloan Sports Analytics Conference March 1, 2013 Will Cairns, Senior Data Scientist, HP Vertica So What s the market s definition of Big Data? Datasets whose volume, velocity, variety

More information

Data Refinery with Big Data Aspects

Data Refinery with Big Data Aspects International Journal of Information and Computation Technology. ISSN 0974-2239 Volume 3, Number 7 (2013), pp. 655-662 International Research Publications House http://www. irphouse.com /ijict.htm Data

More information

What to Look for When Selecting a Master Data Management Solution

What to Look for When Selecting a Master Data Management Solution What to Look for When Selecting a Master Data Management Solution What to Look for When Selecting a Master Data Management Solution Table of Contents Business Drivers of MDM... 3 Next-Generation MDM...

More information

Independent process platform

Independent process platform Independent process platform Megatrend in infrastructure software Dr. Wolfram Jost CTO February 22, 2012 2 Agenda Positioning BPE Strategy Cloud Strategy Data Management Strategy ETS goes Mobile Each layer

More information

Washington State s Use of the IBM Data Governance Unified Process Best Practices

Washington State s Use of the IBM Data Governance Unified Process Best Practices STATS-DC 2012 Data Conference July 12, 2012 Washington State s Use of the IBM Data Governance Unified Process Best Practices Bill Huennekens Washington State Office of Superintendent of Public Instruction,

More information

Big Data and Apache Hadoop Adoption:

Big Data and Apache Hadoop Adoption: Expert Reference Series of White Papers Big Data and Apache Hadoop Adoption: Key Challenges and Rewards 1-800-COURSES www.globalknowledge.com Big Data and Apache Hadoop Adoption: Key Challenges and Rewards

More information

Hadoop Beyond Hype: Complex Adaptive Systems Conference Nov 16, 2012. Viswa Sharma Solutions Architect Tata Consultancy Services

Hadoop Beyond Hype: Complex Adaptive Systems Conference Nov 16, 2012. Viswa Sharma Solutions Architect Tata Consultancy Services Hadoop Beyond Hype: Complex Adaptive Systems Conference Nov 16, 2012 Viswa Sharma Solutions Architect Tata Consultancy Services 1 Agenda What is Hadoop Why Hadoop? The Net Generation is here Sizing the

More information

INTRODUCTION TO APACHE HADOOP MATTHIAS BRÄGER CERN GS-ASE

INTRODUCTION TO APACHE HADOOP MATTHIAS BRÄGER CERN GS-ASE INTRODUCTION TO APACHE HADOOP MATTHIAS BRÄGER CERN GS-ASE AGENDA Introduction to Big Data Introduction to Hadoop HDFS file system Map/Reduce framework Hadoop utilities Summary BIG DATA FACTS In what timeframe

More information

Comprehensive Analytics on the Hortonworks Data Platform

Comprehensive Analytics on the Hortonworks Data Platform Comprehensive Analytics on the Hortonworks Data Platform We do Hadoop. Page 1 Page 2 Back to 2005 Page 3 Vertical Scaling Page 4 Vertical Scaling Page 5 Vertical Scaling Page 6 Horizontal Scaling Page

More information

BIG DATA AND MICROSOFT. Susie Adams CTO Microsoft Federal

BIG DATA AND MICROSOFT. Susie Adams CTO Microsoft Federal BIG DATA AND MICROSOFT Susie Adams CTO Microsoft Federal THE WORLD OF DATA IS CHANGING Cloud What s making this possible? Electrical efficiency of computers doubles every year and ½. Laptops and mobile

More information

Buyer s Guide to Big Data Integration

Buyer s Guide to Big Data Integration SEPTEMBER 2013 Buyer s Guide to Big Data Integration Sponsored by Contents Introduction 1 Challenges of Big Data Integration: New and Old 1 What You Need for Big Data Integration 3 Preferred Technology

More information

BIG DATA-AS-A-SERVICE

BIG DATA-AS-A-SERVICE White Paper BIG DATA-AS-A-SERVICE What Big Data is about What service providers can do with Big Data What EMC can do to help EMC Solutions Group Abstract This white paper looks at what service providers

More information

#TalendSandbox for Big Data

#TalendSandbox for Big Data Evalua&on von Apache Hadoop mit der #TalendSandbox for Big Data Julien Clarysse @whatdoesdatado @talend 2015 Talend Inc. 1 Connecting the Data-Driven Enterprise 2 Talend Overview Founded in 2006 BRAND

More information

Transforming the Telecoms Business using Big Data and Analytics

Transforming the Telecoms Business using Big Data and Analytics Transforming the Telecoms Business using Big Data and Analytics Event: ICT Forum for HR Professionals Venue: Meikles Hotel, Harare, Zimbabwe Date: 19 th 21 st August 2015 AFRALTI 1 Objectives Describe

More information

Business Analytics In a Big Data World Ted Malone Solutions Architect Data Platform and Cloud Microsoft Federal

Business Analytics In a Big Data World Ted Malone Solutions Architect Data Platform and Cloud Microsoft Federal Business Analytics In a Big Data World Ted Malone Solutions Architect Data Platform and Cloud Microsoft Federal Information has gone from scarce to super-abundant. That brings huge new benefits. The Economist

More information

Big Data Analytics: Today's Gold Rush November 20, 2013

Big Data Analytics: Today's Gold Rush November 20, 2013 Copyright 2013 Vivit Worldwide Big Data Analytics: Today's Gold Rush November 20, 2013 Brought to you by Copyright 2013 Vivit Worldwide Hosted by Bernard Szymczak Vivit Leader Ohio Chapter TQA SIG Copyright

More information

White Paper: What You Need To Know About Hadoop

White Paper: What You Need To Know About Hadoop CTOlabs.com White Paper: What You Need To Know About Hadoop June 2011 A White Paper providing succinct information for the enterprise technologist. Inside: What is Hadoop, really? Issues the Hadoop stack

More information

Danny Wang, Ph.D. Vice President of Business Strategy and Risk Management Republic Bank

Danny Wang, Ph.D. Vice President of Business Strategy and Risk Management Republic Bank Danny Wang, Ph.D. Vice President of Business Strategy and Risk Management Republic Bank Agenda» Overview» What is Big Data?» Accelerates advances in computer & technologies» Revolutionizes data measurement»

More information

So What s the Big Deal?

So What s the Big Deal? So What s the Big Deal? Presentation Agenda Introduction What is Big Data? So What is the Big Deal? Big Data Technologies Identifying Big Data Opportunities Conducting a Big Data Proof of Concept Big Data

More information

Customized Report- Big Data

Customized Report- Big Data GINeVRA Digital Research Hub Customized Report- Big Data 1 2014. All Rights Reserved. Agenda Context Challenges and opportunities Solutions Market Case studies Recommendations 2 2014. All Rights Reserved.

More information

The 3 questions to ask yourself about BIG DATA

The 3 questions to ask yourself about BIG DATA The 3 questions to ask yourself about BIG DATA Do you have a big data problem? Companies looking to tackle big data problems are embarking on a journey that is full of hype, buzz, confusion, and misinformation.

More information

BIG DATA TRENDS AND TECHNOLOGIES

BIG DATA TRENDS AND TECHNOLOGIES BIG DATA TRENDS AND TECHNOLOGIES THE WORLD OF DATA IS CHANGING Cloud WHAT IS BIG DATA? Big data are datasets that grow so large that they become awkward to work with using onhand database management tools.

More information

We are Big Data A Sonian Whitepaper

We are Big Data A Sonian Whitepaper EXECUTIVE SUMMARY Big Data is not an uncommon term in the technology industry anymore. It s of big interest to many leading IT providers and archiving companies. But what is Big Data? While many have formed

More information

Big Data Buzzwords From A to Z. By Rick Whiting, CRN 4:00 PM ET Wed. Nov. 28, 2012

Big Data Buzzwords From A to Z. By Rick Whiting, CRN 4:00 PM ET Wed. Nov. 28, 2012 Big Data Buzzwords From A to Z By Rick Whiting, CRN 4:00 PM ET Wed. Nov. 28, 2012 Big Data Buzzwords Big data is one of the, well, biggest trends in IT today, and it has spawned a whole new generation

More information

Beyond the Single View with IBM InfoSphere

Beyond the Single View with IBM InfoSphere Ian Bowring MDM & Information Integration Sales Leader, NE Europe Beyond the Single View with IBM InfoSphere We are at a pivotal point with our information intensive projects 10-40% of each initiative

More information

DEVELOP INSIGHT DRIVEN CUSTOMER EXPERIENCES USING BIG DATA AND ADAVANCED ANALYTICS

DEVELOP INSIGHT DRIVEN CUSTOMER EXPERIENCES USING BIG DATA AND ADAVANCED ANALYTICS DEVELOP INSIGHT DRIVEN CUSTOMER EXPERIENCES USING BIG DATA AND ADAVANCED ANALYTICS by Dave Nash and Mazen Ghalayini; Contributions by Valentin Grasparil This whitepaper is the second in a 3-part series

More information

BIG DATA TECHNOLOGY. Hadoop Ecosystem

BIG DATA TECHNOLOGY. Hadoop Ecosystem BIG DATA TECHNOLOGY Hadoop Ecosystem Agenda Background What is Big Data Solution Objective Introduction to Hadoop Hadoop Ecosystem Hybrid EDW Model Predictive Analysis using Hadoop Conclusion What is Big

More information

Keywords Big Data; OODBMS; RDBMS; hadoop; EDM; learning analytics, data abundance.

Keywords Big Data; OODBMS; RDBMS; hadoop; EDM; learning analytics, data abundance. Volume 4, Issue 11, November 2014 ISSN: 2277 128X International Journal of Advanced Research in Computer Science and Software Engineering Research Paper Available online at: www.ijarcsse.com Analytics

More information

EXECUTIVE REPORT. Big Data and the 3 V s: Volume, Variety and Velocity

EXECUTIVE REPORT. Big Data and the 3 V s: Volume, Variety and Velocity EXECUTIVE REPORT Big Data and the 3 V s: Volume, Variety and Velocity The three V s are the defining properties of big data. It is critical to understand what these elements mean. The main point of the

More information

End to End Solution to Accelerate Data Warehouse Optimization. Franco Flore Alliance Sales Director - APJ

End to End Solution to Accelerate Data Warehouse Optimization. Franco Flore Alliance Sales Director - APJ End to End Solution to Accelerate Data Warehouse Optimization Franco Flore Alliance Sales Director - APJ Big Data Is Driving Key Business Initiatives Increase profitability, innovation, customer satisfaction,

More information

Apache Hadoop: The Big Data Refinery

Apache Hadoop: The Big Data Refinery Architecting the Future of Big Data Whitepaper Apache Hadoop: The Big Data Refinery Introduction Big data has become an extremely popular term, due to the well-documented explosion in the amount of data

More information

Informatica and the Vibe Virtual Data Machine

Informatica and the Vibe Virtual Data Machine White Paper Informatica and the Vibe Virtual Data Machine Preparing for the Integrated Information Age This document contains Confidential, Proprietary and Trade Secret Information ( Confidential Information

More information

W H I T E P A P E R. Deriving Intelligence from Large Data Using Hadoop and Applying Analytics. Abstract

W H I T E P A P E R. Deriving Intelligence from Large Data Using Hadoop and Applying Analytics. Abstract W H I T E P A P E R Deriving Intelligence from Large Data Using Hadoop and Applying Analytics Abstract This white paper is focused on discussing the challenges facing large scale data processing and the

More information

BIG DATA ANALYTICS REFERENCE ARCHITECTURES AND CASE STUDIES

BIG DATA ANALYTICS REFERENCE ARCHITECTURES AND CASE STUDIES BIG DATA ANALYTICS REFERENCE ARCHITECTURES AND CASE STUDIES Relational vs. Non-Relational Architecture Relational Non-Relational Rational Predictable Traditional Agile Flexible Modern 2 Agenda Big Data

More information

International Journal of Advancements in Research & Technology, Volume 3, Issue 5, May-2014 18 ISSN 2278-7763. BIG DATA: A New Technology

International Journal of Advancements in Research & Technology, Volume 3, Issue 5, May-2014 18 ISSN 2278-7763. BIG DATA: A New Technology International Journal of Advancements in Research & Technology, Volume 3, Issue 5, May-2014 18 BIG DATA: A New Technology Farah DeebaHasan Student, M.Tech.(IT) Anshul Kumar Sharma Student, M.Tech.(IT)

More information

Capital Market Day 2015

Capital Market Day 2015 Capital Market Day 2015 Digital Business Platform & Product Roadmap Dr. Wolfram Jost Chief Technology Officer February 4, 2015 1 For Internal use only. Market Application infrastructure and middleware

More information

A financial software company

A financial software company A financial software company Projecting USD10 million revenue lift with the IBM Netezza data warehouse appliance Overview The need A financial software company sought to analyze customer engagements to

More information

GigaSpaces Real-Time Analytics for Big Data

GigaSpaces Real-Time Analytics for Big Data GigaSpaces Real-Time Analytics for Big Data GigaSpaces makes it easy to build and deploy large-scale real-time analytics systems Rapidly increasing use of large-scale and location-aware social media and

More information

Roadmap Talend : découvrez les futures fonctionnalités de Talend

Roadmap Talend : découvrez les futures fonctionnalités de Talend Roadmap Talend : découvrez les futures fonctionnalités de Talend Cédric Carbone Talend Connect 9 octobre 2014 Talend 2014 1 Connecting the Data-Driven Enterprise Talend 2014 2 Agenda Agenda Why a Unified

More information

Cloudera Enterprise Data Hub in Telecom:

Cloudera Enterprise Data Hub in Telecom: Cloudera Enterprise Data Hub in Telecom: Three Customer Case Studies Version: 103 Table of Contents Introduction 3 Cloudera Enterprise Data Hub for Telcos 4 Cloudera Enterprise Data Hub in Telecom: Customer

More information

Big Data: What You Should Know. Mark Child Research Manager - Software IDC CEMA

Big Data: What You Should Know. Mark Child Research Manager - Software IDC CEMA Big Data: What You Should Know Mark Child Research Manager - Software IDC CEMA Agenda Market Dynamics Defining Big Data Technology Trends Information and Intelligence Market Realities Future Applications

More information

Alexander Nikov. 5. Database Systems and Managing Data Resources. Learning Objectives. RR Donnelley Tries to Master Its Data

Alexander Nikov. 5. Database Systems and Managing Data Resources. Learning Objectives. RR Donnelley Tries to Master Its Data INFO 1500 Introduction to IT Fundamentals 5. Database Systems and Managing Data Resources Learning Objectives 1. Describe how the problems of managing data resources in a traditional file environment are

More information

Wikibon Big Data Analytics Adoption Survey, 2014-2015 Frequency Analysis

Wikibon Big Data Analytics Adoption Survey, 2014-2015 Frequency Analysis Wikibon.com - http://wikibon.com Wikibon Big Data Analytics Adoption Survey, 2014-2015 Frequency Analysis by Jeff Kelly - 1 July 2014 http://wikibon.com/wikibon-big-data-analytics-adoption-survey-2014-2015-frequency-analysis/

More information

Tap into Hadoop and Other No SQL Sources

Tap into Hadoop and Other No SQL Sources Tap into Hadoop and Other No SQL Sources Presented by: Trishla Maru What is Big Data really? The Three Vs of Big Data According to Gartner Volume Volume Orders of magnitude bigger than conventional data

More information

Apache Hadoop Patterns of Use

Apache Hadoop Patterns of Use Community Driven Apache Hadoop Apache Hadoop Patterns of Use April 2013 2013 Hortonworks Inc. http://www.hortonworks.com Big Data: Apache Hadoop Use Distilled There certainly is no shortage of hype when

More information

Architecting an Industrial Sensor Data Platform for Big Data Analytics: Continued

Architecting an Industrial Sensor Data Platform for Big Data Analytics: Continued Architecting an Industrial Sensor Data Platform for Big Data Analytics: Continued 2 8 10 Issue 1 Welcome From the Gartner Files: Blueprint for Architecting Sensor Data for Big Data Analytics About OSIsoft,

More information

Forecast of Big Data Trends. Assoc. Prof. Dr. Thanachart Numnonda Executive Director IMC Institute 3 September 2014

Forecast of Big Data Trends. Assoc. Prof. Dr. Thanachart Numnonda Executive Director IMC Institute 3 September 2014 Forecast of Big Data Trends Assoc. Prof. Dr. Thanachart Numnonda Executive Director IMC Institute 3 September 2014 Big Data transforms Business 2 Data created every minute Source http://mashable.com/2012/06/22/data-created-every-minute/

More information

Understanding traffic flow

Understanding traffic flow White Paper A Real-time Data Hub For Smarter City Applications Intelligent Transportation Innovation for Real-time Traffic Flow Analytics with Dynamic Congestion Management 2 Understanding traffic flow

More information

Understanding Your Customer Journey by Extending Adobe Analytics with Big Data

Understanding Your Customer Journey by Extending Adobe Analytics with Big Data SOLUTION BRIEF Understanding Your Customer Journey by Extending Adobe Analytics with Big Data Business Challenge Today s digital marketing teams are overwhelmed by the volume and variety of customer interaction

More information

Data processing goes big

Data processing goes big Test report: Integration Big Data Edition Data processing goes big Dr. Götz Güttich Integration is a powerful set of tools to access, transform, move and synchronize data. With more than 450 connectors,

More information

The Rise of Industrial Big Data

The Rise of Industrial Big Data GE Intelligent Platforms The Rise of Industrial Big Data Leveraging large time-series data sets to drive innovation, competitiveness and growth capitalizing on the big data opportunity The Rise of Industrial

More information

Big Data for Banking. Kaleem Chaudhry Senior Director, Sales Consulting, ASEAN. Copyright 2013, Oracle and/or its affiliates. All rights reserved.

Big Data for Banking. Kaleem Chaudhry Senior Director, Sales Consulting, ASEAN. Copyright 2013, Oracle and/or its affiliates. All rights reserved. Big Data for Banking Kaleem Chaudhry Senior Director, Sales Consulting, ASEAN Big Data in Financial Services Key Business Goals: Looking beyond the credit bureau report to assess consumer credit worthiness

More information

BIRT in the World of Big Data

BIRT in the World of Big Data BIRT in the World of Big Data David Rosenbacher VP Sales Engineering Actuate Corporation 2013 Actuate Customer Days Today s Agenda and Goals Introduction to Big Data Compare with Regular Data Common Approaches

More information

Hadoop Big Data for Processing Data and Performing Workload

Hadoop Big Data for Processing Data and Performing Workload Hadoop Big Data for Processing Data and Performing Workload Girish T B 1, Shadik Mohammed Ghouse 2, Dr. B. R. Prasad Babu 3 1 M Tech Student, 2 Assosiate professor, 3 Professor & Head (PG), of Computer

More information

Generating the Business Value of Big Data:

Generating the Business Value of Big Data: Leveraging People, Processes, and Technology Generating the Business Value of Big Data: Analyzing Data to Make Better Decisions Authors: Rajesh Ramasubramanian, MBA, PMP, Program Manager, Catapult Technology

More information

Smarter Analytics. Barbara Cain. Driving Value from Big Data

Smarter Analytics. Barbara Cain. Driving Value from Big Data Smarter Analytics Driving Value from Big Data Barbara Cain Vice President Product Management - Business Intelligence and Advanced Analytics Business Analytics IBM Software Group 1 Agenda for today 1 Big

More information

Big Data Zurich, November 23. September 2011

Big Data Zurich, November 23. September 2011 Institute of Technology Management Big Data Projektskizze «Competence Center Automotive Intelligence» Zurich, November 11th 23. September 2011 Felix Wortmann Assistant Professor Technology Management,

More information

Deploying Big Data to the Cloud: Roadmap for Success

Deploying Big Data to the Cloud: Roadmap for Success Deploying Big Data to the Cloud: Roadmap for Success James Kobielus Chair, CSCC Big Data in the Cloud Working Group IBM Big Data Evangelist. IBM Data Magazine, Editor-in- Chief. IBM Senior Program Director,

More information

Protecting Big Data Data Protection Solutions for the Business Data Lake

Protecting Big Data Data Protection Solutions for the Business Data Lake White Paper Protecting Big Data Data Protection Solutions for the Business Data Lake Abstract Big Data use cases are maturing and customers are using Big Data to improve top and bottom line revenues. With

More information

Evolution from Big Data to Smart Data

Evolution from Big Data to Smart Data Evolution from Big Data to Smart Data Information is Exploding 120 HOURS VIDEO UPLOADED TO YOUTUBE 50,000 APPS DOWNLOADED 204 MILLION E-MAILS EVERY MINUTE EVERY DAY Intel Corporation 2015 The Data is Changing

More information

www.pwc.com/oracle Next presentation starting soon Business Analytics using Big Data to gain competitive advantage

www.pwc.com/oracle Next presentation starting soon Business Analytics using Big Data to gain competitive advantage www.pwc.com/oracle Next presentation starting soon Business Analytics using Big Data to gain competitive advantage If every image made and every word written from the earliest stirring of civilization

More information

QLIKVIEW DEPLOYMENT FOR BIG DATA ANALYTICS AT KING.COM

QLIKVIEW DEPLOYMENT FOR BIG DATA ANALYTICS AT KING.COM QLIKVIEW DEPLOYMENT FOR BIG DATA ANALYTICS AT KING.COM QlikView Technical Case Study Series Big Data June 2012 qlikview.com Introduction This QlikView technical case study focuses on the QlikView deployment

More information

Industry Impact of Big Data in the Cloud: An IBM Perspective

Industry Impact of Big Data in the Cloud: An IBM Perspective Industry Impact of Big Data in the Cloud: An IBM Perspective Inhi Cho Suh IBM Software Group, Information Management Vice President, Product Management and Strategy email: inhicho@us.ibm.com twitter: @inhicho

More information

Informatica Data Quality Product Family

Informatica Data Quality Product Family Brochure Informatica Product Family Deliver the Right Capabilities at the Right Time to the Right Users Benefits Reduce risks by identifying, resolving, and preventing costly data problems Enhance IT productivity

More information

Microsoft SQL Server 2012 with Hadoop

Microsoft SQL Server 2012 with Hadoop Microsoft SQL Server 2012 with Hadoop Debarchan Sarkar Chapter No. 1 "Introduction to Big Data and Hadoop" In this package, you will find: A Biography of the author of the book A preview chapter from the

More information

How to Leverage Big Data in the Cloud to Gain Competitive Advantage

How to Leverage Big Data in the Cloud to Gain Competitive Advantage How to Leverage Big Data in the Cloud to Gain Competitive Advantage James Kobielus, IBM Big Data Evangelist Editor-in-Chief, IBM Data Magazine Senior Program Director, Product Marketing, Big Data Analytics

More information

Addressing Open Source Big Data, Hadoop, and MapReduce limitations

Addressing Open Source Big Data, Hadoop, and MapReduce limitations Addressing Open Source Big Data, Hadoop, and MapReduce limitations 1 Agenda What is Big Data / Hadoop? Limitations of the existing hadoop distributions Going enterprise with Hadoop 2 How Big are Data?

More information

Managing Cloud Server with Big Data for Small, Medium Enterprises: Issues and Challenges

Managing Cloud Server with Big Data for Small, Medium Enterprises: Issues and Challenges Managing Cloud Server with Big Data for Small, Medium Enterprises: Issues and Challenges Prerita Gupta Research Scholar, DAV College, Chandigarh Dr. Harmunish Taneja Department of Computer Science and

More information

Using Tableau Software with Hortonworks Data Platform

Using Tableau Software with Hortonworks Data Platform Using Tableau Software with Hortonworks Data Platform September 2013 2013 Hortonworks Inc. http:// Modern businesses need to manage vast amounts of data, and in many cases they have accumulated this data

More information

Big Data & Analytics: Your concise guide (note the irony) Wednesday 27th November 2013

Big Data & Analytics: Your concise guide (note the irony) Wednesday 27th November 2013 Big Data & Analytics: Your concise guide (note the irony) Wednesday 27th November 2013 Housekeeping 1. Any questions coming out of today s presentation can be discussed in the bar this evening 2. OCF is

More information

Microsoft Big Data. Solution Brief

Microsoft Big Data. Solution Brief Microsoft Big Data Solution Brief Contents Introduction... 2 The Microsoft Big Data Solution... 3 Key Benefits... 3 Immersive Insight, Wherever You Are... 3 Connecting with the World s Data... 3 Any Data,

More information

Foundations of Business Intelligence: Databases and Information Management

Foundations of Business Intelligence: Databases and Information Management Foundations of Business Intelligence: Databases and Information Management Wienand Omta Fabiano Dalpiaz 1 drs. ing. Wienand Omta Learning Objectives Describe how the problems of managing data resources

More information

Architecting for Big Data Analytics and Beyond: A New Framework for Business Intelligence and Data Warehousing

Architecting for Big Data Analytics and Beyond: A New Framework for Business Intelligence and Data Warehousing Architecting for Big Data Analytics and Beyond: A New Framework for Business Intelligence and Data Warehousing Wayne W. Eckerson Director of Research, TechTarget Founder, BI Leadership Forum Business Analytics

More information

Big Data: Are You Ready? Kevin Lancaster

Big Data: Are You Ready? Kevin Lancaster Big Data: Are You Ready? Kevin Lancaster Director, Engineered Systems Oracle Europe, Middle East & Africa 1 A Data Explosion... Traditional Data Sources Billing engines Custom developed New, Non-Traditional

More information

Sources: Summary Data is exploding in volume, variety and velocity timely

Sources: Summary Data is exploding in volume, variety and velocity timely 1 Sources: The Guardian, May 2010 IDC Digital Universe, 2010 IBM Institute for Business Value, 2009 IBM CIO Study 2010 TDWI: Next Generation Data Warehouse Platforms Q4 2009 Summary Data is exploding

More information

Big Data and Your Data Warehouse Philip Russom

Big Data and Your Data Warehouse Philip Russom Big Data and Your Data Warehouse Philip Russom TDWI Research Director for Data Management April 5, 2012 Sponsor Speakers Philip Russom Research Director, Data Management, TDWI Peter Jeffcock Director,

More information

22 SMARTENTERPRISEMAG.COM

22 SMARTENTERPRISEMAG.COM 22 SMARTENTERPRISEMAG.COM Smart Strategies BIG DATA, Big Innovation Smart CIOs are mining their organizations huge data stores for insights that lead to business innovation. By Tom Farre ILLUSTRATION:

More information

Secure Cloud Computing Concepts Supporting Big Data in Healthcare. Ryan D. Pehrson Director, Solutions & Architecture Integrated Data Storage, LLC

Secure Cloud Computing Concepts Supporting Big Data in Healthcare. Ryan D. Pehrson Director, Solutions & Architecture Integrated Data Storage, LLC Secure Cloud Computing Concepts Supporting Big Data in Healthcare Ryan D. Pehrson Director, Solutions & Architecture Integrated Data Storage, LLC Learning Objectives After this session, the learner should

More information