How To Manage Big Data
|
|
- Alvin Poole
- 3 years ago
- Views:
Transcription
1 The Data Lake: Taking Big Data Beyond the Cloud by Mark Herman Executive Vice President Booz Allen Hamilton Michael Delurey Principal Booz Allen Hamilton The bigger that big data gets, the more it seems to elude our grasp. While it holds great potential for creating new opportunities in every field, big data is growing so fast that it is now outpacing the ability of our current tools to take full advantage of it. Much of the problem lies in the need to extensively prepare the data before it can be analyzed. Data must be converted into recognizable formats a laborious, time-consuming process that becomes increasingly impractical as data collections grow larger. Although organizations are amassing impressive amounts of data, they simply do not have the time or resources to prepare it all in the traditional manner. This is particularly an issue with unstructured data that does not easily lend itself to formatting, such as photographs, doctors examination notes, police accident reports, and posts on social media sites. Unstructured data accounts for much of the explosion in big data today, and is widely seen as holding the most promise for creating new areas of business growth and government efficiency. But because unstructured data is so difficult to prepare, its enormous value remains largely untapped. With such constraints, organizations are now reaching the limits of what they can do with big data. They are going as far as the current tools will take them, but no further. And as big data grows larger, organizations will only be increasingly inundated with information that they have only a narrow ability to use. It is like the line, Water, water, everywhere What is needed is an entirely new approach to this overwhelming flood of data, one that can manage it and make it useful, no matter how big it grows. That is the concept behind Booz Allen Hamilton s data lake, a groundbreaking invention that scales to an organization s growing data, and makes it easily accessible. With the data lake, an organization s repository of information including structured and unstructured data is consolidated in a single, large table. Every inquiry can make use of the entire body of information stored in the data lake and it is all available at once. The data lake completely eliminates the current cumbersome data-preparation process. All types of data, including unstructured data, are smoothly and rapidly ingested into the data lake. There is no longer any need for the rigid, regimented data structures essentially data silos that currently house most data. Such silos are difficult to connect, which has long hampered the ability of organizations to integrate and analyze their data. The data lake solves this problem by eliminating the silos altogether. With the data lake, it now becomes practical in terms of time, cost, and analytic ability to turn big data into 2013 Booz Allen Hamilton Inc. All rights reserved. No part of this document may be reproduced without prior written permission of Booz Allen Hamilton. 1
2 opportunity. We can now ask more far-reaching and complex questions, and find the often-hidden patterns and relationships that can lead to game-changing knowledge and insight. More than the Cloud With the advent of cloud computing, business and government organizations are now storing and analyzing far larger amounts of data than ever before. But simply bringing a great deal of data together in the cloud is not the same as creating a data lake. Organizations may have embraced the cloud, but if they continue to use conventional tools, they still must laboriously prepare the data and place it in its designated location (i.e., the silo). Despite its promise to revolutionize data analysis, the cloud does not truly integrate data it simply makes the data silos taller and fatter. While the data lake relies on cloud computing, it represents a new and different mindset. Big data requires organizations to stop thinking in terms of data mining and data warehouses the equivalent of industrial-era processes and to begin considering how data can be more fluid and expansive, like in a data lake. Since with the conventional approach it is difficult to integrate data even in the cloud we tend to use the cloud mostly for storage, and remove portions of it for analysis. But no matter how powerful our analytics are, because we are applying them only to discrete datasets at any time, we never see the full picture. With the data lake, however, all of our data remains in the cloud, consolidated and connected. We can now apply our analytics to the whole of the data, and get far deeper insights. Organizations may be concerned that by consolidating their data, they might be making it more vulnerable. Just the opposite is true. The data lake incorporates a granular level of data security and privacy not available in conventional cloud computing. 1 The data lake was initially created to achieve a high-stakes goal. The US government needed a way to integrate many sources and types of intelligence data, in a secure manner, to search for terrorists and other threats. Booz Allen assisted the government in developing the data lake to achieve that goal, as part of a larger computing framework known as the Cloud Analytics Reference Architecture. The data lake and Cloud Analytics Reference Architecture are now being adapted to the larger business 1 See Booz Allen Viewpoint Enabling Cloud Analytics with Data-Level Security: Tapping the Full Value of Big Data and the Cloud, and government communities, bringing with them a range of features that have been successfully tested in the most demanding situations. Building the Data Lake One of the biggest limitations of the conventional approach to data analysis is that analysts often need to spend the bulk of their time just readying the data for use. With each new line of inquiry, a specific data structure and analytic is custom-built. All information entered into the data structure must first be converted into a recognizable format, often a slow, painstaking task. For example, an analyst might be faced with merging several different data sources that each use different fields. The analyst must decide which fields to use and whether entirely new ones need to be created. The more complex the query, the more data sources that typically must be homogenized. At some organizations, analysts may spend as much as 80 percent of their time preparing the data, leaving just 20 percent for conducting actual analysis. Formatting also carries the risk of dataentry errors. With the data lake, there are no individual data structures and so there is no need for formal data formatting. Data from a wide range of sources is smoothly and easily ingested into the data lake. One metaphor for the data lake might be a giant collection grid, like a spreadsheet one with billions of rows and billions of columns available to hold data. Each cell of the grid contains a piece of data a document, perhaps, or maybe a paragraph or even a single word from the document. Cells might contain names, photographs, incident reports, or Twitter feeds anything and everything. It does not matter where in the grid each bit of information is located. It also makes no difference where the data comes from, whether it is formatted, or how it might relate to any other piece of information in the data lake. The data simply takes its place in the cell, and after minimal preparation is ready for use. The image of the grid helps describe the difference between data mining and the data lake. If we want to mine precious metals, we have to find where they are, then dig deep to retrieve them. But imagine if, when the Earth was formed, nuggets of precious metals were laid out in a big grid on top of the ground. We could just walk along, picking up what we wanted. The data lake makes information just as readily available. The process of placing the data in open cells as it comes in gives the ingest process remarkable speed. Large amounts of data that might take 3 weeks to prepare using conventional cloud computing can be placed into the data lake in as little as 3 hours. This 2 MARCH 2013
3 enables organizations to achieve substantial savings in IT resources and manpower. Just as important, it frees analysts for the more important task of finding connections and value in the data. Many organizations today are trying to do more with less. That is difficult with the conventional approach, but becomes possible, for the first time, with the data lake. Opening Up the Data The ingest process of the data lake also removes another disadvantage of the conventional approach the need to pre-define our questions. With conventional computing techniques, we have to know in advance what kinds of answers we are looking for and where in the existing data the computer needs to look to answer the inquiry. Analysts do not really ask questions of the data they form hypotheses well in advance of the actual analysis, and then create data structures and analytics that will enable them to test those hypotheses. The only results that come back are the ones that the custom-made databases and analytics happen to provide. What makes this exercise even more constraining is that the data supporting an analysis typically contains only a portion of the potentially available information. Because the process of formatting and structuring the data is so time-intensive, analysts have no choice but to cull the data by some method. One of the most prevalent techniques is to discount (and even ignore) unstructured data. This simplifies the data ingest, but it severely reduces the value of the data for analysis. Hampered by these severe limitations, analysts can pose only narrow questions of the data. And there is a risk that the data structures will become closedloop systems echo chambers that merely validate the original hypotheses. When we ask the system what is important, it points to the data that we happened to put in. The fact that a particular piece of data is included in a database tends to make it de facto significant it is important only because the hypothesis sees it that way. With the data lake, data is ingested with a wide-open view as to the queries that may come later. Because there are no structures, we can get all of the data in all 100 variables, or 500, or any other number, so that the data in its totality becomes available. Organizations may have a great deal of data stored in the cloud, but without the data lake they cannot easily connect it all, and discover the often-hidden relationships in the world around us. It is in those relationships that knowledge and insight and opportunity reside. Tagging the Data The data lake also radically differs from conventional cloud computing in the way the data itself is managed. When a piece of data is ingested, certain details, called metadata (or data about the data ), are added so that the basic information can be quickly located and identified. For example, an investor s portfolio balance (the data) might be stored with the name of the investor, the account number, the location of the account, the types of investments, the country the investor lives in, and so on. These metadata tags serve the same purpose as old-style card catalogues, which allow readers to find a book by searching the author, title, or subject. As with the card catalogues, tags enable us to find particular information from a number of different starting points but with today s tagging abilities, we can characterize data in nearly limitless ways. The more tags, the more complex and rich the analytics can become. With the tags, we can look not only for connections and patterns in the data, but in the tags as well. To consider how this technology might be applied, imagine if a pharmaceutical company were able to fully integrate a wide range of public data to identify drug compounds with few adverse reactions, and a high likelihood of clinical and commercial success. Those sources might include social media and market data to help determine the need and clinical test data, chemical structure, disease analysis, even information about patents to find where gaps might exist. In a sense, the pharmaceutical company is looking for a needle in a haystack, a prohibitively expensive and timeconsuming task with conventional cloud computing. However, if the structured and unstructured data is appropriately tagged and placed in the data lake, it becomes cost-effective to find the essential connections in all that data, and make the needle stand out brightly. The data lake allows us to ask questions and search for patterns using either the data itself, the tags themselves, or a combination of both. We can begin our search with any piece of data or tag for example, a market analysis or the existing patents on a type of drug and pivot off of it in any direction to look for connections. While the process of tagging information is not new, the data lake uses it in a unique way as the primary method of locating and managing the data. With the tags, the rigid data structures that so limit the conventional approach are no longer needed. MARCH
4 Along with the streamlined ingest process, tags help give the data lake its speed. When organizations need to update or search the data in new ways, they do not have to tear down and rebuild data structures, as in the conventional method. They can simply update the tags already in place. Tagging all of the data, and at a much more granular level than is possible in the conventional cloud approach, greatly expands the value that big data can provide. Information in the data lake is not random and chaotic, but rather is purposeful. The tags help make the data lake like a viscous medium that holds the data in place, and at the same time fosters connections. The tags also provide a strong new layer of security. We can tag each piece of data, down to the image or paragraph in a document, with the relevant restrictions, authorities, and security and privacy levels. Organizations can establish rules regarding which information can be shared, with whom, and under what circumstances. A New Way of Storing Data With the conventional approach, data storage is expensive even in the cloud. The reason is that so much space is wasted. Imagine a spreadsheet combining two data sources, an original one with 100 fields and the other with 50. The process of combining means that we will be adding 50 new columns into the original spreadsheet. Rows from the original will hold no data for the new columns, and rows from the new source will hold no data from the original. The result will be a great deal of empty cells. This is wasted storage space, and creates the opportunity for a great many errors. In the data lake, however, every cell is filled no space is wasted. This makes it possible to store vast amounts of data in far less space than would be required for even relatively small conventional cloud databases. As a result, the data lake can cost-effectively scale to an organization s growing data, including multiple outside sources. The data lake s almost limitless capacity enables organizations to store data in a variety of different forms, to aid in later analysis. A financial institution, for example, could store records of certain transactions converted into all of the world s major currencies. Or, a company could translate every document on a particular subject into Chinese, and store it until it might be needed. One of the more transformative aspects of the data lake is that it stores every type of data equally not just structured and unstructured, but also batch and streaming. Batch data is typically collected on an automated basis and then delivered for analysis en masse for example, the utility meter readings from homes. Streaming data is information from a continuous feed, such as video surveillance. Formatting unstructured, batch, and streaming data inevitably strips it of much of its richness. And even if a portion of the information can be put into a conventional cloud database, we are still constrained by limited, pre-defined questions. The data lake holds no such constraints. When unstructured, batch, and streaming data are ingested, analytics can take advantage of the tagging approach to begin to look for patterns that naturally emerge. All types of data, and the value they hold, now become fully accessible. The US military is taking advantage of this capability to help track insurgents and others who are planting improvised explosive devices (IEDs) and other bombs. Many of the military s data sources include unstructured data, and using the conventional approach with its extensive preparation had proved unwieldy and time-consuming. With the data lake, the military is now able to quickly integrate and analyze its vast array of disparate data sources including its unstructured data giving military commanders unprecedented situational awareness. This is another example of why simply amassing large amounts of data does not create a data lake. The military was collecting an enormous quantity of data, but without the data lake could not make full use of it to try to stop IEDs. Commanders have reported that the current approach which has the data lake as its centerpiece is saving more lives, and at a lower operating cost than the traditional methods. Accessing the Data for Analytics One of the chief drawbacks of the conventional approach, which the cloud does not ameliorate, is that it essentially samples the data. When we have questions (or want to test hypotheses), we select a sample of the available data and apply analytics to it. The problem is that we are never quite sure we are pulling the right sample that is, whether it is really representative of the whole. The data lake eliminates sampling. We no longer have to guess about which data to use, because we are using it all. With the data lake, our information is available for analysis on-demand, when the need arises. The conventional approach not only requires extensive data preparation, but it is difficult to change databases as queries change. Say the pharmaceutical company wants to add new data sources to identify promising 4 MARCH 2013
5 drug compounds, or perhaps wants to change the type of financial analyses it uses. With the conventional approach, analysts would have to tear down the initial data and analytics structures, and re-engineer new ones. With the data lake, analysts would simply add the new data, and ask the new questions. Because it is not easy to change conventional data structures, the information they contain can become outdated and even obsolete fairly quickly. By contrast, we are able to add new information to the data lake the moment we need it. This ease in accessibility sets the stage for the advanced, high-powered analytics that can point the way to top-line business growth, and help government achieve its goals in innovative ways. Analytics that search for connections and look for patterns have long been hamstrung by being confined to limited, rigid datasets and databases. The data lake frees them to search for knowledge and insight across all of the data. In essence, it allows the analytics, for the first time, to reach their true potential. Because there is no need to continually engineer and re-engineer data structures, the data lake also becomes accessible to non-technical subject matter experts. They no longer need to rely on computer scientists and others to explore the data they can ask the questions themselves. Subject matter experts best understand how big data can provide value to their businesses and agencies. The data lake helps put the answers directly in their hands. amounts of data no matter how large will not necessarily yield more knowledge and insight. The trick is to connect the data and make it useful essentially, to create the kinds of conditions that can turn big data into opportunity. The data lake and the larger Cloud Analytics Reference Architecture represent a revolutionary approach and a new mindset that make those conditions possible. Opportunity is out there, if we have the tools to look for it. FOR MORE INFORMATION Mark Herman herman_mark@bah.com Michael Delurey delurey_mike@bah.com A New Mindset Virtually every aspect of the data lake creates cost savings and efficiencies, from freeing up analysts to its ability to easily and inexpensively scale to an organization s growing data. Because the data lake enables organizations to gather and analyze ever-greater amounts of data, it also gives them new opportunities for top-line revenue growth. The data lake enables both business and government to reach that tipping point at which data helps us to do things not just cheaper and better, but in ways we have not yet imagined. Organizations may believe that because they are now in the cloud and can put all their data in one place, they already have a version of the data lake. But greater This document is part of a collection of papers developed by Booz Allen Hamilton to introduce new concepts and ideas spanning cloud solutions, challenges, and opportunities across government and business. For media inquiries or more information on reproducing this document, please contact: James Fisher Senior Manager, Media Relations, , fisher_james_w@bah.com Carrie Lake Manager, Media Relations, , lake_carrie@bah.com MARCH P
Turning Big Data into Opportunity
Turning Big Data into Opportunity The Data Lake by Mark Herman herman_mark@bah.com Michael Delurey delurey_mike@bah.com Table of Contents Introduction... 1 A New Mindset... 1 Ingesting Data into the Data
More informationHOW THE DATA LAKE WORKS
HOW THE DATA LAKE WORKS by Mark Jacobsohn Senior Vice President Booz Allen Hamilton Michael Delurey, EngD Principal Booz Allen Hamilton As organizations rush to take advantage of large and diverse data
More informationDELIVERING ON THE PROMISE OF BIG DATA AND THE CLOUD
DELIVERING ON THE PROMISE OF BIG DATA AND THE CLOUD by Mark Jacobsohn Senior Vice President Booz Allen Hamilton Joshua Sullivan, PhD Vice President Booz Allen Hamilton WHY CAN T WE SEEM TO DO MORE WITH
More informationData Lake-based Approaches to Regulatory- Driven Technology Challenges
Data Lake-based Approaches to Regulatory- Driven Technology Challenges How a Data Lake Approach Improves Accuracy and Cost Effectiveness in the Extract, Transform, and Load Process for Business and Regulatory
More informationHarnessing Big Data to Solve Complex Problems: The Cloud Analytics Reference Architecture
Harnessing Big Data to Solve Complex Problems: The Cloud Analytics Reference Architecture Table of Contents Introduction... 1 Cloud Analytics Reference Architecture... 1 Using All the Data... 3 Better
More informationDetecting Anomalous Behavior with the Business Data Lake. Reference Architecture and Enterprise Approaches.
Detecting Anomalous Behavior with the Business Data Lake Reference Architecture and Enterprise Approaches. 2 Detecting Anomalous Behavior with the Business Data Lake Pivotal the way we see it Reference
More informationUnlocking The Value of the Deep Web. Harvesting Big Data that Google Doesn t Reach
Unlocking The Value of the Deep Web Harvesting Big Data that Google Doesn t Reach Introduction Every day, untold millions search the web with Google, Bing and other search engines. The volumes truly are
More informationWe are Big Data A Sonian Whitepaper
EXECUTIVE SUMMARY Big Data is not an uncommon term in the technology industry anymore. It s of big interest to many leading IT providers and archiving companies. But what is Big Data? While many have formed
More informationBIG Data Analytics Move to Competitive Advantage
BIG Data Analytics Move to Competitive Advantage where is technology heading today Standardization Open Source Automation Scalability Cloud Computing Mobility Smartphones/ tablets Internet of Things Wireless
More informationDanny Wang, Ph.D. Vice President of Business Strategy and Risk Management Republic Bank
Danny Wang, Ph.D. Vice President of Business Strategy and Risk Management Republic Bank Agenda» Overview» What is Big Data?» Accelerates advances in computer & technologies» Revolutionizes data measurement»
More informationIgnite Your Creative Ideas with Fast and Engaging Data Discovery
SAP Brief SAP BusinessObjects BI s SAP Crystal s SAP Lumira Objectives Ignite Your Creative Ideas with Fast and Engaging Data Discovery Tap into your data big and small Tap into your data big and small
More informationBig Data Trends A Basis for Personalized Medicine
Big Data Trends A Basis for Personalized Medicine Dr. Hellmuth Broda, Principal Technology Architect emedikation: Verordnung, Support Prozesse & Logistik 5. Juni, 2013, Inselspital Bern Over 150,000 Employees
More informationEnabling Cloud Analytics with Data-Level Security
Enabling Cloud Analytics with Data-Level Security Tapping the Full Value of Big Data and the Cloud by Jason Escaravage escaravage_jason@bah.com Peter Guerra guerra_peter@bah.com Table of Contents Introduction...
More informationVIEWPOINT. High Performance Analytics. Industry Context and Trends
VIEWPOINT High Performance Analytics Industry Context and Trends In the digital age of social media and connected devices, enterprises have a plethora of data that they can mine, to discover hidden correlations
More informationDATA MANAGEMENT FOR THE INTERNET OF THINGS
DATA MANAGEMENT FOR THE INTERNET OF THINGS February, 2015 Peter Krensky, Research Analyst, Analytics & Business Intelligence Report Highlights p2 p4 p6 p7 Data challenges Managing data at the edge Time
More informationIntelligent Systems: Unlocking hidden business value with data. 2011 Microsoft Corporation. All Right Reserved
Intelligent Systems: Unlocking hidden business value with data Intelligent Systems 2 Microsoft Corporation September 2011 Applies to: Windows Embedded Summary: An intelligent system enables data to flow
More informationReaping the Rewards of Big Data
Reaping the Rewards of Big Data TABLE OF CONTENTS INTRODUCTION: 2 TABLE OF CONTENTS FINDING #1: BIG DATA PLATFORMS ARE ESSENTIAL FOR A MAJORITY OF ORGANIZATIONS TO MANAGE FUTURE BIG DATA CHALLENGES. 4
More informationIT Workload Automation: Control Big Data Management Costs with Cisco Tidal Enterprise Scheduler
White Paper IT Workload Automation: Control Big Data Management Costs with Cisco Tidal Enterprise Scheduler What You Will Learn Big data environments are pushing the performance limits of business processing
More informationScalable Enterprise Data Integration Your business agility depends on how fast you can access your complex data
Transforming Data into Intelligence Scalable Enterprise Data Integration Your business agility depends on how fast you can access your complex data Big Data Data Warehousing Data Governance and Quality
More informationThe New Normal: Get Ready for the Era of Extreme Information Management. John Mancini President, AIIM @jmancini77 DigitalLandfill.
The New Normal: Get Ready for the Era of Extreme Information Management John Mancini President, AIIM @jmancini77 DigitalLandfill.org Giving Credit Where Credit is Due I didn t make up the term Extreme
More informationAccelerate BI Initiatives With Self-Service Data Discovery And Integration
A Custom Technology Adoption Profile Commissioned By Attivio June 2015 Accelerate BI Initiatives With Self-Service Data Discovery And Integration Introduction The rapid advancement of technology has ushered
More informationThe Emergence of Security Business Intelligence: Risk
The Emergence of Security Business Intelligence: Risk Management through Deep Analytics & Automation Mike Curtis Vice President of Technology Strategy December, 2011 Introduction As an industry we are
More informationEMC ADVERTISING ANALYTICS SERVICE FOR MEDIA & ENTERTAINMENT
EMC ADVERTISING ANALYTICS SERVICE FOR MEDIA & ENTERTAINMENT Leveraging analytics for actionable insight ESSENTIALS Put your Big Data to work for you Pick the best-fit, priority business opportunity and
More informationImprove Cooperation in R&D. Catalyze Drug Repositioning. Optimize Clinical Trials. Respect Information Governance and Security
SINEQUA FOR LIFE SCIENCES DRIVE INNOVATION. ACCELERATE RESEARCH. SHORTEN TIME-TO-MARKET. 6 Ways to Leverage Big Data Search & Content Analytics for a Pharmaceutical Company Improve Cooperation in R&D Catalyze
More informationBig Data. Fast Forward. Putting data to productive use
Big Data Putting data to productive use Fast Forward What is big data, and why should you care? Get familiar with big data terminology, technologies, and techniques. Getting started with big data to realize
More informationTap into Big Data at the Speed of Business
SAP Brief SAP Technology SAP Sybase IQ Objectives Tap into Big Data at the Speed of Business A simpler, more affordable approach to Big Data analytics A simpler, more affordable approach to Big Data analytics
More informationDemystifying Big Data Government Agencies & The Big Data Phenomenon
Demystifying Big Data Government Agencies & The Big Data Phenomenon Today s Discussion If you only remember four things 1 Intensifying business challenges coupled with an explosion in data have pushed
More informationTIBCO Spotfire Helps Organon Bridge the Data Gap Between Basic Research and Clinical Trials
TIBCO Spotfire Helps Organon Bridge the Data Gap Between Basic Research and Clinical Trials Pharmaceutical leader deploys TIBCO Spotfire enterprise analytics platform across its drug discovery organization
More informationHow To Create An Insight Analysis For Cyber Security
IBM i2 Enterprise Insight Analysis for Cyber Analysis Protect your organization with cyber intelligence Highlights Quickly identify threats, threat actors and hidden connections with multidimensional analytics
More informationBIG DATA & ANALYTICS. Transforming the business and driving revenue through big data and analytics
BIG DATA & ANALYTICS Transforming the business and driving revenue through big data and analytics Collection, storage and extraction of business value from data generated from a variety of sources are
More informationMake the right decisions with Distribution Intelligence
Make the right decisions with Distribution Intelligence Bengt Jensfelt, Business Product Manager, Distribution Intelligence, April 2010 Introduction It is not so very long ago that most companies made
More informationBig Data: Business Insight for Power and Utilities
Big Data: Business Insight for Power and Utilities A Look at Big Data By now, most enterprises have encountered the term Big Data. What they encounter less is an understanding of what Big Data means for
More informationTaming Big Data. 1010data ACCELERATES INSIGHT
Taming Big Data 1010data ACCELERATES INSIGHT Lightning-fast and transparent, 1010data analytics gives you instant access to all your data, without technical expertise or expensive infrastructure. TAMING
More informationIBM Analytics. Just the facts: Four critical concepts for planning the logical data warehouse
IBM Analytics Just the facts: Four critical concepts for planning the logical data warehouse 1 2 3 4 5 6 Introduction Complexity Speed is businessfriendly Cost reduction is crucial Analytics: The key to
More informationThe Definitive Guide to Data Blending. White Paper
The Definitive Guide to Data Blending White Paper Leveraging Alteryx Analytics for data blending you can: Gather and blend data from virtually any data source including local, third-party, and cloud/ social
More informationFROM DATA STORE TO DATA SERVICES - DEVELOPING SCALABLE DATA ARCHITECTURE AT SURS. Summary
UNITED NATIONS ECONOMIC COMMISSION FOR EUROPE CONFERENCE OF EUROPEAN STATISTICIANS Working paper 27 February 2015 Workshop on the Modernisation of Statistical Production Meeting, 15-17 April 2015 Topic
More informationSolving the Big Data Intention-Deployment Gap
Whitepaper Solving the Big Data Intention-Deployment Gap Big Data is on virtually every enterprise s to-do list these days. Recognizing both its potential and competitive advantage, companies are aligning
More informationMaking Business Intelligence Easy. White Paper Spreadsheet reporting within a BI framework
Making Business Intelligence Easy White Paper Spreadsheet reporting within a BI framework Contents Overview...4 What is spreadsheet reporting and why does it exist?...5 Risks and issues with spreadsheets
More informationMining for Insight: Rediscovering the Data Archive
WHITE PAPER Mining for Insight: Rediscovering the Data Archive Sponsored by: Iron Mountain Laura DuBois June 2015 Sean Pike EXECUTIVE SUMMARY In the past, the main drivers for data archiving centered on
More informationIBM Cognos TM1. Enterprise planning, budgeting and analysis. Highlights. IBM Software Data Sheet
IBM Software IBM Cognos TM1 Enterprise planning, budgeting and analysis Highlights Reduces planning cycles by as much as 75% and reporting from days to minutes Owned and managed by Finance and lines of
More informationLean manufacturing in the age of the Industrial Internet
Lean manufacturing in the age of the Industrial Internet From Henry Ford s moving assembly line to Taiichi Ohno s Toyota production system, now known as lean production, manufacturers globally have constantly
More informationKPMG Unlocks Hidden Value in Client Information with Smartlogic Semaphore
CASE STUDY KPMG Unlocks Hidden Value in Client Information with Smartlogic Semaphore Sponsored by: IDC David Schubmehl July 2014 IDC OPINION Dan Vesset Big data in all its forms and associated technologies,
More informationAnnex: Concept Note. Big Data for Policy, Development and Official Statistics New York, 22 February 2013
Annex: Concept Note Friday Seminar on Emerging Issues Big Data for Policy, Development and Official Statistics New York, 22 February 2013 How is Big Data different from just very large databases? 1 Traditionally,
More informationNext-Generation Cloud Analytics with Amazon Redshift
Next-Generation Cloud Analytics with Amazon Redshift What s inside Introduction Why Amazon Redshift is Great for Analytics Cloud Data Warehousing Strategies for Relational Databases Analyzing Fast, Transactional
More informationA SAS White Paper: Implementing the Customer Relationship Management Foundation Analytical CRM
A SAS White Paper: Implementing the Customer Relationship Management Foundation Analytical CRM Table of Contents Introduction.......................................................................... 1
More informationATA DRIVEN GLOBAL VISION CLOUD PLATFORM STRATEG N POWERFUL RELEVANT PERFORMANCE SOLUTION CLO IRTUAL BIG DATA SOLUTION ROI FLEXIBLE DATA DRIVEN V
ATA DRIVEN GLOBAL VISION CLOUD PLATFORM STRATEG N POWERFUL RELEVANT PERFORMANCE SOLUTION CLO IRTUAL BIG DATA SOLUTION ROI FLEXIBLE DATA DRIVEN V WHITE PAPER Create the Data Center of the Future Accelerate
More informationANALYTICS STRATEGY: creating a roadmap for success
ANALYTICS STRATEGY: creating a roadmap for success Companies in the capital and commodity markets are looking at analytics for opportunities to improve revenue and cost savings. Yet, many firms are struggling
More informationEnterprise Content Management discovering
Enterprise Content Management discovering content as an asset boost productivity and collaboration Your business technologists. Powering progress Collaboration underpins productivity Every business generates
More informationAutomated Business Intelligence
Automated Business Intelligence Delivering real business value,quickly, easily, and affordably 2 Executive Summary For years now, the greatest weakness of the Business Intelligence (BI) industry has been
More informationBeyond the Data Lake
WHITE PAPER Beyond the Data Lake Managing Big Data for Value Creation In this white paper 1 The Data Lake Fallacy 2 Moving Beyond Data Lakes 3 A Big Data Warehouse Supports Strategy, Value Creation Beyond
More informationSoftware: Driving Innovation for Engineered Products. Page
Software: Driving Innovation for Engineered Products Software in products holds the key to innovations that improve quality, safety, and ease-of-use, as well as add new functions. Software simply makes
More informationData Virtualization and ETL. Denodo Technologies Architecture Brief
Data Virtualization and ETL Denodo Technologies Architecture Brief Contents Data Virtualization and ETL... 3 Summary... 3 Data Virtualization... 7 What is Data Virtualization good for?... 8 Applications
More informationMaking confident decisions with the full spectrum of analysis capabilities
IBM Software Business Analytics Analysis Making confident decisions with the full spectrum of analysis capabilities Making confident decisions with the full spectrum of analysis capabilities Contents 2
More informationBig Data Solutions Ease Financial Services Compliance and Reporting
Big Data Solutions Ease Financial Services Compliance and Reporting Real-time data collection and analysis helps ensure compliance and uncover new business opportunities. THINK SMART. ACT FAST. FLEX YOUR
More informationThe Principles of the Business Data Lake
The Principles of the Business Data Lake The Business Data Lake Culture eats Strategy for Breakfast, so said Peter Drucker, elegantly making the point that the hardest thing to change in any organization
More informationPUSH INTELLIGENCE. Bridging the Last Mile to Business Intelligence & Big Data. 2013 Copyright Metric Insights, Inc.
PUSH INTELLIGENCE Bridging the Last Mile to Business Intelligence & Big Data 2013 Copyright Metric Insights, Inc. INTRODUCTION... 3 CHALLENGES WITH BI... 4 The Dashboard Dilemma... 4 Architectural Limitations
More informationHow To Understand The Benefits Of Big Data
Findings from the research collaboration of IBM Institute for Business Value and Saïd Business School, University of Oxford Analytics: The real-world use of big data How innovative enterprises extract
More informationOvercoming Obstacles to Retail Supply Chain Efficiency and Vendor Compliance
Overcoming Obstacles to Retail Supply Chain Efficiency and Vendor Compliance 0 GreenLionDigital.com How process automation, data integration and visibility, advanced analytics, and collaboration improve
More information*Big Risks. Vast stores of information can provide organizations endless insight on their business. Managing and safeguarding all that data is
*Big Risks. Vast stores of information can provide organizations endless insight on their business. Managing and safeguarding all that data is another story. S tories are emerging from the campaign trail
More informationWHITEPAPER BIG DATA GOVERNANCE. How To Avoid The Pitfalls of Big Data Governance? www.analytixds.com
BIG DATA GOVERNANCE How To Avoid The Pitfalls of Big Data Governance? of The need to provide answers quickly... 3 You can t measure what you don t manage... 3 Aligning the overall architecture with the
More informationDelivering Real-Time Business Value for Healthcare Providers SAP Business Suite Powered by SAP HANA
Delivering Real-Time Business Value for Healthcare Providers SAP Business Suite Powered by SAP HANA July 2013 Public The real-time opportunity Best-run healthcare facilities improve patient outcomes by
More informationCloud Computing on a Smarter Planet. Smarter Computing
Cloud Computing on a Smarter Planet Smarter Computing 2 Cloud Computing on a Smarter Planet As our planet gets smarter more instrumented, interconnected and intelligent the underlying infrastructure needs
More informationSymantec Global Intelligence Network 2.0 Architecture: Staying Ahead of the Evolving Threat Landscape
WHITE PAPER: SYMANTEC GLOBAL INTELLIGENCE NETWORK 2.0.... ARCHITECTURE.................................... Symantec Global Intelligence Network 2.0 Architecture: Staying Ahead of the Evolving Threat Who
More informationTraditional BI vs. Business Data Lake A comparison
Traditional BI vs. Business Data Lake A comparison The need for new thinking around data storage and analysis Traditional Business Intelligence (BI) systems provide various levels and kinds of analyses
More informationLost in Space? Methodology for a Guided Drill-Through Analysis Out of the Wormhole
Paper BB-01 Lost in Space? Methodology for a Guided Drill-Through Analysis Out of the Wormhole ABSTRACT Stephen Overton, Overton Technologies, LLC, Raleigh, NC Business information can be consumed many
More informationHow Big Data is Different
FALL 2012 VOL.54 NO.1 Thomas H. Davenport, Paul Barth and Randy Bean How Big Data is Different Brought to you by Please note that gray areas reflect artwork that has been intentionally removed. The substantive
More informationMaking Business Intelligence Easy. White Paper Agile Business Intelligence
Making Business Intelligence Easy White Paper Agile Business Intelligence Contents Overview... 3 The need for Agile Business Intelligence... 4 Technology: Critical features of an Agile Business Intelligence
More informationIn-Depth Understanding: Teaching Search Engines to Interpret Meaning
P O I N T O F V I E W In-Depth Understanding: Teaching Search Engines to Interpret Meaning By C. DAVID SEUSS Northern Light Group, LLC, Cambridge, MA 02141 USA If a person from 1994 jumped forward into
More informationThe Power of Analysis Framework
All too often, users must create real-time planning and analysis reports with static and inconsistent sources of information. Data is locked in an Excel spreadsheet or a rigidly customized application
More informationThe Network Approach to Inventory Management
The Network Approach to Inventory Management Getting Accurate Inventory Information Across the Entire Supply Chain Stream Creates Healthy Companies A GT Nexus White Paper The Inventory Challenge The problem
More informationCisco Data Preparation
Data Sheet Cisco Data Preparation Unleash your business analysts to develop the insights that drive better business outcomes, sooner, from all your data. As self-service business intelligence (BI) and
More informationThought Paper: Business Process Automation
Thought Paper: Business Process Automation Rapid development and deployment of automation to eliminate repetitive, labor intensive, and computer related tasks throughout the enterprise. Joe Kosco Vice
More informationMORE DATA - MORE PROBLEMS
July 2014 MORE DATA - MORE PROBLEMS HOW CAN SMBs ADDRESS DATA ISSUES? Data Source In this report, Mint Jutras references data collected from its 2014 Enterprise Solution Study, which investigated goals,
More informationManagement Consulting Systems Integration Managed Services WHITE PAPER DATA DISCOVERY VS ENTERPRISE BUSINESS INTELLIGENCE
Management Consulting Systems Integration Managed Services WHITE PAPER DATA DISCOVERY VS ENTERPRISE BUSINESS INTELLIGENCE INTRODUCTION Over the past several years a new category of Business Intelligence
More informationThe Liaison ALLOY Platform
PRODUCT OVERVIEW The Liaison ALLOY Platform WELCOME TO YOUR DATA-INSPIRED FUTURE Data is a core enterprise asset. Extracting insights from data is a fundamental business need. As the volume, velocity,
More informationCloud Integration and the Big Data Journey - Common Use-Case Patterns
Cloud Integration and the Big Data Journey - Common Use-Case Patterns A White Paper August, 2014 Corporate Technologies Business Intelligence Group OVERVIEW The advent of cloud and hybrid architectures
More informationSolving the Big Data Intention-Deployment Gap
Solving the Big Data Intention-Deployment Gap Big Data is on virtually every enterprise s to-do list these days. Recognizing both its potential and competitive advantage, companies are aligning a vast
More informationOperational Excellence, Data Driven Transformation Now Available at American Hospitals
Operational Excellence, Data Driven Transformation Now Available at American Hospitals It's Time to Get LEAN White Paper Operational Excellence, Data Driven Transformation Now Available at American Hospitals
More informationReimagining Business with SAP HANA Cloud Platform for the Internet of Things
SAP Brief SAP HANA SAP HANA Cloud Platform for the Internet of Things Objectives Reimagining Business with SAP HANA Cloud Platform for the Internet of Things Connect, transform, and reimagine Connect,
More informationBANKING ON WILL BIG DATA TRANSFORM THE CUSTOMER EXPERIENCE? A Retail Banking perspective
BANKING ON WILL BIG DATA TRANSFORM THE CUSTOMER EXPERIENCE? A Retail Banking perspective Big data provides an opportunity to deliver exceptional customer experiences and competitive advantage in an industry
More informationEmail Archiving Whitepaper. Why Email Archiving is Essential (and Not the Same as Backup) www.fusemail.com
Why Email Archiving is Essential (and Not the Same as Backup) Why Email Archiving is Essential (and Not the Same as Backup) If your job depended on it, could you clearly explain right this moment the principal
More informationUnderstanding Your Customer Journey by Extending Adobe Analytics with Big Data
SOLUTION BRIEF Understanding Your Customer Journey by Extending Adobe Analytics with Big Data Business Challenge Today s digital marketing teams are overwhelmed by the volume and variety of customer interaction
More informationForward Thinking for Tomorrow s Projects Requirements for Business Analytics
Seilevel Whitepaper Forward Thinking for Tomorrow s Projects Requirements for Business Analytics By: Joy Beatty, VP of Research & Development & Karl Wiegers, Founder Process Impact We are seeing a change
More informationBetter Business Analytics with Powerful Business Intelligence Tools
Better Business Analytics with Powerful Business Intelligence Tools Business Intelligence Defined There are many interpretations of what BI (Business Intelligence) really is and the benefits that it can
More informationWatson to Gain Ability to See with Planned $1B Acquisition of Merge Healthcare Deal Brings Watson Technology Together with Leader in Medical Images
Watson to Gain Ability to See with Planned $1B Acquisition of Merge Healthcare Deal Brings Watson Technology Together with Leader in Medical Images Armonk, NY and CHICAGO -- [August 6, 2015]: IBM (NYSE:
More informationTapping the benefits of business analytics and optimization
IBM Sales and Distribution Chemicals and Petroleum White Paper Tapping the benefits of business analytics and optimization A rich source of intelligence for the chemicals and petroleum industries 2 Tapping
More informationALIGNING DATA SCIENCE MAKING ORGANIZATIONAL STRUCTURE WORK
ALIGNING DATA SCIENCE MAKING ORGANIZATIONAL STRUCTURE WORK by Ezmeralda Khalil Principal Booz Allen Hamilton Katherine Wood Lead Associate Booz Allen Hamilton As commercial and government entities develop
More informationICD-10 Advantages Require Advanced Analytics
Cognizant 20-20 Insights ICD-10 Advantages Require Advanced Analytics Compliance alone will not deliver on ICD-10 s potential to improve quality of care, reduce costs and elevate efficiency. Organizations
More informationBIG DATA: A TOP- OF-THE- AGENDA ISSUE FOR BUSINESS LEADERS
BIG DATA: A TOP- OF-THE- AGENDA ISSUE FOR BUSINESS LEADERS 1 Big data is a top-of-the-agenda issue for business leaders Contents The current buyer context...1 Defining analytics capabilities to serve business
More informationWhy Data Management Matters Right Now
Why Data Management Matters Right Now Why Data Management Matters Right Now Chapter 1: The Exponential Growth of Data Chapter 2: Data Retention Policies and Compliance Chapter 3: Accessing your Data Chapter
More informationThe Future-ready Enterprise Simplicity, flexibility, and the art of delivering business outcomes.
The Future-ready Enterprise Simplicity, flexibility, and the art of delivering business outcomes. Every day business leaders make decisions designed to move their companies toward specific outcomes. Whether
More informationORGANIZATIONAL PROFILES. Getting Past the Bumps in the Road
ORGANIZATIONAL PROFILES Getting Past the Bumps in the Road With the explosion of data in virtually every aspect of society, a growing number of organizations are seeking to take full advantage of analytics
More informationA Hurwitz white paper. Inventing the Future. Judith Hurwitz President and CEO. Sponsored by Hitachi
Judith Hurwitz President and CEO Sponsored by Hitachi Introduction Only a few years ago, the greatest concern for businesses was being able to link traditional IT with the requirements of business units.
More informationCA Technologies Big Data Infrastructure Management Unified Management and Visibility of Big Data
Research Report CA Technologies Big Data Infrastructure Management Executive Summary CA Technologies recently exhibited new technology innovations, marking its entry into the Big Data marketplace with
More informationBIG DATA SURVEY 2014 SURVEY
BIG DATA SURVEY 2014 SURVEY There has been a tremendous amount of hype around Big Data projects and applications in recent years, but relatively little quantifiable evidence proving what, if any, business
More informationData Virtualization Usage Patterns for Business Intelligence/ Data Warehouse Architectures
DATA VIRTUALIZATION Whitepaper Data Virtualization Usage Patterns for / Data Warehouse Architectures www.denodo.com Incidences Address Customer Name Inc_ID Specific_Field Time New Jersey Chevron Corporation
More informationBIG DATA FUNDAMENTALS
BIG DATA FUNDAMENTALS Timeframe Minimum of 30 hours Use the concepts of volume, velocity, variety, veracity and value to define big data Learning outcomes Critically evaluate the need for big data management
More informationIBM Cognos Performance Management Solutions for Oracle
IBM Cognos Performance Management Solutions for Oracle Gain more value from your Oracle technology investments Highlights Deliver the power of predictive analytics across the organization Address diverse
More informationTRANSFORMING MEETINGS & EVENTS WITH BETTER INTELLIGENCE
TRANSFORMING MEETINGS & EVENTS WITH BETTER INTELLIGENCE MAKING YOUR EVENTS SMARTER C-suite executives everywhere are rushing to formalize their Big Data strategies. Articles on the subject abound in the
More information