Building Successful Big Data Solutions

Size: px
Start display at page:

Download "Building Successful Big Data Solutions"

Transcription

1 Building Successful Big Data Solutions

2 2 Executive Summary The decision to invest in and leverage the widespread Big Data 1 revolution, whether you re a large multinational corporation or the smallest sole- proprietorship, is no longer an option as data growth has outstripped the ability for people and 20 th Century technology to make sense of it all. Differentiation and successful execution requires a 21 st Century approach to intelligent analytics, which go beyond the ability to count and sort methodologies, but rather approach all data automatically, whether structured or unstructured. The successful business requires tools which continuously learn and reveal actionable and unforeseen connections, while also being able to flexibly move between legacy data (which may reside in highly organized silos) and unstructured data generated in real- time. Atigeo s xpatterns TM intelligent Big Data platform is capable of providing the required level of analytics visibility into data, both structured and unstructured, against any application now and into the future. According to McKinsey & Company 2, there is a growing shortage of both data managers and skilled data analysts necessary to handle the continued exponential unstructured data growth. Current technologies require multiple analyst touch points, volume limits and strict data policies; today s solutions must lead with complex and constantly evolving sets of open source technologies. However, solutions must go beyond the ability to store, manage, and retrieve the copious amounts of data (that is simply the point of entry), and provide advanced analytics, which can lead to quantifiably more effective marketing and optimized operations. The question to ask is, Does my solution just enable search, recommendations and classifications over large volumes of data, or does it also achieve unprecedented relevance necessary for robust ROI? Additionally, privacy, compliance and security of one s data is paramount. As data explodes, these concerns explode with it; xpatterns was designed innately to solve for these concerns in a Big Data world. Merchants, governments, and hackers are all looking for ways to leverage personal data and consumers are right to be wary about the shifting boundary between more services and less privacy. The question to ask is, Does my Big Data platform secure my data out of the box? As noted above, the shortage of personnel is magnified by the investment of time required to retrain current IT staff, as the Big Data learning curve is significantly higher due to the number of technologies and components involved. Companies must decide: Do I train my personnel or do I partner with 21st Century technology? While all of the above is challenging, Atigeo s seven- year head start in addressing Big Data analytics has ensured xpatterns is the appropriate application framework for building enterprise- grade, intelligent Big Data applications, which can be deployed on- premise or in the cloud with minimal IT support required. The robust Software Developers Kit allows data scientists to easily plug and play to try with the best in class tools. In summary, Atigeo s xpatterns platform makes it easy to combine different kinds of intelligent components in Big Data applications, including those built by our partners and available in open source. 1 Big Data definition: the unprecedented growth in the volume, velocity and variety of data in our world. 2 The McKinsey Global Institute: Big Data: The next frontier for innovation, competition and productivity June 2011

3 3 The Big Data Opportunity Big Data has been defined to address the unprecedented growth in the volume, velocity, variety; in addition Atigeo believes it is necessary to address both the visualization of data in our world and its accessibility for all. This explosion in unstructured and semi- structured data is expected to account for 90% of newly created data going forward 3. This opens up significant business opportunities to leverage Big Data through advanced analytics that tie directly into business processes and applications. As the following diagram presented by McKinsey & Company in September shows, early adopters in this space have significantly outperformed their respective markets. The question today is no longer about whether or not a company should invest in Big Data Analytics to stay competitive, but how to gain insight hidden beneath the surface of the data as well as lower the total cost of ownership and the time to market in order to increase their chance of success and return. The difference between Big Data and traditional, smaller transactional data sets is that Big Data, being large in sample, provides more insightful patterns when applying advanced analytics such as statistical analysis, machine learning, data mining, natural language processing, information retrieval and predictive analytics in automated ways, otherwise your ability to unlock the value of your data (relevance) will dramatically fall off because there aren t enough people on the planet to analyze and structure for this global data growth. 3 J.P. Morgan Big Data Primer June McKinsey & Company Presentation at the September 2011 Strata Conference

4 4 However, according to a 2011 analysis by McKinsey & Company 5, for the first time in history there is a current shortage of 1.5 million data- savvy managers to tackle the unstructured data relevant to enterprises. The diagram on the right summarizes the general trend we see where the combination of massive amounts of data (volume), coming from multiple sources (variety) at real time (velocity), causes traditional approaches (in particular those that rely on human tagging, prioritization and analysis) to become ineffective or impractical. Atigeo believes that we are at an inflection, or decision point, where the growth of unstructured data overwhelms the growth of analysts who identify structure within unstructured data. Thus, relevance falls off if no adjustment is made to handle unstructured data. This gap will grow exponentially for the foreseeable future. Hence, the collection, analysis and integration of Big Data into business operations must be automated and requires an expanded portfolio of technologies. For enterprises to capture the Big Data opportunity effectively, accessibility in their Big Data solution is extremely paramount. Access is the mean to democratize the Big Data tools to empower every employee throughout the organization to maximize value of the data for the company, instead of leaving the job to a small group of specialists. Therefore, building successful Big Data solutions is about taking advantage of volume, velocity, variety and visualization through analytics and making it accessible to all. Implementing Big Data Solutions Our framework for Big Data analytics implementation, successfully applied across multiple verticals to date, confirms that the best solution requires a different technology mix per customer, substantial domain- specific knowledge and data, and multiple iterations of data- driven continuous improvement. Until now, there has been no end- to- end solution that fits all these criteria; we expect to see tremendous advancements in technology in the next few years from both incumbents and new entrants. Companies must therefore consider a platform that is flexible in quickly adopting new technologies, for both distributed data processing and advanced analytics, as they become available and enterprise- ready. In addition, such a platform must also have the ability to comply with each company s unique requirements while leveraging existing data and infrastructure. Traditional database technologies, analytics, etc. have served industry well until recently where in the 21 st Century we can take advantage of real time advances available in Open Source, across high speed networks, breakthroughs in compute power and systems, and advances in intelligence technologies like xpatterns are game changing. Introducing xpatterns xpatterns is an application framework for building enterprise- grade, intelligent Big Data applications and an abstraction platform which can leverage all these advances by ISVs, Open Source community technologies, NLP, machine learning, semantic, academics, etc.. Our roadmap is guided by our belief 5 The McKinsey Global Institute: see footnote 2

5 5 that the opportunity to capture value of Big Data is through access, analytics and visualization. xpatterns democratizes the current technologies by abstracting the complexity of usage of i.e. open source Hadoop framework (Access) and adding ever increasing proprietary and open source Analytics and Visualization tools to enable automated and easy manipulation of data to fit all business needs. It can be deployed either on- premise or in the cloud. xpatterns provides an SDK for data scientists to easily configure plug- and- play components and to experiment with best in class tools, reusing and integrating with the company s existing assets. Data scientists can then directly deploy apps as web services or analytical jobs, providing a seamless transition from analysis to production. The runtime environment (Hadoop, NoSQL, search, etc.) is completely abstracted away, allowing for faster time to market, no need for in- house expertise and easy transition between underlying technologies. xpatterns what s included out of the box? Distributed Processing Scalable, reliable processing Scalable, reliable storage NoSql (key/value) storage Pig & Hive queries Workflows & Scheduling High availability Backups Auto- scaling Search, filtering, faceting Real- time dataset updates Shared schema mgmt. Advanced Analytics Natural language toolkit Supervised learning Unsupervised learning Concept extraction Ontologies Plotting & Visualization Information Retrieval Data Mining Scientific computing Predictive analytics Inference Framework Features Create & deploy apps Scheduled workflows Data ingestion, push or pull Normalize, filter and de- dup incoming data Plug & play analytical tools Continuous measurement Automated feedback loop Data lifecycle management Logging & monitoring Personalization On the next page is the xpatterns architectural diagram consisting of the infrastructure layer, horizontal and domain specific intelligence layer and development and administrative environment layer. The framework is designed to achieve flexibility for customers to choose the right intelligence to solve their specific Big Data problem using a simple high- level programming language. While customers focus on business solutions, xpatterns take care of Big Data environment. Thus, xpatterns can lower the barrier of entry for any enterprises or application to take advantage of Big Data opportunities.

6 6 Intelligence Components xpatterns makes it easy to combine different kinds of intelligence components in Big Data applications. Some of these components are open source including popular Python libraries such as nltk 6 for natural language processing, scikit 7 for machine learning and matplotlib 8 for visualization. Additional intelligence components are those built by our partners such as IBM s SystemT 9 and SystemML 10. The third category learn.org/stable/

7 7 of components comprises patented innovations enabling xpatterns to deliver better results, using algorithms available by Atigeo and exposed through a rich set of APIs. Examples of these are: Relevance: xpatterns Relevance takes a "relevance discovery" approach that delivers on the promise of deriving actionable intelligence from an enterprise's disparate sources of structured and unstructured data. xpatterns automatically creates and dynamically maintains semantic ontologies known as domain experts (DEs). At the core of the relevance technology is the creation of high- quality DEs in near real- time. The DE is built as a Relevance Neural Network (RNN) that maps relationships between a set of terms (i.e., semantic concepts) and related terms (output layer), intermediated by context (i.e., documents or articles). The network weights are initialized (or bootstrapped) with statistically optimal values based on frequency statistics. Thereafter, the weights are strengthened or weakened through training by live interaction with users, as well as with new data. This learning capability enables better relevance by leveraging the wisdom of crowds. The figure on the left below shows a depiction of a DE RNN; the figure on the right is an xpatterns visualization of network relationships, showing relevant documents for a concept and relevant concepts for a document: A DE captures and represents relationships between concepts within a given domain. DEs are created automatically by analyzing and processing large bodies of unstructured text information about the domain. They can be leveraged to determine indirect semantic relationships between queried concepts and related concepts, and to facilitate understanding of the relevance of a specific document to a specific concept. DEs represent "IsAssociatedWith" relationships for domains, derived simply from reading and reviewing large bodies of unstructured text information about a given area of interest. Inference: xpatterns Inference delivers complex predictions from evidence. Combined with a Bayesian Model Average (BMA) approach to integrate user preferences embodied in a Bayesian network (BN), xpatterns Inference can provide higher accuracy even when collective preferences are sparse. The power of Inference is attributable to its ability to integrate evidence from different domains at various levels of scope in a scalable way.

8 8 Inference incorporates ontological information in the task of prediction. This information can be captured through a representation of DEs thereby allowing the incorporation of unstructured information, which is particularly well suited to cold- start prediction scenarios. Cold- start prediction describes a situation where the data sample is still small and forming, and there is not enough sample to make prediction using traditional statistics models. An example is shown in the diagram below, where user A provided a small set of cuisine preferences and the task is to infer user A s other preferences on cuisines not listed. The algorithm takes into account the preferences of all users and the additional relationship weightings represented by Domain Experts to infer the likelihood of user A s other preferences in cuisine. This allows us to calculate with high confidence the probability whether A likes Chinese Food even if preferences collected from the population are too small of a sample, especially in the beginning of the sample collection process. As the scenario evolves, personal or local evidence grows in tandem with population level behavior. This evidence may be structured or unstructured. The three main components of ontology, personal/local behavior and population level behavior combine to render optimally informed inference. Classification: xpatterns Classification infers type or class from complex information. Classification integrates structured and unstructured data into classification scenarios, which may have large scales in the volume of data, the size of the input space and the number of possible classes that may be inferred. Classification develops deeper understanding of unstructured data through processing natural language to decipher complex relationships. The deeper understanding enables qualities of sentiment, time and reference, which are applied to distinguish among subtly distinct classes. For example, a domain- specific classification tool is incorporated for healthcare professionals, leveraging the Unified Medical Language System (UMLS ) from US National Library of Medicine as well as International Classification of Diseases (ICD- 9 and ICD- 10) data to return precise classifications for unstructured medical text.

9 9 Cooperative Distributed Inferencing (CDI): xpatterns CDI is a new paradigm for Inferencing and Optimal Control in real time. It is a distributed optimziation approach with built- in synchronication in a continuous optimization of all types of rules, soft and hard rules. The paradigm for inferencing converts multiple knowledge bases from exponential complexity to polynomic complexity. Then, constraints are build with a pareto strategy that synchronize different rules to form a converging optimal result. The application of this inferencing model is vast. One example is optimizing the power grid, which has multiple knowledge bases and rules that are not all taken into account by the out- dated algorithms. This leads to local ad- hoc adjustments and empirical corrections, which are sub- optimal. The figure below shows the current model and an xpatterns CDI model. In summary, the three main differentiating points for CDI are: 1. Deal with large size rule sets in real time 2. Express variety of rules and constraint with optimization 3. Distributed cooperation between independent nodes, without needing trust among nodes

10 10 Personas - The unique xpatterns privacy model makes it possible for individual users to create, build and control their own digital personas. These anonymous, secure profiles keep users identities completely private while accurately reflecting their interests and behaviors in the digital landscape. In this way, it becomes possible to deliver highly relevant, personalized content and experiences to individuals without learning those individuals actual identities; instead, only their relevance scores are visible. Here is how the xpatterns persona module works: All content types are given a relevance score based on the personalized attributes of the user The user profile can be initialized from existing enterprise data sources Profile attributes can be dynamically updated from real- time inferred or explicit behavioral data Applications can be designed to give consumers full management of their personas Persona attributes are unstructured, meaning they don t have to be selected from static lists NLP- P (Natural Language Pre- Processing): Atigeo has a set of healthcare domain- specific natural language processing built on top of the existing open source projects and mutliple sources of references. The pre- processing, which can be applied to any domain, consists of body and sentence extraction, negation tagging, normalization, lemmatization and removal of stop words. This is used to improve overall relevance of xpatterns at time of generation of corpuses and at query time. Applications Atigeo has been working with several partners to solve their real life important Big Data analytic challenges. The following are some examples: xpatterns Clinical Auto- Coding: Often times, there is an under- coding problem where hospitals are not billing the insurance companies correctly to get paid an accurate amount. Hospitals are facing a shortage of trained staff to translate Electronic Medical Records (EMRs) to required ICD- 9 and ICD- 10, CPT, HCPCS, APC Grouper, Charge Master, DRG codes and more. Atigeo has developed an intelligence system to automatically suggest correct codes for any number of EMRs. We are also able to take big data sets of past EMRs and run them through our intelligence system to perform an audit or add more accurate

11 11 codes, creating a complete view of actual clinical services for compliance or research purposes. In addition to NLP, the product assembles multiple intelligent methodologies including inference, classification, ontology and machine learning that differentiate Atigeo from its competitors. Research - Document Discovery: As our analytics algorithms are specially designed to solve unstructured data relevance problems, we have applied them to a large set of unstructured text documents as our first Big Data usage scenario. We processed gigabytes of medical research documents (PubMed) and patents (USPTO) by assigning relevance scores and generating domain concepts. Users can submit search queries to find relevant documents organized in clusters. The platform continues to improve through applying machine learning to users interactions with the documents. Through xpatterns Relevance, document discovery is no longer a linear search problem. We have developed a visualization tool that allows users to easily navigate among clusters of many relevant documents and sometimes even discover relevant concepts and documents that are non- obvious to the original search query.

12 12 Clinical Analytics/Intelligence as a Service: Atigeo developed a clinical intelligence layer on the xpatterns framework. With easy access to pre- loaded medical domain toolboxes in the cloud, users can run analytics against their own large data set such as EMRs. xpatterns analytical toolset allows users to do natural language data mining, correlations, etc. to find insightful patterns on a given research topic. For this specific use case, there are tremendous benefits of leveraging a cloud service, which xpatterns supports. Benefits include: 1. Scalability and agility: Initially, processing Big Data requires a large number of servers, which will then not be required once the data is processed and the results stored. Cloud services provide the flexibility for scaling up and down as needed. Leveraging the cloud, an enterprise can optimize their processing power without waste. 2. Deployment and maintenance cost: Upfront investment is high for infrastructure deployments and skilled staffing. The cost of keeping up with the latest software is expensive when advancement is happening very rapidly in this space. 3. Time: Cloud flexibility takes deployment time out of the equation, and it also gives an enterprise the ability to control turnaround time for output. Conclusion The question today is no longer about whether or not a company should invest in Big Data Analytics to stay competitive, but how to gain insight hidden beneath the surface of the data as well as how to lower the total cost of ownership and improve time to market in the face of these challenges. Big Data brings both opportunities and challenges are met by xpatterns, which lowers barriers to entry in the Big Data space by taking away the complexity and advancing the insight. Infrastructure and talent acquisition should not be any enterprise s major concern. The focus should be on the solution, which means Atigeo s xpatterns is the enabling Big Data Intelligence platform for the 21 st Century.

THE FUTURE OF CODING IS NOW

THE FUTURE OF CODING IS NOW THE FUTURE OF CODING IS NOW xpatterns Computer-Assisted Coding: Features and Benefits: Automatically generates medical codes directly from clinical encounter notes Maps clinical codes to appropriate billing

More information

Big Data 101: Harvest Real Value & Avoid Hollow Hype

Big Data 101: Harvest Real Value & Avoid Hollow Hype Big Data 101: Harvest Real Value & Avoid Hollow Hype 2 Executive Summary Odds are you are hearing the growing hype around the potential for big data to revolutionize our ability to assimilate and act on

More information

Extend your analytic capabilities with SAP Predictive Analysis

Extend your analytic capabilities with SAP Predictive Analysis September 9 11, 2013 Anaheim, California Extend your analytic capabilities with SAP Predictive Analysis Charles Gadalla Learning Points Advanced analytics strategy at SAP Simplifying predictive analytics

More information

Capitalize on Big Data for Competitive Advantage with Bedrock TM, an integrated Management Platform for Hadoop Data Lakes

Capitalize on Big Data for Competitive Advantage with Bedrock TM, an integrated Management Platform for Hadoop Data Lakes Capitalize on Big Data for Competitive Advantage with Bedrock TM, an integrated Management Platform for Hadoop Data Lakes Highly competitive enterprises are increasingly finding ways to maximize and accelerate

More information

Oracle Big Data Discovery Unlock Potential in Big Data Reservoir

Oracle Big Data Discovery Unlock Potential in Big Data Reservoir Oracle Big Data Discovery Unlock Potential in Big Data Reservoir Gokula Mishra Premjith Balakrishnan Business Analytics Product Group September 29, 2014 Copyright 2014, Oracle and/or its affiliates. All

More information

BIG DATA IN THE CLOUD : CHALLENGES AND OPPORTUNITIES MARY- JANE SULE & PROF. MAOZHEN LI BRUNEL UNIVERSITY, LONDON

BIG DATA IN THE CLOUD : CHALLENGES AND OPPORTUNITIES MARY- JANE SULE & PROF. MAOZHEN LI BRUNEL UNIVERSITY, LONDON BIG DATA IN THE CLOUD : CHALLENGES AND OPPORTUNITIES MARY- JANE SULE & PROF. MAOZHEN LI BRUNEL UNIVERSITY, LONDON Overview * Introduction * Multiple faces of Big Data * Challenges of Big Data * Cloud Computing

More information

III Big Data Technologies

III Big Data Technologies III Big Data Technologies Today, new technologies make it possible to realize value from Big Data. Big data technologies can replace highly customized, expensive legacy systems with a standard solution

More information

Transforming the Telecoms Business using Big Data and Analytics

Transforming the Telecoms Business using Big Data and Analytics Transforming the Telecoms Business using Big Data and Analytics Event: ICT Forum for HR Professionals Venue: Meikles Hotel, Harare, Zimbabwe Date: 19 th 21 st August 2015 AFRALTI 1 Objectives Describe

More information

BIG Data Analytics Move to Competitive Advantage

BIG Data Analytics Move to Competitive Advantage BIG Data Analytics Move to Competitive Advantage where is technology heading today Standardization Open Source Automation Scalability Cloud Computing Mobility Smartphones/ tablets Internet of Things Wireless

More information

Databricks. A Primer

Databricks. A Primer Databricks A Primer Who is Databricks? Databricks was founded by the team behind Apache Spark, the most active open source project in the big data ecosystem today. Our mission at Databricks is to dramatically

More information

Databricks. A Primer

Databricks. A Primer Databricks A Primer Who is Databricks? Databricks vision is to empower anyone to easily build and deploy advanced analytics solutions. The company was founded by the team who created Apache Spark, a powerful

More information

Augmented Search for Web Applications. New frontier in big log data analysis and application intelligence

Augmented Search for Web Applications. New frontier in big log data analysis and application intelligence Augmented Search for Web Applications New frontier in big log data analysis and application intelligence Business white paper May 2015 Web applications are the most common business applications today.

More information

Big Data for the Rest of Us Technical White Paper

Big Data for the Rest of Us Technical White Paper Big Data for the Rest of Us Technical White Paper Treasure Data - Big Data for the Rest of Us 1 Introduction The importance of data warehousing and analytics has increased as companies seek to gain competitive

More information

Increase Agility and Reduce Costs with a Logical Data Warehouse. February 2014

Increase Agility and Reduce Costs with a Logical Data Warehouse. February 2014 Increase Agility and Reduce Costs with a Logical Data Warehouse February 2014 Table of Contents Summary... 3 Data Virtualization & the Logical Data Warehouse... 4 What is a Logical Data Warehouse?... 4

More information

Big Data at Cloud Scale

Big Data at Cloud Scale Big Data at Cloud Scale Pushing the limits of flexible & powerful analytics Copyright 2015 Pentaho Corporation. Redistribution permitted. All trademarks are the property of their respective owners. For

More information

SAP Predictive Analytics Roadmap Charles Gadalla SAP SESSION CODE: #####

SAP Predictive Analytics Roadmap Charles Gadalla SAP SESSION CODE: ##### SAP Predictive Analytics Roadmap Charles Gadalla SAP SESSION CODE: ##### LEARNING POINTS What are SAP s Advanced Analytics offerings Advanced Analytics gives a competitive advantage, it can no longer be

More information

Apigee Insights Increase marketing effectiveness and customer satisfaction with API-driven adaptive apps

Apigee Insights Increase marketing effectiveness and customer satisfaction with API-driven adaptive apps White provides GRASP-powered big data predictive analytics that increases marketing effectiveness and customer satisfaction with API-driven adaptive apps that anticipate, learn, and adapt to deliver contextual,

More information

Hexaware E-book on Predictive Analytics

Hexaware E-book on Predictive Analytics Hexaware E-book on Predictive Analytics Business Intelligence & Analytics Actionable Intelligence Enabled Published on : Feb 7, 2012 Hexaware E-book on Predictive Analytics What is Data mining? Data mining,

More information

Accelerate BI Initiatives With Self-Service Data Discovery And Integration

Accelerate BI Initiatives With Self-Service Data Discovery And Integration A Custom Technology Adoption Profile Commissioned By Attivio June 2015 Accelerate BI Initiatives With Self-Service Data Discovery And Integration Introduction The rapid advancement of technology has ushered

More information

IBM Cloud Security Draft for Discussion September 12, 2011. 2011 IBM Corporation

IBM Cloud Security Draft for Discussion September 12, 2011. 2011 IBM Corporation IBM Cloud Security Draft for Discussion September 12, 2011 IBM Point of View: Cloud can be made secure for business As with most new technology paradigms, security concerns surrounding cloud computing

More information

Mitra Innovation Leverages WSO2's Open Source Middleware to Build BIM Exchange Platform

Mitra Innovation Leverages WSO2's Open Source Middleware to Build BIM Exchange Platform Mitra Innovation Leverages WSO2's Open Source Middleware to Build BIM Exchange Platform May 2015 Contents 1. Introduction... 3 2. What is BIM... 3 2.1. History of BIM... 3 2.2. Why Implement BIM... 4 2.3.

More information

Big Data Discovery: Five Easy Steps to Value

Big Data Discovery: Five Easy Steps to Value Big Data Discovery: Five Easy Steps to Value Big data could really be called big frustration. For all the hoopla about big data being poised to reshape industries from healthcare to retail to financial

More information

Augmented Search for IT Data Analytics. New frontier in big log data analysis and application intelligence

Augmented Search for IT Data Analytics. New frontier in big log data analysis and application intelligence Augmented Search for IT Data Analytics New frontier in big log data analysis and application intelligence Business white paper May 2015 IT data is a general name to log data, IT metrics, application data,

More information

Cisco Data Preparation

Cisco Data Preparation Data Sheet Cisco Data Preparation Unleash your business analysts to develop the insights that drive better business outcomes, sooner, from all your data. As self-service business intelligence (BI) and

More information

Big Data: Rethinking Text Visualization

Big Data: Rethinking Text Visualization Big Data: Rethinking Text Visualization Dr. Anton Heijs anton.heijs@treparel.com Treparel April 8, 2013 Abstract In this white paper we discuss text visualization approaches and how these are important

More information

White Paper. Software Development Best Practices: Enterprise Code Portal

White Paper. Software Development Best Practices: Enterprise Code Portal White Paper Software Development Best Practices: Enterprise Code Portal An Enterprise Code Portal is an inside the firewall software solution that enables enterprise software development organizations

More information

BIG DATA WITHIN THE LARGE ENTERPRISE 9/19/2013. Navigating Implementation and Governance

BIG DATA WITHIN THE LARGE ENTERPRISE 9/19/2013. Navigating Implementation and Governance BIG DATA WITHIN THE LARGE ENTERPRISE 9/19/2013 Navigating Implementation and Governance Purpose of Today s Talk John Adler - Data Management Group Madina Kassengaliyeva - Think Big Analytics Growing data

More information

HOW TO MAKE SENSE OF BIG DATA TO BETTER DRIVE BUSINESS PROCESSES, IMPROVE DECISION-MAKING, AND SUCCESSFULLY COMPETE IN TODAY S MARKETS.

HOW TO MAKE SENSE OF BIG DATA TO BETTER DRIVE BUSINESS PROCESSES, IMPROVE DECISION-MAKING, AND SUCCESSFULLY COMPETE IN TODAY S MARKETS. HOW TO MAKE SENSE OF BIG DATA TO BETTER DRIVE BUSINESS PROCESSES, IMPROVE DECISION-MAKING, AND SUCCESSFULLY COMPETE IN TODAY S MARKETS. ALTILIA turns Big Data into Smart Data and enables businesses to

More information

locuz.com Big Data Services

locuz.com Big Data Services locuz.com Big Data Services Big Data At Locuz, we help the enterprise move from being a data-limited to a data-driven one, thereby enabling smarter, faster decisions that result in better business outcome.

More information

The Future of Data Management

The Future of Data Management The Future of Data Management with Hadoop and the Enterprise Data Hub Amr Awadallah (@awadallah) Cofounder and CTO Cloudera Snapshot Founded 2008, by former employees of Employees Today ~ 800 World Class

More information

The Liaison ALLOY Platform

The Liaison ALLOY Platform PRODUCT OVERVIEW The Liaison ALLOY Platform WELCOME TO YOUR DATA-INSPIRED FUTURE Data is a core enterprise asset. Extracting insights from data is a fundamental business need. As the volume, velocity,

More information

Augmented Search for Software Testing

Augmented Search for Software Testing Augmented Search for Software Testing For Testers, Developers, and QA Managers New frontier in big log data analysis and application intelligence Business white paper May 2015 During software testing cycles,

More information

W H I T E P A P E R. Deriving Intelligence from Large Data Using Hadoop and Applying Analytics. Abstract

W H I T E P A P E R. Deriving Intelligence from Large Data Using Hadoop and Applying Analytics. Abstract W H I T E P A P E R Deriving Intelligence from Large Data Using Hadoop and Applying Analytics Abstract This white paper is focused on discussing the challenges facing large scale data processing and the

More information

How to Enhance Traditional BI Architecture to Leverage Big Data

How to Enhance Traditional BI Architecture to Leverage Big Data B I G D ATA How to Enhance Traditional BI Architecture to Leverage Big Data Contents Executive Summary... 1 Traditional BI - DataStack 2.0 Architecture... 2 Benefits of Traditional BI - DataStack 2.0...

More information

BIG DATA & ANALYTICS. Transforming the business and driving revenue through big data and analytics

BIG DATA & ANALYTICS. Transforming the business and driving revenue through big data and analytics BIG DATA & ANALYTICS Transforming the business and driving revenue through big data and analytics Collection, storage and extraction of business value from data generated from a variety of sources are

More information

InfiniteGraph: The Distributed Graph Database

InfiniteGraph: The Distributed Graph Database A Performance and Distributed Performance Benchmark of InfiniteGraph and a Leading Open Source Graph Database Using Synthetic Data Objectivity, Inc. 640 West California Ave. Suite 240 Sunnyvale, CA 94086

More information

Big Data on Microsoft Platform

Big Data on Microsoft Platform Big Data on Microsoft Platform Prepared by GJ Srinivas Corporate TEG - Microsoft Page 1 Contents 1. What is Big Data?...3 2. Characteristics of Big Data...3 3. Enter Hadoop...3 4. Microsoft Big Data Solutions...4

More information

Visualization methods for patent data

Visualization methods for patent data Visualization methods for patent data Treparel 2013 Dr. Anton Heijs (CTO & Founder) Delft, The Netherlands Introduction Treparel can provide advanced visualizations for patent data. This document describes

More information

Big Data and Analytics: Challenges and Opportunities

Big Data and Analytics: Challenges and Opportunities Big Data and Analytics: Challenges and Opportunities Dr. Amin Beheshti Lecturer and Senior Research Associate University of New South Wales, Australia (Service Oriented Computing Group, CSE) Talk: Sharif

More information

NEXT GENERATION DECISION SCIENCE FOR the INSURANCE INDUSTRY

NEXT GENERATION DECISION SCIENCE FOR the INSURANCE INDUSTRY INFINILYTICS, INC. NEXT GENERATION DECISION SCIENCE FOR the INSURANCE INDUSTRY Whitepaper series: Big Data, Data Science, Fact-based Decisions, Machine Learning and Advanced Analytics: An Introduction

More information

Improve Cooperation in R&D. Catalyze Drug Repositioning. Optimize Clinical Trials. Respect Information Governance and Security

Improve Cooperation in R&D. Catalyze Drug Repositioning. Optimize Clinical Trials. Respect Information Governance and Security SINEQUA FOR LIFE SCIENCES DRIVE INNOVATION. ACCELERATE RESEARCH. SHORTEN TIME-TO-MARKET. 6 Ways to Leverage Big Data Search & Content Analytics for a Pharmaceutical Company Improve Cooperation in R&D Catalyze

More information

Executive Summary WHO SHOULD READ THIS PAPER?

Executive Summary WHO SHOULD READ THIS PAPER? The Business Value of Business Intelligence in SharePoint 2010 Executive Summary SharePoint 2010 is The Business Collaboration Platform for the Enterprise & the Web that enables you to connect & empower

More information

BEYOND BI: Big Data Analytic Use Cases

BEYOND BI: Big Data Analytic Use Cases BEYOND BI: Big Data Analytic Use Cases Big Data Analytics Use Cases This white paper discusses the types and characteristics of big data analytics use cases, how they differ from traditional business intelligence

More information

www.pwc.com Implementation of Big Data and Analytics Projects with Big Data Discovery and BICS March 2015

www.pwc.com Implementation of Big Data and Analytics Projects with Big Data Discovery and BICS March 2015 www.pwc.com Implementation of Big Data and Analytics Projects with Big Data Discovery and BICS Agenda Big Data Discovery Oracle Business Intelligence Cloud Services (BICS) Use Cases How to start and our

More information

5 Big Data Use Cases to Understand Your Customer Journey CUSTOMER ANALYTICS EBOOK

5 Big Data Use Cases to Understand Your Customer Journey CUSTOMER ANALYTICS EBOOK 5 Big Data Use Cases to Understand Your Customer Journey CUSTOMER ANALYTICS EBOOK CUSTOMER JOURNEY Technology is radically transforming the customer journey. Today s customers are more empowered and connected

More information

Trends and Research Opportunities in Spatial Big Data Analytics and Cloud Computing NCSU GeoSpatial Forum

Trends and Research Opportunities in Spatial Big Data Analytics and Cloud Computing NCSU GeoSpatial Forum Trends and Research Opportunities in Spatial Big Data Analytics and Cloud Computing NCSU GeoSpatial Forum Siva Ravada Senior Director of Development Oracle Spatial and MapViewer 2 Evolving Technology Platforms

More information

Big Data and Hadoop for the Executive A Reference Guide

Big Data and Hadoop for the Executive A Reference Guide Big Data and Hadoop for the Executive A Reference Guide Overview The amount of information being collected by companies today is incredible. Wal- Mart has 460 terabytes of data, which, according to the

More information

BIG DATA THE NEW OPPORTUNITY

BIG DATA THE NEW OPPORTUNITY Feature Biswajit Mohapatra is an IBM Certified Consultant and a global integrated delivery leader for IBM s AMS business application modernization (BAM) practice. He is IBM India s competency head for

More information

A financial software company

A financial software company A financial software company Projecting USD10 million revenue lift with the IBM Netezza data warehouse appliance Overview The need A financial software company sought to analyze customer engagements to

More information

COMP9321 Web Application Engineering

COMP9321 Web Application Engineering COMP9321 Web Application Engineering Semester 2, 2015 Dr. Amin Beheshti Service Oriented Computing Group, CSE, UNSW Australia Week 11 (Part II) http://webapps.cse.unsw.edu.au/webcms2/course/index.php?cid=2411

More information

Oracle Big Data Discovery The Visual Face of Hadoop

Oracle Big Data Discovery The Visual Face of Hadoop Disclaimer: This document is for informational purposes. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development,

More information

Big Data and Natural Language: Extracting Insight From Text

Big Data and Natural Language: Extracting Insight From Text An Oracle White Paper October 2012 Big Data and Natural Language: Extracting Insight From Text Table of Contents Executive Overview... 3 Introduction... 3 Oracle Big Data Appliance... 4 Synthesys... 5

More information

Structured Content: the Key to Agile. Web Experience Management. Introduction

Structured Content: the Key to Agile. Web Experience Management. Introduction Structured Content: the Key to Agile CONTENTS Introduction....................... 1 Structured Content Defined...2 Structured Content is Intelligent...2 Structured Content and Customer Experience...3 Structured

More information

Tapping the benefits of business analytics and optimization

Tapping the benefits of business analytics and optimization IBM Sales and Distribution Chemicals and Petroleum White Paper Tapping the benefits of business analytics and optimization A rich source of intelligence for the chemicals and petroleum industries 2 Tapping

More information

IBM Enterprise Content Management Product Strategy

IBM Enterprise Content Management Product Strategy White Paper July 2007 IBM Information Management software IBM Enterprise Content Management Product Strategy 2 IBM Innovation Enterprise Content Management (ECM) IBM Investment in ECM IBM ECM Vision Contents

More information

MarkLogic Enterprise Data Layer

MarkLogic Enterprise Data Layer MarkLogic Enterprise Data Layer MarkLogic Enterprise Data Layer MarkLogic Enterprise Data Layer September 2011 September 2011 September 2011 Table of Contents Executive Summary... 3 An Enterprise Data

More information

Advanced Big Data Analytics with R and Hadoop

Advanced Big Data Analytics with R and Hadoop REVOLUTION ANALYTICS WHITE PAPER Advanced Big Data Analytics with R and Hadoop 'Big Data' Analytics as a Competitive Advantage Big Analytics delivers competitive advantage in two ways compared to the traditional

More information

This Symposium brought to you by www.ttcus.com

This Symposium brought to you by www.ttcus.com This Symposium brought to you by www.ttcus.com Linkedin/Group: Technology Training Corporation @Techtrain Technology Training Corporation www.ttcus.com Big Data Analytics as a Service (BDAaaS) Big Data

More information

The 2-Tier Business Intelligence Imperative

The 2-Tier Business Intelligence Imperative Business Intelligence Imperative Enterprise-grade analytics that keeps pace with today s business speed Table of Contents 3 4 5 7 9 Overview The Historical Conundrum The Need For A New Class Of Platform

More information

SAP Solution Brief SAP HANA. Transform Your Future with Better Business Insight Using Predictive Analytics

SAP Solution Brief SAP HANA. Transform Your Future with Better Business Insight Using Predictive Analytics SAP Brief SAP HANA Objectives Transform Your Future with Better Business Insight Using Predictive Analytics Dealing with the new reality Dealing with the new reality Organizations like yours can identify

More information

Integrated Social and Enterprise Data = Enhanced Analytics

Integrated Social and Enterprise Data = Enhanced Analytics ORACLE WHITE PAPER, DECEMBER 2013 THE VALUE OF SOCIAL DATA Integrated Social and Enterprise Data = Enhanced Analytics #SocData CONTENTS Executive Summary 3 The Value of Enterprise-Specific Social Data

More information

Hurwitz ValuePoint: Predixion

Hurwitz ValuePoint: Predixion Predixion VICTORY INDEX CHALLENGER Marcia Kaufman COO and Principal Analyst Daniel Kirsch Principal Analyst The Hurwitz Victory Index Report Predixion is one of 10 advanced analytics vendors included in

More information

The Future of Business Analytics is Now! 2013 IBM Corporation

The Future of Business Analytics is Now! 2013 IBM Corporation The Future of Business Analytics is Now! 1 The pressures on organizations are at a point where analytics has evolved from a business initiative to a BUSINESS IMPERATIVE More organization are using analytics

More information

From Data to Foresight:

From Data to Foresight: Laura Haas, IBM Fellow IBM Research - Almaden From Data to Foresight: Leveraging Data and Analytics for Materials Research 1 2011 IBM Corporation The road from data to foresight is long? Consumer Reports

More information

APPROACHABLE ANALYTICS MAKING SENSE OF DATA

APPROACHABLE ANALYTICS MAKING SENSE OF DATA APPROACHABLE ANALYTICS MAKING SENSE OF DATA AGENDA SAS DELIVERS PROVEN SOLUTIONS THAT DRIVE INNOVATION AND IMPROVE PERFORMANCE. About SAS SAS Business Analytics Framework Approachable Analytics SAS for

More information

Apache Hadoop: The Big Data Refinery

Apache Hadoop: The Big Data Refinery Architecting the Future of Big Data Whitepaper Apache Hadoop: The Big Data Refinery Introduction Big data has become an extremely popular term, due to the well-documented explosion in the amount of data

More information

How In-Memory Data Grids Can Analyze Fast-Changing Data in Real Time

How In-Memory Data Grids Can Analyze Fast-Changing Data in Real Time SCALEOUT SOFTWARE How In-Memory Data Grids Can Analyze Fast-Changing Data in Real Time by Dr. William Bain and Dr. Mikhail Sobolev, ScaleOut Software, Inc. 2012 ScaleOut Software, Inc. 12/27/2012 T wenty-first

More information

Big Data Analytics Roadmap Energy Industry

Big Data Analytics Roadmap Energy Industry Douglas Moore, Principal Consultant, Architect June 2013 Big Data Analytics Energy Industry Agenda Why Big Data in Energy? Imagine Overview - Use Cases - Readiness Analysis - Architecture - Development

More information

Using Tableau Software with Hortonworks Data Platform

Using Tableau Software with Hortonworks Data Platform Using Tableau Software with Hortonworks Data Platform September 2013 2013 Hortonworks Inc. http:// Modern businesses need to manage vast amounts of data, and in many cases they have accumulated this data

More information

CORPORATE OVERVIEW. Big Data. Shared. Simply. Securely.

CORPORATE OVERVIEW. Big Data. Shared. Simply. Securely. CORPORATE OVERVIEW Big Data. Shared. Simply. Securely. INTRODUCING PHEMI SYSTEMS PHEMI unlocks the power of your data with out-of-the-box privacy, sharing, and governance PHEMI Systems brings advanced

More information

Data Science Certificate Program

Data Science Certificate Program Information Technologies Programs Data Science Certificate Program Accelerate Your Career extension.uci.edu/datascience Offered in partnership with University of California, Irvine Extension s professional

More information

The Next Wave of Data Management. Is Big Data The New Normal?

The Next Wave of Data Management. Is Big Data The New Normal? The Next Wave of Data Management Is Big Data The New Normal? Table of Contents Introduction 3 Separating Reality and Hype 3 Why Are Firms Making IT Investments In Big Data? 4 Trends In Data Management

More information

can you effectively plan for the migration and management of systems and applications on Vblock Platforms?

can you effectively plan for the migration and management of systems and applications on Vblock Platforms? SOLUTION BRIEF CA Capacity Management and Reporting Suite for Vblock Platforms can you effectively plan for the migration and management of systems and applications on Vblock Platforms? agility made possible

More information

THE ANALYTICS HUB LEVERAGING A SHARED SERVICES MODEL TO UNLOCK BIG DATA. Thomas Roland Managing Director. David Roggen Director CONTENTS

THE ANALYTICS HUB LEVERAGING A SHARED SERVICES MODEL TO UNLOCK BIG DATA. Thomas Roland Managing Director. David Roggen Director CONTENTS THE ANALYTICS HUB LEVERAGING A SHARED SERVICES MODEL TO UNLOCK BIG DATA David Roggen Director Thomas Roland Managing Director CONTENTS Shared Services Today 2 What Is an Analytics Hub? 3 Analytics Hub

More information

Klarna Tech Talk: Mind the Data! Jeff Pollock InfoSphere Information Integration & Governance

Klarna Tech Talk: Mind the Data! Jeff Pollock InfoSphere Information Integration & Governance Klarna Tech Talk: Mind the Data! Jeff Pollock InfoSphere Information Integration & Governance IBM s statements regarding its plans, directions, and intent are subject to change or withdrawal without notice

More information

White Paper. How Streaming Data Analytics Enables Real-Time Decisions

White Paper. How Streaming Data Analytics Enables Real-Time Decisions White Paper How Streaming Data Analytics Enables Real-Time Decisions Contents Introduction... 1 What Is Streaming Analytics?... 1 How Does SAS Event Stream Processing Work?... 2 Overview...2 Event Stream

More information

Buyer s Guide to Big Data Integration

Buyer s Guide to Big Data Integration SEPTEMBER 2013 Buyer s Guide to Big Data Integration Sponsored by Contents Introduction 1 Challenges of Big Data Integration: New and Old 1 What You Need for Big Data Integration 3 Preferred Technology

More information

W H I T E P A P E R E d u c a t i o n a t t h e C r o s s r o a d s o f B i g D a t a a n d C l o u d

W H I T E P A P E R E d u c a t i o n a t t h e C r o s s r o a d s o f B i g D a t a a n d C l o u d Global Headquarters: 5 Speen Street Framingham, MA 01701 USA P.508.872.8200 F.508.935.4015 www.idc.com W H I T E P A P E R E d u c a t i o n a t t h e C r o s s r o a d s o f B i g D a t a a n d C l o

More information

The Purview Solution Integration With Splunk

The Purview Solution Integration With Splunk The Purview Solution Integration With Splunk Integrating Application Management and Business Analytics With Other IT Management Systems A SOLUTION WHITE PAPER WHITE PAPER Introduction Purview Integration

More information

What do Big Data & HAVEn mean? Robert Lejnert HP Autonomy

What do Big Data & HAVEn mean? Robert Lejnert HP Autonomy What do Big Data & HAVEn mean? Robert Lejnert HP Autonomy Much higher Volumes. Processed with more Velocity. With much more Variety. Is Big Data so big? Big Data Smart Data Project HAVEn: Adaptive Intelligence

More information

DATA EXPERTS MINE ANALYZE VISUALIZE. We accelerate research and transform data to help you create actionable insights

DATA EXPERTS MINE ANALYZE VISUALIZE. We accelerate research and transform data to help you create actionable insights DATA EXPERTS We accelerate research and transform data to help you create actionable insights WE MINE WE ANALYZE WE VISUALIZE Domains Data Mining Mining longitudinal and linked datasets from web and other

More information

Pentaho High-Performance Big Data Reference Configurations using Cisco Unified Computing System

Pentaho High-Performance Big Data Reference Configurations using Cisco Unified Computing System Pentaho High-Performance Big Data Reference Configurations using Cisco Unified Computing System By Jake Cornelius Senior Vice President of Products Pentaho June 1, 2012 Pentaho Delivers High-Performance

More information

DATAMEER WHITE PAPER. Beyond BI. Big Data Analytic Use Cases

DATAMEER WHITE PAPER. Beyond BI. Big Data Analytic Use Cases DATAMEER WHITE PAPER Beyond BI Big Data Analytic Use Cases This white paper discusses the types and characteristics of big data analytics use cases, how they differ from traditional business intelligence

More information

Cross-Domain Service Management vs. Traditional IT Service Management for Service Providers

Cross-Domain Service Management vs. Traditional IT Service Management for Service Providers Position Paper Cross-Domain vs. Traditional IT for Providers Joseph Bondi Copyright-2013 All rights reserved. Ni², Ni² logo, other vendors or their logos are trademarks of Network Infrastructure Inventory

More information

Big Data Analytics Platform @ Nokia

Big Data Analytics Platform @ Nokia Big Data Analytics Platform @ Nokia 1 Selecting the Right Tool for the Right Workload Yekesa Kosuru Nokia Location & Commerce Strata + Hadoop World NY - Oct 25, 2012 Agenda Big Data Analytics Platform

More information

How Financial Services Firms Can Benefit From Streaming Analytics

How Financial Services Firms Can Benefit From Streaming Analytics How Financial Services Firms Can Benefit From Streaming Analytics > 2 VITRIA TECHNOLOGY, INC. > How Financial Services Firms Can Benefit From Streaming Analytics Streaming Analytics: Why It s Important

More information

Business Analytics and the Nexus of Information

Business Analytics and the Nexus of Information Business Analytics and the Nexus of Information 2 The Impact of the Nexus of Forces 4 From the Gartner Files: Information and the Nexus of Forces: Delivering and Analyzing Data 6 About IBM Business Analytics

More information

Big Data for Investment Research Management

Big Data for Investment Research Management IDT Partners www.idtpartners.com Big Data for Investment Research Management Discover how IDT Partners helps Financial Services, Market Research, and Investment Management firms turn big data into actionable

More information

perspective Progressive Organization

perspective Progressive Organization perspective Progressive Organization Progressive organization Owing to rapid changes in today s digital world, the data landscape is constantly shifting and creating new complexities. Today, organizations

More information

Demystifying Big Data Government Agencies & The Big Data Phenomenon

Demystifying Big Data Government Agencies & The Big Data Phenomenon Demystifying Big Data Government Agencies & The Big Data Phenomenon Today s Discussion If you only remember four things 1 Intensifying business challenges coupled with an explosion in data have pushed

More information

IBM Content Analytics with Enterprise Search, Version 3.0

IBM Content Analytics with Enterprise Search, Version 3.0 IBM Content Analytics with Enterprise Search, Version 3.0 Highlights Enables greater accuracy and control over information with sophisticated natural language processing capabilities to deliver the right

More information

Knowledge Discovery from patents using KMX Text Analytics

Knowledge Discovery from patents using KMX Text Analytics Knowledge Discovery from patents using KMX Text Analytics Dr. Anton Heijs anton.heijs@treparel.com Treparel Abstract In this white paper we discuss how the KMX technology of Treparel can help searchers

More information

Understanding Your Customer Journey by Extending Adobe Analytics with Big Data

Understanding Your Customer Journey by Extending Adobe Analytics with Big Data SOLUTION BRIEF Understanding Your Customer Journey by Extending Adobe Analytics with Big Data Business Challenge Today s digital marketing teams are overwhelmed by the volume and variety of customer interaction

More information

Auto Days 2011 Predictive Analytics in Auto Finance

Auto Days 2011 Predictive Analytics in Auto Finance Auto Days 2011 Predictive Analytics in Auto Finance Vick Panwar SAS Risk Practice Copyright 2010 SAS Institute Inc. All rights reserved. Agenda Introduction Changing Risk Landscape - Key Drivers and Challenges

More information

Mastering Big Data. Steve Hoskin, VP and Chief Architect INFORMATICA MDM. October 2015

Mastering Big Data. Steve Hoskin, VP and Chief Architect INFORMATICA MDM. October 2015 Mastering Big Data Steve Hoskin, VP and Chief Architect INFORMATICA MDM October 2015 Agenda About Big Data MDM and Big Data The Importance of Relationships Big Data Use Cases About Big Data Big Data is

More information

BIG DATA: FROM HYPE TO REALITY. Leandro Ruiz Presales Partner for C&LA Teradata

BIG DATA: FROM HYPE TO REALITY. Leandro Ruiz Presales Partner for C&LA Teradata BIG DATA: FROM HYPE TO REALITY Leandro Ruiz Presales Partner for C&LA Teradata Evolution in The Use of Information Action s ACTIVATING MAKE it happen! Insights OPERATIONALIZING WHAT IS happening now? PREDICTING

More information

Simple. Extensible. Open.

Simple. Extensible. Open. White Paper Simple. Extensible. Open. Unleash the Value of Data with EMC ViPR Global Data Services Abstract The following paper opens with the evolution of enterprise storage infrastructure in the era

More information