ICT Perspectives on Big Data: Well Sorted Materials

Size: px
Start display at page:

Download "ICT Perspectives on Big Data: Well Sorted Materials"

Transcription

1 ICT Perspectives on Big Data: Well Sorted Materials 3 March 2015 Contents Introduction 1 Dendrogram 2 Tree Map 3 Heat Map 4 Raw Group Data 5 For an online, interactive version of the visualisations in this document, go here:

2 Introduction Dear participant, Thank you for taking part in submitting and sorting your ideas. This document contains several visualisations of your ideas, grouped by the average of your online sorts. They are: Dendrogram - This tree shows each submitted idea and its similarity to the others. The lower two ideas 'join' the more people grouped those two ideas together. For example, if two ideas join at the bottom, every person grouped those two together. Tree Map - This visualisation presents an 'average' grouping. It is calculated by 'cutting' the Dendrogram at the dashed line so that any items which join lower than that line are placed in the same group. In addition, rectangles which share a side of the same length are more similar to each other than their peers. Heat Map - This visualisation shows a similarity matrix where each idea is given a colour at the intersection with another idea, showing how similar the two are. This is useful to see how well formed a group is. The more red there is in a group (shown by the black lines), the more similar the ideas inside it were judged to be. Raw Group Data - This table shows every submitted idea and its longer description. They are shown in the same order as the Dendrogram (so similar ideas are close to each other) and split into the coloured groups used in the Tree Map. In addition, each idea has been given a unique number so they are easier to find. Page 1

3 Dendrogram Page 2

4 Tree Map Page 3

5 Heat Map Page 4

6 Raw Group Data Red 1 Data privacy issue Big data offers a great opportunity to tackle societal challenges. But data privacy is one of barriers to maximize such an opportunity. One key research question: novel data processing approaches to enable efficient analysis without violating privacy 2 Security The scale and impact of a big data breach is likely to dwarf scale and impact of data breaches in more traditional systems. The distributed nature of big data systems makes them more vulnerable. Can existing approaches to big data security scale? Blue 3 How Big Data better improve manufacturing process? Big Data will be more prominent with more and more IoT devices getting introduced into manufacturing. Advanced analytics helps decode complex manufacturing processes to improve yield through data prioritization, root causes analysis and process modelling. 4 Building Performance Determinants Research shows higher than expected consumption of energy and resources to create and maintain healthy & comfortable indoor environments. Disparate data sources need to be linked and analysed to identify key determinants for excessive resource consumption Page 5

7 Green 5 Managing Volume and Velocity of data Complex sensor networks and the like will output actionable data at volumes and velocities that we have not worked with yet. There is huge potential from machine learning and automated control systems taking informed next best actions. 6 Big Data is growing faster than Moore's Law! The volume and velocity of data generation is increasing faster than Moore's law (and other laws for scaling performance of computing systems). How do we engineer compute platforms and software infrastructure to analyse big data workloads? 7 Scalable and Expressive Platforms for Big Data We need new types of distributed software platforms for Big Data processing that are scalable, expressive and performant, but remain intuitive to use. They need to offer a real-time unified view of large sets of streaming and historic data. 8 Capacity Planning for converging digital Networks - dealing with data volumes from multi-sensorscapacity planning from the network- cloud computingsufficient statistics- energy-aware data centres and proxy-clouds 9 Research Data Management (RDM) at scale The costs of storing, curating, sharing and publishing research data at scale demands new techniques for sharing data without copying it and applying analytics without direct access to the data. New levels of collaboration and resource sharing are needed. 10 Reliable software We need software to analyse large quantities of data but the functionality is quite different from 'normal' software. This creates a challenge - how can we have confidence in the software used? Orange 11 Data Wrangling Data Wrangling is the process of collecting and preparing data before analysis can take place. It emerged that data scientists spend 50%-80% of their time for data wrangling (DW). Current approaches to DW are rather ad hoc. A methodology is sorely needed. 12 Determining what data are valuable Data size and rate of creation are predicted to rise rapidly. Much of the data will be of low quality, have limited provenance and be subject to errors. Determining rapidly and efficiently subsets of the data have value and what to keep will be essential. 13 Integration of heterogeneous data Data is now taking many complex forms from time series, images, video, to massive scale sequences of amino acids, clinical images and so forth. How these are to be integrated in a coherent manner that improves SNR is a major open issue. 14 Merging Heterogeneous Big Data Sources The key research challenge in the area of big data in my opinion is the problem of merging heterogeneous Page 6

8 sources of information, quantifying at the same time the quality of the different data sources and the uncertainty associated to them. 15 Heterogeneous data - mixed data types Objects are described by multiple data types, e.g. a patient may be described by a mixture of structured data, time series representing results of a particular text over time, images, text reports and more. Analysis of such complex data is underesearched 16 Merging Disparate Data Sets (Data Fusion) Developing computational tools to merge and contrast, at large scale, related but distinct data sources. Can we find patterns that are (a) shared or (b) distinct? Can we quantify which data sources provide the most insight and which are contradictory? 17 Big Data Integration Value is increasingly going to be found in latent connections amongst datasets that originate from independent sources. Diversity is both an opportunity and a challenge for predictive analytics techniques that aim to distill knowledge from data. 18 Multi-source information fusion for big data Application of AI techniques and development of data fusion methods for the extraction of salient data for decision support in cluttered, congested and conflicted big data environments. 19 Tools for addressing heterogeneity of big data Scale is not our biggest challenge - it is heterogeneity in form (structured/unstructured data, numbers/text) and format (numerous haphazardly-organised sources). Tools are urgently needed - generalised but powerful enough to be useful analytically. 20 Reference Data It is challenging to create a single database with all data from different intrusion systems. The data in different formats needs to be combined to make an informed decision by the security analyst. 21 Modelling ill-structured data Ill-structured data include unstructured crowd source data (e.g. Social media) and irregular samples (e.g. cross section surveys) at different spatial and/or temporal density/frequency. Most existing analytical tools are suitable for well-structured data. 22 Flexible, scalable and semanticbased approaches Techniques for analysing, modeling and reasoning over heterogeneous and dynamic datasets whose structure may potentially change with time from multiple sources will be needed. This calls for highly flexible, scalable and semantic-based approaches. Page 7

9 Purple 23 Algorithmic Breakthroughs in Big Data Analytics While MapReduce etc. simplifies parallelism, most big data computations are just sums and counts. New directions promising breakthroughs: parallel optimization and approximation algorithms; streaming & summary techniques; efficient graph/matrix algorithms 24 New algorithmic frameworks Shifting attention in algorithm design and analysis towards (1) relational rather than numerical data (2) data streams rather than sets (3) approximate solutions to imperfectly posed problems (4) randomised methods for partial data 25 Robust and Theory Driven Statistical Modeling Big Data Analytics can produce statistics and features of interest when studying socio-technical phenomena,but what is the theory behind the analytics? Can we incorporate theory from different disciplines,and are we sure the methods used are appropriate? 26 New methods for exploratory data analysis We need to understand properties of microscopic events in large information spaces where information meets coincidentally rather than causally determined. This becomes important for exploratory data analysis to generate hypothesis in big data scenarios. 27 How can we accurately quantify uncertainty? There are established methods for quantifying uncertainty, but these will not be appropriate for big data. For example, some methods involve modelling assumptions. For big data, model errors will lead to substantial underestimation of uncertainty. 28 Combining Big Data with Prior Knowledge Often in machine learning, we want to leverage prior knowledge about the data to improve the analysis. Current approaches include feature design and probabilistic modelling. But it can be hard to express vague knowledge in the language of these formalisms 29 Impedance Mismatch: Advanced Analyses v BD Systems Math/Statistical principles vs algorithms and engineering. Need to know how both sides work and how one is mapped to the other. Guaranteed effects in analyses may be lost by underlying infrastructure. Scalable BIG DATA analytics depends on above. Page 8

10 Yellow 30 Human-Centred Analytics The black box nature of mining algorithms, need for parameter tuning, difficulty of coping with outliers/bad data requires experts to mine data. We need a humancentred approach using effective visualisation and visual analytics. 31 Big data HCI How can we visualise, summarise, or otherwise expose what is contained in big datasets (or what we learn from analysing them), such that that people can understand and exploit this knowledge? 32 User oriented visualisation and configuration How do we develop methodologies and technology that allow users to explore and configure large complex data spaces and their visualisations? i.e. can we take the data analytics expert out of the loop? 33 Interacting with data Humans are often left out when talking about making sense from data. Users will need to interact with complex data in cognitively demanding situations. Understanding the needs of users in these situations is key to fully exploiting the new data rich lands 34 Remove barriers in the use and understanding Massive volumes of data are generated every second from corporate and public sources which could potentially be relevant to decision-makers. How can we build interfaces to support human decision-making? 35 The Emotions of Data What is humans' emotional connection with data and how does that affect data-based decision making processes? Data-based decision making implicitly assumes rational actors. But humans are not rational. How can analytics support emotional decision making? 36 Delivery of Big Data Analytics Impacts The challenges for big data analytics are not only from the pure analytic parts, say, improving learning algorithms and etc. The key issue is how ordinary people would benefit from those analytic results given the huge knowledge gap in between. 37 Targeted decision support The key challenge is to understand what could be extracted from big data that would allow us to take more informed decisions for particular uses. This would involve collaboration between data scientists and experts in the field of the origin of the data. 38 Scale of the results of big data analysis We do not only need methods that are scaled to cope with big data, we also need to consider how to scale down the results to make sure that we (humans) are able to understand and use all of its potential. 39 Visual Big Data: Images and Video The evolution of the internet from a text to a visual medium is well recognised and especially evident in social media (CicsoTF: by % of internet will be video). How do we make sense of this deluge of visual data often with little/no metadata? Powered by TCPDF ( Page 9

EPSRC Cross-SAT Big Data Workshop: Well Sorted Materials

EPSRC Cross-SAT Big Data Workshop: Well Sorted Materials EPSRC Cross-SAT Big Data Workshop: Well Sorted Materials 5th August 2015 Contents Introduction 1 Dendrogram 2 Tree Map 3 Heat Map 4 Raw Group Data 5 For an online, interactive version of the visualisations

More information

ICT Perspectives on Big Data Analytics Workshop. Report on Outputs and Next Steps

ICT Perspectives on Big Data Analytics Workshop. Report on Outputs and Next Steps ICT Perspectives on Big Data Analytics Workshop 3 March 2015 Hilton Birmingham Metropole, NEC Report on Outputs and Next Steps Dr Miriam Dowle EPSRC ICT Theme Portfolio Manager October 2015 Table of Contents

More information

Big Data: Rethinking Text Visualization

Big Data: Rethinking Text Visualization Big Data: Rethinking Text Visualization Dr. Anton Heijs anton.heijs@treparel.com Treparel April 8, 2013 Abstract In this white paper we discuss text visualization approaches and how these are important

More information

Detecting Anomalous Behavior with the Business Data Lake. Reference Architecture and Enterprise Approaches.

Detecting Anomalous Behavior with the Business Data Lake. Reference Architecture and Enterprise Approaches. Detecting Anomalous Behavior with the Business Data Lake Reference Architecture and Enterprise Approaches. 2 Detecting Anomalous Behavior with the Business Data Lake Pivotal the way we see it Reference

More information

ATA DRIVEN GLOBAL VISION CLOUD PLATFORM STRATEG N POWERFUL RELEVANT PERFORMANCE SOLUTION CLO IRTUAL BIG DATA SOLUTION ROI FLEXIBLE DATA DRIVEN V

ATA DRIVEN GLOBAL VISION CLOUD PLATFORM STRATEG N POWERFUL RELEVANT PERFORMANCE SOLUTION CLO IRTUAL BIG DATA SOLUTION ROI FLEXIBLE DATA DRIVEN V ATA DRIVEN GLOBAL VISION CLOUD PLATFORM STRATEG N POWERFUL RELEVANT PERFORMANCE SOLUTION CLO IRTUAL BIG DATA SOLUTION ROI FLEXIBLE DATA DRIVEN V WHITE PAPER Create the Data Center of the Future Accelerate

More information

Information Visualization WS 2013/14 11 Visual Analytics

Information Visualization WS 2013/14 11 Visual Analytics 1 11.1 Definitions and Motivation Lot of research and papers in this emerging field: Visual Analytics: Scope and Challenges of Keim et al. Illuminating the path of Thomas and Cook 2 11.1 Definitions and

More information

W H I T E P A P E R. Deriving Intelligence from Large Data Using Hadoop and Applying Analytics. Abstract

W H I T E P A P E R. Deriving Intelligence from Large Data Using Hadoop and Applying Analytics. Abstract W H I T E P A P E R Deriving Intelligence from Large Data Using Hadoop and Applying Analytics Abstract This white paper is focused on discussing the challenges facing large scale data processing and the

More information

Decision Support Optimization through Predictive Analytics - Leuven Statistical Day 2010

Decision Support Optimization through Predictive Analytics - Leuven Statistical Day 2010 Decision Support Optimization through Predictive Analytics - Leuven Statistical Day 2010 Ernst van Waning Senior Sales Engineer May 28, 2010 Agenda SPSS, an IBM Company SPSS Statistics User-driven product

More information

Gerard Mc Nulty Systems Optimisation Ltd gmcnulty@iol.ie/0876697867 BA.,B.A.I.,C.Eng.,F.I.E.I

Gerard Mc Nulty Systems Optimisation Ltd gmcnulty@iol.ie/0876697867 BA.,B.A.I.,C.Eng.,F.I.E.I Gerard Mc Nulty Systems Optimisation Ltd gmcnulty@iol.ie/0876697867 BA.,B.A.I.,C.Eng.,F.I.E.I Data is Important because it: Helps in Corporate Aims Basis of Business Decisions Engineering Decisions Energy

More information

Analyzing Big Data: The Path to Competitive Advantage

Analyzing Big Data: The Path to Competitive Advantage White Paper Analyzing Big Data: The Path to Competitive Advantage by Marcia Kaplan Contents Introduction....2 How Big is Big Data?................................................................................

More information

Data Refinery with Big Data Aspects

Data Refinery with Big Data Aspects International Journal of Information and Computation Technology. ISSN 0974-2239 Volume 3, Number 7 (2013), pp. 655-662 International Research Publications House http://www. irphouse.com /ijict.htm Data

More information

Big Data: Overview and Roadmap. 2015 eglobaltech. All rights reserved.

Big Data: Overview and Roadmap. 2015 eglobaltech. All rights reserved. Big Data: Overview and Roadmap 2015 eglobaltech. All rights reserved. What is Big Data? Large volumes of complex and variable data that require advanced techniques and technologies to enable capture, storage,

More information

Navigating Big Data business analytics

Navigating Big Data business analytics mwd a d v i s o r s Navigating Big Data business analytics Helena Schwenk A special report prepared for Actuate May 2013 This report is the third in a series and focuses principally on explaining what

More information

The Scientific Data Mining Process

The Scientific Data Mining Process Chapter 4 The Scientific Data Mining Process When I use a word, Humpty Dumpty said, in rather a scornful tone, it means just what I choose it to mean neither more nor less. Lewis Carroll [87, p. 214] In

More information

Trends and Research Opportunities in Spatial Big Data Analytics and Cloud Computing NCSU GeoSpatial Forum

Trends and Research Opportunities in Spatial Big Data Analytics and Cloud Computing NCSU GeoSpatial Forum Trends and Research Opportunities in Spatial Big Data Analytics and Cloud Computing NCSU GeoSpatial Forum Siva Ravada Senior Director of Development Oracle Spatial and MapViewer 2 Evolving Technology Platforms

More information

Trustworthiness of Big Data

Trustworthiness of Big Data Trustworthiness of Big Data International Journal of Computer Applications (0975 8887) Akhil Mittal Technical Test Lead Infosys Limited ABSTRACT Big data refers to large datasets that are challenging to

More information

Visualization methods for patent data

Visualization methods for patent data Visualization methods for patent data Treparel 2013 Dr. Anton Heijs (CTO & Founder) Delft, The Netherlands Introduction Treparel can provide advanced visualizations for patent data. This document describes

More information

Why big data? Lessons from a Decade+ Experiment in Big Data

Why big data? Lessons from a Decade+ Experiment in Big Data Why big data? Lessons from a Decade+ Experiment in Big Data David Belanger PhD Senior Research Fellow Stevens Institute of Technology dbelange@stevens.edu 1 What Does Big Look Like? 7 Image Source Page:

More information

Concept and Project Objectives

Concept and Project Objectives 3.1 Publishable summary Concept and Project Objectives Proactive and dynamic QoS management, network intrusion detection and early detection of network congestion problems among other applications in the

More information

Beyond the Single View with IBM InfoSphere

Beyond the Single View with IBM InfoSphere Ian Bowring MDM & Information Integration Sales Leader, NE Europe Beyond the Single View with IBM InfoSphere We are at a pivotal point with our information intensive projects 10-40% of each initiative

More information

3rd International Symposium on Big Data and Cloud Computing Challenges (ISBCC-2016) March 10-11, 2016 VIT University, Chennai, India

3rd International Symposium on Big Data and Cloud Computing Challenges (ISBCC-2016) March 10-11, 2016 VIT University, Chennai, India 3rd International Symposium on Big Data and Cloud Computing Challenges (ISBCC-2016) March 10-11, 2016 VIT University, Chennai, India Call for Papers Cloud computing has emerged as a de facto computing

More information

Are You Big Data Ready?

Are You Big Data Ready? ACS 2015 Annual Canberra Conference Are You Big Data Ready? Vladimir Videnovic Business Solutions Director Oracle Big Data and Analytics Introduction Introduction What is Big Data? If you can't explain

More information

The Business Analyst s Guide to Hadoop

The Business Analyst s Guide to Hadoop White Paper The Business Analyst s Guide to Hadoop Get Ready, Get Set, and Go: A Three-Step Guide to Implementing Hadoop-based Analytics By Alteryx and Hortonworks (T)here is considerable evidence that

More information

Cisco Data Preparation

Cisco Data Preparation Data Sheet Cisco Data Preparation Unleash your business analysts to develop the insights that drive better business outcomes, sooner, from all your data. As self-service business intelligence (BI) and

More information

How To Handle Big Data With A Data Scientist

How To Handle Big Data With A Data Scientist III Big Data Technologies Today, new technologies make it possible to realize value from Big Data. Big data technologies can replace highly customized, expensive legacy systems with a standard solution

More information

Real-Time Solutions to Big Data Problems

Real-Time Solutions to Big Data Problems Real-Time Solutions to Big Data Problems IT Infrastructure (analysis / storage) Internet of Everything Big Data Big Data The term Big Data refers to data that overwhelms, IT infrastructure and complicates

More information

IBM Big Data in Government

IBM Big Data in Government IBM Big in Government Turning big data into smarter decisions Deepak Mohapatra Sr. Consultant Government IBM Software Group dmohapatra@us.ibm.com The Big Paradigm Shift 2 Big Creates A Challenge And an

More information

High-Performance Business Analytics: SAS and IBM Netezza Data Warehouse Appliances

High-Performance Business Analytics: SAS and IBM Netezza Data Warehouse Appliances High-Performance Business Analytics: SAS and IBM Netezza Data Warehouse Appliances Highlights IBM Netezza and SAS together provide appliances and analytic software solutions that help organizations improve

More information

Pitfalls and Best Practices in Role Engineering

Pitfalls and Best Practices in Role Engineering Bay31 Role Designer in Practice Series Pitfalls and Best Practices in Role Engineering Abstract: Role Based Access Control (RBAC) and role management are a proven and efficient way to manage user permissions.

More information

BIG DATA & ANALYTICS. Transforming the business and driving revenue through big data and analytics

BIG DATA & ANALYTICS. Transforming the business and driving revenue through big data and analytics BIG DATA & ANALYTICS Transforming the business and driving revenue through big data and analytics Collection, storage and extraction of business value from data generated from a variety of sources are

More information

BIG Data Analytics Move to Competitive Advantage

BIG Data Analytics Move to Competitive Advantage BIG Data Analytics Move to Competitive Advantage where is technology heading today Standardization Open Source Automation Scalability Cloud Computing Mobility Smartphones/ tablets Internet of Things Wireless

More information

Big Data in the context of Preservation and Value Adding

Big Data in the context of Preservation and Value Adding Big Data in the context of Preservation and Value Adding R. Leone, R. Cosac, I. Maggio, D. Iozzino ESRIN 06/11/2013 ESA UNCLASSIFIED Big Data Background ESA/ESRIN organized a 'Big Data from Space' event

More information

Tapping the benefits of business analytics and optimization

Tapping the benefits of business analytics and optimization IBM Sales and Distribution Chemicals and Petroleum White Paper Tapping the benefits of business analytics and optimization A rich source of intelligence for the chemicals and petroleum industries 2 Tapping

More information

Big Data & Analytics for Semiconductor Manufacturing

Big Data & Analytics for Semiconductor Manufacturing Big Data & Analytics for Semiconductor Manufacturing 半 導 体 生 産 におけるビッグデータ 活 用 Ryuichiro Hattori 服 部 隆 一 郎 Intelligent SCM and MFG solution Leader Global CoC (Center of Competence) Electronics team General

More information

Databricks. A Primer

Databricks. A Primer Databricks A Primer Who is Databricks? Databricks was founded by the team behind Apache Spark, the most active open source project in the big data ecosystem today. Our mission at Databricks is to dramatically

More information

Lluis Belanche + Alfredo Vellido. Intelligent Data Analysis and Data Mining

Lluis Belanche + Alfredo Vellido. Intelligent Data Analysis and Data Mining Lluis Belanche + Alfredo Vellido Intelligent Data Analysis and Data Mining a.k.a. Data Mining II Office 319, Omega, BCN EET, office 107, TR 2, Terrassa avellido@lsi.upc.edu skype, gtalk: avellido Tels.:

More information

REFLECTIONS ON THE USE OF BIG DATA FOR STATISTICAL PRODUCTION

REFLECTIONS ON THE USE OF BIG DATA FOR STATISTICAL PRODUCTION REFLECTIONS ON THE USE OF BIG DATA FOR STATISTICAL PRODUCTION Pilar Rey del Castillo May 2013 Introduction The exploitation of the vast amount of data originated from ICT tools and referring to a big variety

More information

The Liaison ALLOY Platform

The Liaison ALLOY Platform PRODUCT OVERVIEW The Liaison ALLOY Platform WELCOME TO YOUR DATA-INSPIRED FUTURE Data is a core enterprise asset. Extracting insights from data is a fundamental business need. As the volume, velocity,

More information

Reference Architecture, Requirements, Gaps, Roles

Reference Architecture, Requirements, Gaps, Roles Reference Architecture, Requirements, Gaps, Roles The contents of this document are an excerpt from the brainstorming document M0014. The purpose is to show how a detailed Big Data Reference Architecture

More information

Integrating a Big Data Platform into Government:

Integrating a Big Data Platform into Government: Integrating a Big Data Platform into Government: Drive Better Decisions for Policy and Program Outcomes John Haddad, Senior Director Product Marketing, Informatica Digital Government Institute s Government

More information

A Visualization is Worth a Thousand Tables: How IBM Business Analytics Lets Users See Big Data

A Visualization is Worth a Thousand Tables: How IBM Business Analytics Lets Users See Big Data White Paper A Visualization is Worth a Thousand Tables: How IBM Business Analytics Lets Users See Big Data Contents Executive Summary....2 Introduction....3 Too much data, not enough information....3 Only

More information

North Highland Data and Analytics. Data Governance Considerations for Big Data Analytics

North Highland Data and Analytics. Data Governance Considerations for Big Data Analytics North Highland and Analytics Governance Considerations for Big Analytics Agenda Traditional BI/Analytics vs. Big Analytics Types of Requiring Governance Key Considerations Information Framework Organizational

More information

15.00 15.30 30 XML enabled databases. Non relational databases. Guido Rotondi

15.00 15.30 30 XML enabled databases. Non relational databases. Guido Rotondi Programme of the ESTP training course on BIG DATA EFFECTIVE PROCESSING AND ANALYSIS OF VERY LARGE AND UNSTRUCTURED DATA FOR OFFICIAL STATISTICS Rome, 5 9 May 2014 Istat Piazza Indipendenza 4, Room Vanoni

More information

Big Data Analytics. Chances and Challenges. Volker Markl

Big Data Analytics. Chances and Challenges. Volker Markl Volker Markl Professor and Chair Database Systems and Information Management (DIMA), Technische Universität Berlin www.dima.tu-berlin.de Big Data Analytics Chances and Challenges Volker Markl DIMA BDOD

More information

Framework and key technologies for big data based on manufacturing Shan Ren 1, a, Xin Zhao 2, b

Framework and key technologies for big data based on manufacturing Shan Ren 1, a, Xin Zhao 2, b International Conference on Materials Engineering and Information Technology Applications (MEITA 2015) Framework and key technologies for big data based on manufacturing Shan Ren 1, a, Xin Zhao 2, b 1

More information

Apache Hadoop: The Big Data Refinery

Apache Hadoop: The Big Data Refinery Architecting the Future of Big Data Whitepaper Apache Hadoop: The Big Data Refinery Introduction Big data has become an extremely popular term, due to the well-documented explosion in the amount of data

More information

Big Data - Infrastructure Considerations

Big Data - Infrastructure Considerations April 2014, HAPPIEST MINDS TECHNOLOGIES Big Data - Infrastructure Considerations Author Anand Veeramani / Deepak Shivamurthy SHARING. MINDFUL. INTEGRITY. LEARNING. EXCELLENCE. SOCIAL RESPONSIBILITY. Copyright

More information

How To Create A Data Science System

How To Create A Data Science System Enhance Collaboration and Data Sharing for Faster Decisions and Improved Mission Outcome Richard Breakiron Senior Director, Cyber Solutions Rbreakiron@vion.com Office: 571-353-6127 / Cell: 803-443-8002

More information

BIG DATA THE NEW OPPORTUNITY

BIG DATA THE NEW OPPORTUNITY Feature Biswajit Mohapatra is an IBM Certified Consultant and a global integrated delivery leader for IBM s AMS business application modernization (BAM) practice. He is IBM India s competency head for

More information

IBM System x reference architecture solutions for big data

IBM System x reference architecture solutions for big data IBM System x reference architecture solutions for big data Easy-to-implement hardware, software and services for analyzing data at rest and data in motion Highlights Accelerates time-to-value with scalable,

More information

Wikibon Big Data Analytics Survey: Barriers to Adoption by Role

Wikibon Big Data Analytics Survey: Barriers to Adoption by Role Wikibon.com - http://wikibon.com Wikibon Big Data Analytics Survey: Barriers to Adoption by Role by George Gilbert - 12 August 2015 http://wikibon.com/wikibon-big-data-analytics-survey-barriers-to-adoption-by-role/

More information

Databricks. A Primer

Databricks. A Primer Databricks A Primer Who is Databricks? Databricks vision is to empower anyone to easily build and deploy advanced analytics solutions. The company was founded by the team who created Apache Spark, a powerful

More information

Predicting the future of predictive analytics. December 2013

Predicting the future of predictive analytics. December 2013 Predicting the future of predictive analytics December 2013 Executive Summary Organizations are now exploring the possibilities of using historical data to exploit growth opportunities The proliferation

More information

IBM SECURITY QRADAR INCIDENT FORENSICS

IBM SECURITY QRADAR INCIDENT FORENSICS IBM SECURITY QRADAR INCIDENT FORENSICS DELIVERING CLARITY TO CYBER SECURITY INVESTIGATIONS Gyenese Péter Channel Sales Leader, CEE IBM Security Systems 12014 IBM Corporation Harsh realities for many enterprise

More information

1 st Symposium on Colossal Data and Networking (CDAN-2016) March 18-19, 2016 Medicaps Group of Institutions, Indore, India

1 st Symposium on Colossal Data and Networking (CDAN-2016) March 18-19, 2016 Medicaps Group of Institutions, Indore, India 1 st Symposium on Colossal Data and Networking (CDAN-2016) March 18-19, 2016 Medicaps Group of Institutions, Indore, India Call for Papers Colossal Data Analysis and Networking has emerged as a de facto

More information

The Intersection of Big Data, Data Science, and The Internet of Things

The Intersection of Big Data, Data Science, and The Internet of Things The Intersection of Big Data, Data Science, and The Internet of Things Bebo White SLAC National Accelerator Laboratory/ Stanford University bebo@slac.stanford.edu SLAC is a US national laboratory operated

More information

Advanced Big Data Analytics with R and Hadoop

Advanced Big Data Analytics with R and Hadoop REVOLUTION ANALYTICS WHITE PAPER Advanced Big Data Analytics with R and Hadoop 'Big Data' Analytics as a Competitive Advantage Big Analytics delivers competitive advantage in two ways compared to the traditional

More information

Model, Analyze and Optimize the Supply Chain

Model, Analyze and Optimize the Supply Chain Model, Analyze and Optimize the Supply Chain Optimize networks Improve product flow Right-size inventory Simulate service Balance production Optimize routes The Leading Supply Chain Design and Analysis

More information

How To Create An Insight Analysis For Cyber Security

How To Create An Insight Analysis For Cyber Security IBM i2 Enterprise Insight Analysis for Cyber Analysis Protect your organization with cyber intelligence Highlights Quickly identify threats, threat actors and hidden connections with multidimensional analytics

More information

Data Centric Computing Revisited

Data Centric Computing Revisited Piyush Chaudhary Technical Computing Solutions Data Centric Computing Revisited SPXXL/SCICOMP Summer 2013 Bottom line: It is a time of Powerful Information Data volume is on the rise Dimensions of data

More information

Bruhati Technologies. About us. ISO 9001:2008 certified. Technology fit for Business

Bruhati Technologies. About us. ISO 9001:2008 certified. Technology fit for Business Bruhati Technologies ISO 9001:2008 certified Technology fit for Business About us 1 Strong, agile and adaptive Leadership Geared up technologies for and fast moving long lasting With sound understanding

More information

Big Data Mining: Challenges and Opportunities to Forecast Future Scenario

Big Data Mining: Challenges and Opportunities to Forecast Future Scenario Big Data Mining: Challenges and Opportunities to Forecast Future Scenario Poonam G. Sawant, Dr. B.L.Desai Assist. Professor, Dept. of MCA, SIMCA, Savitribai Phule Pune University, Pune, Maharashtra, India

More information

I. TODAY S UTILITY INFRASTRUCTURE vs. FUTURE USE CASES...1 II. MARKET & PLATFORM REQUIREMENTS...2

I. TODAY S UTILITY INFRASTRUCTURE vs. FUTURE USE CASES...1 II. MARKET & PLATFORM REQUIREMENTS...2 www.vitria.com TABLE OF CONTENTS I. TODAY S UTILITY INFRASTRUCTURE vs. FUTURE USE CASES...1 II. MARKET & PLATFORM REQUIREMENTS...2 III. COMPLEMENTING UTILITY IT ARCHITECTURES WITH THE VITRIA PLATFORM FOR

More information

Data Catalogs for Hadoop Achieving Shared Knowledge and Re-usable Data Prep. Neil Raden Hired Brains Research, LLC

Data Catalogs for Hadoop Achieving Shared Knowledge and Re-usable Data Prep. Neil Raden Hired Brains Research, LLC Data Catalogs for Hadoop Achieving Shared Knowledge and Re-usable Data Prep Neil Raden Hired Brains Research, LLC Traditionally, the job of gathering and integrating data for analytics fell on data warehouses.

More information

Essential Elements of an IoT Core Platform

Essential Elements of an IoT Core Platform Essential Elements of an IoT Core Platform Judith Hurwitz President and CEO Daniel Kirsch Principal Analyst and Vice President Sponsored by Hitachi Introduction The maturation of the enterprise cloud,

More information

Knowledge Discovery from patents using KMX Text Analytics

Knowledge Discovery from patents using KMX Text Analytics Knowledge Discovery from patents using KMX Text Analytics Dr. Anton Heijs anton.heijs@treparel.com Treparel Abstract In this white paper we discuss how the KMX technology of Treparel can help searchers

More information

Introduction to Data Mining

Introduction to Data Mining Introduction to Data Mining 1 Why Data Mining? Explosive Growth of Data Data collection and data availability Automated data collection tools, Internet, smartphones, Major sources of abundant data Business:

More information

BIG DATA What it is and how to use?

BIG DATA What it is and how to use? BIG DATA What it is and how to use? Lauri Ilison, PhD Data Scientist 21.11.2014 Big Data definition? There is no clear definition for BIG DATA BIG DATA is more of a concept than precise term 1 21.11.14

More information

Testing Big data is one of the biggest

Testing Big data is one of the biggest Infosys Labs Briefings VOL 11 NO 1 2013 Big Data: Testing Approach to Overcome Quality Challenges By Mahesh Gudipati, Shanthi Rao, Naju D. Mohan and Naveen Kumar Gajja Validate data quality by employing

More information

Government Technology Trends to Watch in 2014: Big Data

Government Technology Trends to Watch in 2014: Big Data Government Technology Trends to Watch in 2014: Big Data OVERVIEW The federal government manages a wide variety of civilian, defense and intelligence programs and services, which both produce and require

More information

White Paper. Version 1.2 May 2015 RAID Incorporated

White Paper. Version 1.2 May 2015 RAID Incorporated White Paper Version 1.2 May 2015 RAID Incorporated Introduction The abundance of Big Data, structured, partially-structured and unstructured massive datasets, which are too large to be processed effectively

More information

Sanjeev Kumar. contribute

Sanjeev Kumar. contribute RESEARCH ISSUES IN DATAA MINING Sanjeev Kumar I.A.S.R.I., Library Avenue, Pusa, New Delhi-110012 sanjeevk@iasri.res.in 1. Introduction The field of data mining and knowledgee discovery is emerging as a

More information

Anuradha Bhatia, Faculty, Computer Technology Department, Mumbai, India

Anuradha Bhatia, Faculty, Computer Technology Department, Mumbai, India Volume 3, Issue 9, September 2013 ISSN: 2277 128X International Journal of Advanced Research in Computer Science and Software Engineering Research Paper Available online at: www.ijarcsse.com A Real Time

More information

IBM Software Delivering trusted information for the modern data warehouse

IBM Software Delivering trusted information for the modern data warehouse Delivering trusted information for the modern data warehouse Make information integration and governance a best practice in the big data era Contents 2 Introduction In ever-changing business environments,

More information

ANALYTICS BUILT FOR INTERNET OF THINGS

ANALYTICS BUILT FOR INTERNET OF THINGS ANALYTICS BUILT FOR INTERNET OF THINGS Big Data Reporting is Out, Actionable Insights are In In recent years, it has become clear that data in itself has little relevance, it is the analysis of it that

More information

Luncheon Webinar Series May 13, 2013

Luncheon Webinar Series May 13, 2013 Luncheon Webinar Series May 13, 2013 InfoSphere DataStage is Big Data Integration Sponsored By: Presented by : Tony Curcio, InfoSphere Product Management 0 InfoSphere DataStage is Big Data Integration

More information

Industry 4.0 and Big Data

Industry 4.0 and Big Data Industry 4.0 and Big Data Marek Obitko, mobitko@ra.rockwell.com Senior Research Engineer 03/25/2015 PUBLIC PUBLIC - 5058-CO900H 2 Background Joint work with Czech Institute of Informatics, Robotics and

More information

Integrated Social and Enterprise Data = Enhanced Analytics

Integrated Social and Enterprise Data = Enhanced Analytics ORACLE WHITE PAPER, DECEMBER 2013 THE VALUE OF SOCIAL DATA Integrated Social and Enterprise Data = Enhanced Analytics #SocData CONTENTS Executive Summary 3 The Value of Enterprise-Specific Social Data

More information

Big Data better business benefits

Big Data better business benefits Big Data better business benefits Paul Edwards, HouseMark 2 December 2014 What I ll cover.. Explain what big data is Uses for Big Data and the potential for social housing What Big Data means for HouseMark

More information

IEEE International Conference on Computing, Analytics and Security Trends CAST-2016 (19 21 December, 2016) Call for Paper

IEEE International Conference on Computing, Analytics and Security Trends CAST-2016 (19 21 December, 2016) Call for Paper IEEE International Conference on Computing, Analytics and Security Trends CAST-2016 (19 21 December, 2016) Call for Paper CAST-2015 provides an opportunity for researchers, academicians, scientists and

More information

Big Data. Fast Forward. Putting data to productive use

Big Data. Fast Forward. Putting data to productive use Big Data Putting data to productive use Fast Forward What is big data, and why should you care? Get familiar with big data terminology, technologies, and techniques. Getting started with big data to realize

More information

Demonstration of SAP Predictive Analysis 1.0, consumption from SAP BI clients and best practices

Demonstration of SAP Predictive Analysis 1.0, consumption from SAP BI clients and best practices September 10-13, 2012 Orlando, Florida Demonstration of SAP Predictive Analysis 1.0, consumption from SAP BI clients and best practices Vishwanath Belur, Product Manager, SAP Predictive Analysis Learning

More information

Innovative Advances in. Big Data and Analytics

Innovative Advances in. Big Data and Analytics Innovative Advances in Big Data and Analytics STANLEY is leading the way with innovative advances in big data and analytics, providing unparalleled visibility into your organization s activities and operations.

More information

Associate Professor, Department of CSE, Shri Vishnu Engineering College for Women, Andhra Pradesh, India 2

Associate Professor, Department of CSE, Shri Vishnu Engineering College for Women, Andhra Pradesh, India 2 Volume 6, Issue 3, March 2016 ISSN: 2277 128X International Journal of Advanced Research in Computer Science and Software Engineering Research Paper Available online at: www.ijarcsse.com Special Issue

More information

Using In-Memory Computing to Simplify Big Data Analytics

Using In-Memory Computing to Simplify Big Data Analytics SCALEOUT SOFTWARE Using In-Memory Computing to Simplify Big Data Analytics by Dr. William Bain, ScaleOut Software, Inc. 2012 ScaleOut Software, Inc. 12/27/2012 T he big data revolution is upon us, fed

More information

De la Business Intelligence aux Big Data. Marie- Aude AUFAURE Head of the Business Intelligence team Ecole Centrale Paris. 22/01/14 Séminaire Big Data

De la Business Intelligence aux Big Data. Marie- Aude AUFAURE Head of the Business Intelligence team Ecole Centrale Paris. 22/01/14 Séminaire Big Data De la Business Intelligence aux Big Data Marie- Aude AUFAURE Head of the Business Intelligence team Ecole Centrale Paris 22/01/14 Séminaire Big Data 1 Agenda EvoluHon of Business Intelligence SemanHc Technologies

More information

HadoopTM Analytics DDN

HadoopTM Analytics DDN DDN Solution Brief Accelerate> HadoopTM Analytics with the SFA Big Data Platform Organizations that need to extract value from all data can leverage the award winning SFA platform to really accelerate

More information

Introduction to Engineering Using Robotics Experiments Lecture 17 Big Data

Introduction to Engineering Using Robotics Experiments Lecture 17 Big Data Introduction to Engineering Using Robotics Experiments Lecture 17 Big Data Yinong Chen 2 Big Data Big Data Technologies Cloud Computing Service and Web-Based Computing Applications Industry Control Systems

More information

APPLICATION OF MULTI-AGENT SYSTEMS FOR NETWORK AND INFORMATION PROTECTION

APPLICATION OF MULTI-AGENT SYSTEMS FOR NETWORK AND INFORMATION PROTECTION 18-19 September 2014, BULGARIA 137 Proceedings of the International Conference on Information Technologies (InfoTech-2014) 18-19 September 2014, Bulgaria APPLICATION OF MULTI-AGENT SYSTEMS FOR NETWORK

More information

Data Isn't Everything

Data Isn't Everything June 17, 2015 Innovate Forward Data Isn't Everything The Challenges of Big Data, Advanced Analytics, and Advance Computation Devices for Transportation Agencies. Using Data to Support Mission, Administration,

More information

BIG DATA & DATA SCIENCE

BIG DATA & DATA SCIENCE BIG DATA & DATA SCIENCE ACADEMY PROGRAMS IN-COMPANY TRAINING PORTFOLIO 2 TRAINING PORTFOLIO 2016 Synergic Academy Solutions BIG DATA FOR LEADING BUSINESS Big data promises a significant shift in the way

More information

International Journal of Innovative Research in Computer and Communication Engineering

International Journal of Innovative Research in Computer and Communication Engineering FP Tree Algorithm and Approaches in Big Data T.Rathika 1, J.Senthil Murugan 2 Assistant Professor, Department of CSE, SRM University, Ramapuram Campus, Chennai, Tamil Nadu,India 1 Assistant Professor,

More information

HOW TO DO A SMART DATA PROJECT

HOW TO DO A SMART DATA PROJECT April 2014 Smart Data Strategies HOW TO DO A SMART DATA PROJECT Guideline www.altiliagroup.com Summary ALTILIA s approach to Smart Data PROJECTS 3 1. BUSINESS USE CASE DEFINITION 4 2. PROJECT PLANNING

More information

Smarter Planet evolution

Smarter Planet evolution Smarter Planet evolution 13/03/2012 2012 IBM Corporation Ignacio Pérez González Enterprise Architect ignacio.perez@es.ibm.com @ignaciopr Mike May Technologies of the Change Capabilities Tendencies Vision

More information

Transforming the Telecoms Business using Big Data and Analytics

Transforming the Telecoms Business using Big Data and Analytics Transforming the Telecoms Business using Big Data and Analytics Event: ICT Forum for HR Professionals Venue: Meikles Hotel, Harare, Zimbabwe Date: 19 th 21 st August 2015 AFRALTI 1 Objectives Describe

More information

Master big data to optimize the oil and gas lifecycle

Master big data to optimize the oil and gas lifecycle Viewpoint paper Master big data to optimize the oil and gas lifecycle Information management and analytics (IM&A) helps move decisions from reactive to predictive Table of contents 4 Getting a handle on

More information

Architecture 3.0 Landscape Analytics

Architecture 3.0 Landscape Analytics Architecture 3.0 Landscape Analytics Jürgen Döllner Hasso- Plattner- Institut Landscape Analytics Big Data Big Data Analytics Visual Analytics Predictive Analytics Landscape Analytics Big Data Data is

More information

CONNECTING DATA WITH BUSINESS

CONNECTING DATA WITH BUSINESS CONNECTING DATA WITH BUSINESS Big Data and Data Science consulting Business Value through Data Knowledge Synergic Partners is a specialized Big Data, Data Science and Data Engineering consultancy firm

More information

Convergence of Big Data and Cloud

Convergence of Big Data and Cloud American Journal of Engineering Research (AJER) e-issn : 2320-0847 p-issn : 2320-0936 Volume-03, Issue-05, pp-266-270 www.ajer.org Research Paper Open Access Convergence of Big Data and Cloud Sreevani.Y.V.

More information

Kingdom Big Data & Analytics Summit 28 FEB 1 March 2016 Agenda MASTERCLASS A 28 Feb 2016

Kingdom Big Data & Analytics Summit 28 FEB 1 March 2016 Agenda MASTERCLASS A 28 Feb 2016 Kingdom Big Data & Analytics Summit 28 FEB 1 March 2016 Agenda MASTERCLASS A 28 Feb 2016 9.00am To 12.00pm Big Data Technology and Analytics Workshop MASTERCLASS LEADERS Venkata P. Alla A highly respected

More information