Big Data Introduction, Importance and Current Perspective of Challenges

Size: px
Start display at page:

Download "Big Data Introduction, Importance and Current Perspective of Challenges"

Transcription

1 International Journal of Advances in Engineering Science and Technology 221 Available online at ISSN: Big Data Introduction, Importance and Current Perspective of Challenges Vinod Sharma 1, Prof. N. K. Joshi 2, Manisha 3 Department of Computer Science Research Scholar, Career Point University Kota, Rajathan, India 1 2 Director, Modi Institute of Management & Technology Dadabari, Kota, Rajathan, India 2 Asst. Professor, Arya Girls College, Ambala Cantt, Haryana, India 3 1, 2, 3 Abstract- Man and Machine are rapidly generating data. While writing an , calling, texting, tweeting, streaming audio and video, playing game, online shopping, reservation, user activity generates data about their needs and preferences, as well as the quality of their experience with millions of machines and process every moment of every fraction of a time. Even when we are not using our devices, the network is generating time, location, offline records, pending messages and other data that keeps services running and ready for next use. Organization are starting to appreciate the importance of using more data in order to support decisions and arrangements for their strategies. There considered and proved through the study and practical implementation cases that More Data usually beats better structure and management. A massive sets of data gives a better output and wide range of correlations but also working with it can become a challenge due to limitations of processing. So Organizations initiated to evaluate that they can chose to invest more in processing massive sets of data rather than investing in costly algorithms and techniques. Vision of big data is that organizations will be able to reap and harness every byte of relevant and interrelated data and use it to make the best decisions. Big data technologies not only consider solution for massive sets of data, but more importantly, the ability to understand and take advantage of its full value. There is a lots of new challenges to handle big data, real-time communication, and decisions create lots of other challenges and issues regarding analyzing, management and processing. This article intends to define the concept of Big Data and stress the importance of Big Data. This paper also throws some light on other challenges and issues. Keywords Massive Data Sets, Big Data, Analytics, Correlations, Scalability. I. INTRODUCTION big data is an all broad term for any collection of data sets so massive and complex and that increase complexity to process them using classical data processing applications. The processing applications cover analysis, capture, curation, search, sharing, storing, transfer, visualization, privacy violation, and secure infrastructure that is too divers, fast changing or massive for conventional technologies, skill and infrastructure to address efficiently. Every second, minute, and an hour, we create millions bytes of data and that s too much from last decades. This data comes from everywhere: social media sites and response, digital records like picture, movie, fingerprint data, climate information from sensors, GPS and GIS data, Financial data, Transportations, Health and Hospitalities to name a few and this is a big data. Nowadays big data is a growing torrent some of examples are 1639 millions of mobile users worldwide in 2014[2] and they are producing data, 600TB data per day processed by Facebook[3], PB data stored in NSA data center[4], 235 terabytes data collected by the US library of congress by April 2011, 10 gigabytes of data may be transferred from its servers every second in CERN LHC Project, Amazon generates over $80,000 in online sales, 2.7 ZB of data exist in the digital universe today and they are few. With expanding of data, big data is capturing its value.other way we are living in a voluminous informational environment and we are moving a knowledge based environment. But voluminous informational environment provide better decision after analysis it. How much data generate by company, government and any entity that belongs to universe represents a big space where great amount of information added every time. Google, Facebook, YouTube, CERNS, Twitter, Apple,

2 IJAEST, Volume 4, Number 3 Vinod Sharma et al. LinkedIn, Netflix, Expedia, National and Local political campaigns. They know that that the age of Big Data is here and its here to stay. The swelling ranks of organizations that increasingly depend on big-data technologies include dozens of familiar names and a growing number you ve never heard of. II. DEFINITION OF BIG DATA There are some of definitions of big data In IBM s view Big Data has four aspects: volume, velocity, variety and veracity. Volume: refers to the quantity of data gathered by a company. This data must be used further to obtain important knowledge; Velocity: refers to the time in which Big Data can be processed. Some activities are very important and need immediate responses, that is why fast processing maximizes efficiency; Variety: Refers to the type of data that Big Data can comprise. This data can be structured as well as unstructured; Veracity: refers to the degree in which a leader trusts the used information in order to take decision. So getting the right correlations in Big Data is very important for the business future. [4] VOLUME Data at Rest, Ofload or cold data in TB, PB Redundancy elimination Transactions, Retention Records,Tables, Files VELOCITY Data in Moion, In Stream, In Batch Analysis to enable decisions within fractions of a seconds VARIETY Data in many forms Structured Unstructured Semistructured Data access midleware and ETLM VERACITY Data in doubts Data uncertainity Managing the reliablity and predictability of inherently imprecise data types Big data is the term increasingly used to describe the process of applying serious computing power the latest in machine learning and artificial intelligence to seriously massive and often highly complex sets of information."[5] Big Data: when the size and performance requirements for data management become significant design and decision factors for implementing a data management and analysis system. For some organizations, facing hundreds of gigabytes of data for the first time may trigger a need to reconsider data management options. For others, it may take tens or hundreds of terabytes before data size becomes a significant consideration. [6] "Big data (also spelled Big Data) is a general term used to describe the voluminous amount of unstructured and semistructured data a company creates -- data that would take too much time and cost too much money to load into a relational database for analysis. Although big data doesn't refer to any specific quantity, the term is often used when speaking about petabytes and Exabyte s of data."[7] The definition of Big Data is very fluid, as it is a moving target what can be easily manipulated with common tools and specific to the organization: what can be managed and stewarded by any one institution in its infrastructure. One researcher or organization s concept of a large data set is small to another. [8] We define Big Data as a cultural, technological, and scholarly phenomenon that rests on the interplay of: (1) Technology: maximizing computation power and algorithmic accuracy to gather, analyze, link, and compare large data sets. (2) Analysis: drawing on large data sets to identify patterns in order to make economic, social, technical, and legal claims. (3) Mythology: the widespread belief that large data sets offer a higher form of intelligence and knowledge that can generate insights that were previously impossible, with the aura of truth, objectivity, and accuracy.[9]

3 Big Data Introduction, Importance and Current Perspective of Challenges 223 By Roger Magoulas from O Reilly media first introduced to the computing world Big Data term in 2005 in order to define massive amount of data that traditional data management techniques cannot manage and process due to the complexity and size of this data A study on the Evolution of Big Data as a Research and Scientific Topic shows that the term Big Data was present in research starting with 1970s but has been comprised in publications in Nowadays the Big Data concept is treated from different points of view covering its implications in many fields. According to MiKE 2.0, the open source standard for Information Management, Big Data is defined by its size, comprising a large, complex and independent collection of small data sets, each with the potential to interact. In addition, an important aspect of Big Data is the fact that it cannot be handled with standard data management techniques due to the inconsistency and unpredictability of the possible combinations. The Oxford English Dictionary (OED) defines big data as an extremely large data sets that may be analysed computationally to reveal patterns, trends, and associations, especially relating to human behaviour and interactions: much IT investment is going towards managing and maintaining big data In 2011 big data study by McKinsey highlighted that definitional challenge. Defining big data as datasets whose size is beyond the ability of typical database software tools to capture, store, manage, and analyze, the McKinsey researchers acknowledged that this definition is intentionally subjective and incorporates a moving definition of how big a dataset needs to be in order to be considered big data. As a result, all the quantitative insights of the study, including the updating of the UC Berkeley numbers by estimating how much new data is stored by enterprises and consumers annually, relate to digital data, rather than just big data, e.g., no attempt was made to estimate how much of the data (or datasets ) enterprises store is big data. In addition, in Gartner s IT Glossary Big Data is defined as high volume, velocity and variety information assets that demand cost-effective, innovative forms of information processing for enhanced insight and decision making. In a simpler definition we consider Big Data to be an expression that comprises different data sets of very large, highly complex, unstructured, organized, stored and processed using specific methods and techniques used for business processes. There are a lot of definitions on Big Data circulating around the world, but we consider that the most important one is the one that each leader gives to its one company s data. The way that Big Data is defined has implication in the strategy of a business. Each leader has to define the concept in order to bring competitive advantage for the company. III. IMPORTANCE OF BIG DATA The main importance of big data exist because more data more accurate results. Big data consist in the potential to improve efficiency in the context of use a large volume of data, of different type and various kind of dimensions. If big data is defined properly and used efficiently all over, organization can get a better view on their business therefore leading to efficiency in different areas like sales, improving the manufactured product, health report, data selection, statistical sample and so forth. Big data can be used effectively in the context of- 1. Big Data can provide transparent way of significant value by making and processing information. There is still a massive information not in digital form, e.g. traditional and classical records, sensors based data, or not textual based data not easily accessible and searchable through networks due to source of efficiency. 2. As organization create, store and process more transactional data in digital form, they can process more accurately and detailed efficiency and performance on everything from records and expose flightiness and boost functionality. In fact organization can store, analyse, process, and control big data using controlled experiments to make accurate decision. 3. Big Data allows segmentation of records and therefore much more precisely made-to-measure products or services.

4 IJAEST, Volume 4, Number 3 Vinod Sharma et al. 4. More data more analysis more accurate results that improve decision-making, minimize risk, and expose valuable fact from insights that would otherwise remain hidden. 5. Big Data can be used to develop the next generation of products and services. 6. Big Data act as a bridge the Gap between IT and Business 7. Big data can create new business model based on legacy and classical data 8. Getting the Right Information for Your Business 9. In information technology in order to improve security and troubleshooting by analyzing the patterns in the existing logs and events; 10. In customer service by using information from call centers in order to get the customer pattern and thus enhance customer satisfaction by customizing services; 11. In improving services and products through the use of social media content. By knowing the potential customers preferences the company can modify its product in order to address a larger area of people; 12. In the detection of fraud in the online transactions for any industry; 13. In risk assessment by analyzing information from the transactions on the financial market. 14. Big data can provide better results and transformative potential in Health care, Public Sector Administration, Retails, Manufacturing, and Personal Location, Proving the Value in IT to Business. Big data can do much more that are listed from above but that follow some inconsistencies and issue due its wide features like Volume, Velocity, Variety, Veracity, Value and many others. IV. BIG DATA CHALLENGES AND DILEMMAS Big Data is rapidly growing day by day due to massive data sets that comes from various resources but also reflects the trend in field of science and technology, this trend opens the door to understanding the data world and its hidden part for making decisions. Volume is not entirely the issue of big data but processing, analysis and controlling of traditional, structured, and unstructured data. Structured data depend on process and type s storage but need to differentiate in between data-driven-process and process-driven-data. Nowadays big data handling requirements are completely different from traditional data platform. Some of challenges are facing nowadays 1. The understanding of big data structure is major concerns before storing, processing, analyse it. According to IBM Volume, Velocity, variety, and Veracity is important factor but handling big data require more (e.g. values). There the time is important factor in transactional analysis because some of them need to be require highly frequent in order to accessing fast any change in the environment. 2. Considering the concept that Big Data is new in Organization, it is necessary for these organization to learn how to use new technology after invention of it in competitive age. 3. Prepare IT Specialist for handling big data. Only right talent can handle new technology and interpret the data for meaningful information for business. According to McKinsey s study on Big Data called Big Data: The next frontier for innovation, there is a need for up to 190,000 more workers with analytical expertise and 1.5 million more data- literate managers only in the United States. This statistics are a proof that in order for a company to take the Big Data initiative has to either hire experts or train existing employees on the new field. [10] 4. Privacy and Security is major challenges for big data. Because Big Data consists a massive amount of complex structure data, and it is very tedious for a company for considering security, privacy and authentication levels according to users and process. If we developing big data platform for handling global business need then consider capability of security in various range.

5 Big Data Introduction, Importance and Current Perspective of Challenges There is difficult to identify the right data for right process or user and determining best utilization of this and that very different from the traditional platform. 6. Accessibility and connectivity of big data from third party processing data points is in initial phase of development. A majority of data points are not yet connected today, and organization often do not providing right platforms for access and processing the data across the enterprise. Organization offering some of utilities to integrate operational technologies, such a real-time grid management, cloud, with information technologies like smart metering. 7. Big Data rapidly change day by day and data world is evolving fast. IT architecture require a strong innovative technological partner that can help for adapt rapidly changes in big data. 8. Major organization need continuous availability and high scalability of Big Data Platform. Because your data never go down due to high scalability from different data centers. 9. Flexible design for multiformity of workload. Because dimension of big data in various range. 10. Cost of platform doesn t have to break the bank. IV.CONCLUSION Nowadays big data entering in our life in the context of better processing, and results. This technology emerging various kind of technology and provide values from discovery of knowledge. In this paper we discuss about basically three issues Definition, Importance and Challenges of big data. We discuss which definition is suitable for current scenario of big data then answer of why we using big data after that challenges of acquire big data. Today lot of platforms and challenging situations are there ahead of this world to make new technologies for both processing and storage. Extensions of traditional database technologies are also discussed to deal with big data. So more technologies can be made to deal with big data processing and storage to make it more concise and meaningful. REFERENCE [1] IBM, IBM BIG DATA. [Online]. Available: [2] EMarketers, 2 Billion Consumer WorldWide to Get Smart(phones), emarketer,2014.[online].available [3] P. Vagata and K. Wilfong, Scaling the Facebook data warehouse to 300 PB Engineering Blog Facebook Code Facebook, [Online]. Available: https://code.facebook.com/posts/ /scaling-the-facebook-data-warehouse-to-300-pb/. [4] R. Jennings, NetAppVoice NSA s Huge Utah Datacenter How Much Of Your Data Will It Store Experts Disagree, Forbes, [Online]. Available: [5] Microsoft Research, The Big Bang: How the Big Data Explosion Is Changing the World, [Online]. Available: [6] J. Guterman, Release 2.0: Issue 11 Big Data, [Online]. Available: [7] R. More, Big Data Definition ->, [Online]. Available: [8] L. Johnston, Data is the New Black, [Online]. Available: [9] D. Boyd and K. Crawford, Critical Questions for Big Data, Information, Commun. Soc., vol. 15, no. 5, pp , [10] A. H. B. James Manyika, Michael Chui, Brad Brown, Jacques Bughin, Richard Dobbs, Charles Roxburgh, Big data: The next frontier for innovation, competition, and productivity, McKinsey Global Institute, [Online]. Available: =0

Danny Wang, Ph.D. Vice President of Business Strategy and Risk Management Republic Bank

Danny Wang, Ph.D. Vice President of Business Strategy and Risk Management Republic Bank Danny Wang, Ph.D. Vice President of Business Strategy and Risk Management Republic Bank Agenda» Overview» What is Big Data?» Accelerates advances in computer & technologies» Revolutionizes data measurement»

More information

The Big Deal about Big Data. Mike Skinner, CPA CISA CITP HORNE LLP

The Big Deal about Big Data. Mike Skinner, CPA CISA CITP HORNE LLP The Big Deal about Big Data Mike Skinner, CPA CISA CITP HORNE LLP Mike Skinner, CPA CISA CITP Senior Manager, IT Assurance & Risk Services HORNE LLP Focus areas: IT security & risk assessment IT governance,

More information

International Journal of Advanced Engineering Research and Applications (IJAERA) ISSN: 2454-2377 Vol. 1, Issue 6, October 2015. Big Data and Hadoop

International Journal of Advanced Engineering Research and Applications (IJAERA) ISSN: 2454-2377 Vol. 1, Issue 6, October 2015. Big Data and Hadoop ISSN: 2454-2377, October 2015 Big Data and Hadoop Simmi Bagga 1 Satinder Kaur 2 1 Assistant Professor, Sant Hira Dass Kanya MahaVidyalaya, Kala Sanghian, Distt Kpt. INDIA E-mail: simmibagga12@gmail.com

More information

INTERNATIONAL JOURNAL OF PURE AND APPLIED RESEARCH IN ENGINEERING AND TECHNOLOGY

INTERNATIONAL JOURNAL OF PURE AND APPLIED RESEARCH IN ENGINEERING AND TECHNOLOGY INTERNATIONAL JOURNAL OF PURE AND APPLIED RESEARCH IN ENGINEERING AND TECHNOLOGY A PATH FOR HORIZING YOUR INNOVATIVE WORK A SURVEY ON BIG DATA ISSUES AMRINDER KAUR Assistant Professor, Department of Computer

More information

BIG DATA FUNDAMENTALS

BIG DATA FUNDAMENTALS BIG DATA FUNDAMENTALS Timeframe Minimum of 30 hours Use the concepts of volume, velocity, variety, veracity and value to define big data Learning outcomes Critically evaluate the need for big data management

More information

Indexed Terms: Big Data, benefits, characteristics, definition, problems, unstructured data

Indexed Terms: Big Data, benefits, characteristics, definition, problems, unstructured data Managing Data through Big Data: A Review Harsimran Singh Anand Assistant Professor, PG Dept of Computer Science & IT, DAV College, Amritsar Email id: harsimran_anand@yahoo.com A B S T R A C T Big Data

More information

www.pwc.com/oracle Next presentation starting soon Business Analytics using Big Data to gain competitive advantage

www.pwc.com/oracle Next presentation starting soon Business Analytics using Big Data to gain competitive advantage www.pwc.com/oracle Next presentation starting soon Business Analytics using Big Data to gain competitive advantage If every image made and every word written from the earliest stirring of civilization

More information

CSC590: Selected Topics BIG DATA & DATA MINING. Lecture 2 Feb 12, 2014 Dr. Esam A. Alwagait

CSC590: Selected Topics BIG DATA & DATA MINING. Lecture 2 Feb 12, 2014 Dr. Esam A. Alwagait CSC590: Selected Topics BIG DATA & DATA MINING Lecture 2 Feb 12, 2014 Dr. Esam A. Alwagait Agenda Introduction What is Big Data Why Big Data? Characteristics of Big Data Applications of Big Data Problems

More information

Data Refinery with Big Data Aspects

Data Refinery with Big Data Aspects International Journal of Information and Computation Technology. ISSN 0974-2239 Volume 3, Number 7 (2013), pp. 655-662 International Research Publications House http://www. irphouse.com /ijict.htm Data

More information

Why is BIG Data Important?

Why is BIG Data Important? Why is BIG Data Important? March 2012 1 Why is BIG Data Important? A Navint Partners White Paper May 2012 Why is BIG Data Important? March 2012 2 What is Big Data? Big data is a term that refers to data

More information

Busin i ess I n I t n e t ll l i l g i e g nce c T r T e r nds For 2013

Busin i ess I n I t n e t ll l i l g i e g nce c T r T e r nds For 2013 Business Intelligence Trends For 2013 10 Trends The last few years the change in Business Intelligence seems to accelerate under the pressure of increased business demand and technology innovations. Here

More information

Now, Next and the Future: IT, Big Data and other Implications for RIM. Presented by Michael S. Smith / http://about.me/mikessmith

Now, Next and the Future: IT, Big Data and other Implications for RIM. Presented by Michael S. Smith / http://about.me/mikessmith Now, Next and the Future: IT, Big Data and other Implications for RIM Agenda for This Afternoon Now: What trends are creating implications within the profession? Next: Why is IT now concerned about RIM?

More information

Are You Ready for Big Data?

Are You Ready for Big Data? Are You Ready for Big Data? Jim Gallo National Director, Business Analytics February 11, 2013 Agenda What is Big Data? How do you leverage Big Data in your company? How do you prepare for a Big Data initiative?

More information

What happens when Big Data and Master Data come together?

What happens when Big Data and Master Data come together? What happens when Big Data and Master Data come together? Jeremy Pritchard Master Data Management fgdd 1 What is Master Data? Master data is data that is shared by multiple computer systems. The Information

More information

How Big Is Big Data Adoption? Survey Results. Survey Results... 4. Big Data Company Strategy... 6

How Big Is Big Data Adoption? Survey Results. Survey Results... 4. Big Data Company Strategy... 6 Survey Results Table of Contents Survey Results... 4 Big Data Company Strategy... 6 Big Data Business Drivers and Benefits Received... 8 Big Data Integration... 10 Big Data Implementation Challenges...

More information

Big Data a threat or a chance?

Big Data a threat or a chance? Big Data a threat or a chance? Helwig Hauser University of Bergen, Dept. of Informatics Big Data What is Big Data? well, lots of data, right? we come back to this in a moment. certainly, a buzz-word but

More information

Managing Cloud Server with Big Data for Small, Medium Enterprises: Issues and Challenges

Managing Cloud Server with Big Data for Small, Medium Enterprises: Issues and Challenges Managing Cloud Server with Big Data for Small, Medium Enterprises: Issues and Challenges Prerita Gupta Research Scholar, DAV College, Chandigarh Dr. Harmunish Taneja Department of Computer Science and

More information

IoT and Big Data- The Current and Future Technologies: A Review

IoT and Big Data- The Current and Future Technologies: A Review Available Online at www.ijcsmc.com International Journal of Computer Science and Mobile Computing A Monthly Journal of Computer Science and Information Technology IJCSMC, Vol. 5, Issue. 1, January 2016,

More information

BIG DATA CHALLENGES AND PERSPECTIVES

BIG DATA CHALLENGES AND PERSPECTIVES BIG DATA CHALLENGES AND PERSPECTIVES Meenakshi Sharma 1, Keshav Kishore 2 1 Student of Master of Technology, 2 Head of Department, Department of Computer Science and Engineering, A P Goyal Shimla University,

More information

CIS 4930/6930 Spring 2014 Introduction to Data Science Data Intensive Computing. University of Florida, CISE Department Prof.

CIS 4930/6930 Spring 2014 Introduction to Data Science Data Intensive Computing. University of Florida, CISE Department Prof. CIS 4930/6930 Spring 2014 Introduction to Data Science Data Intensive Computing University of Florida, CISE Department Prof. Daisy Zhe Wang Data Science Overview Why, What, How, Who Outline Why Data Science?

More information

Are You Ready for Big Data?

Are You Ready for Big Data? Are You Ready for Big Data? Jim Gallo National Director, Business Analytics April 10, 2013 Agenda What is Big Data? How do you leverage Big Data in your company? How do you prepare for a Big Data initiative?

More information

WELCOME TO THE WORLD OF BIG DATA. NEW WORLD PROBLEMS, NEW WORLD SOLUTIONS

WELCOME TO THE WORLD OF BIG DATA. NEW WORLD PROBLEMS, NEW WORLD SOLUTIONS WELCOME TO THE WORLD OF BIG DATA. NEW WORLD PROBLEMS, NEW WORLD SOLUTIONS TECHNOLOGY by Zachary Zeus Data in our world has been exploding. According to IBM research, 90% of today s data was created in

More information

Deploying Big Data to the Cloud: Roadmap for Success

Deploying Big Data to the Cloud: Roadmap for Success Deploying Big Data to the Cloud: Roadmap for Success James Kobielus Chair, CSCC Big Data in the Cloud Working Group IBM Big Data Evangelist. IBM Data Magazine, Editor-in- Chief. IBM Senior Program Director,

More information

Statistical Challenges with Big Data in Management Science

Statistical Challenges with Big Data in Management Science Statistical Challenges with Big Data in Management Science Arnab Kumar Laha Indian Institute of Management Ahmedabad Analytics vs Reporting Competitive Advantage Reporting Prescriptive Analytics (Decision

More information

Data Centric Computing Revisited

Data Centric Computing Revisited Piyush Chaudhary Technical Computing Solutions Data Centric Computing Revisited SPXXL/SCICOMP Summer 2013 Bottom line: It is a time of Powerful Information Data volume is on the rise Dimensions of data

More information

COMP9321 Web Application Engineering

COMP9321 Web Application Engineering COMP9321 Web Application Engineering Semester 2, 2015 Dr. Amin Beheshti Service Oriented Computing Group, CSE, UNSW Australia Week 11 (Part II) http://webapps.cse.unsw.edu.au/webcms2/course/index.php?cid=2411

More information

Surfing the Data Tsunami: A New Paradigm for Big Data Processing and Analytics

Surfing the Data Tsunami: A New Paradigm for Big Data Processing and Analytics Surfing the Data Tsunami: A New Paradigm for Big Data Processing and Analytics Dr. Liangxiu Han Future Networks and Distributed Systems Group (FUNDS) School of Computing, Mathematics and Digital Technology,

More information

Analyzing Big Data: The Path to Competitive Advantage

Analyzing Big Data: The Path to Competitive Advantage White Paper Analyzing Big Data: The Path to Competitive Advantage by Marcia Kaplan Contents Introduction....2 How Big is Big Data?................................................................................

More information

Of all the data in recorded human history, 90 percent has been created in the last two years. - Mark van Rijmenam, Think Bigger, 2014

Of all the data in recorded human history, 90 percent has been created in the last two years. - Mark van Rijmenam, Think Bigger, 2014 What is Big Data? Of all the data in recorded human history, 90 percent has been created in the last two years. - Mark van Rijmenam, Think Bigger, 2014 Data in the Twentieth Century and before In 1663,

More information

CONNECTING DATA WITH BUSINESS

CONNECTING DATA WITH BUSINESS CONNECTING DATA WITH BUSINESS Big Data and Data Science consulting Business Value through Data Knowledge Synergic Partners is a specialized Big Data, Data Science and Data Engineering consultancy firm

More information

Industry Impact of Big Data in the Cloud: An IBM Perspective

Industry Impact of Big Data in the Cloud: An IBM Perspective Industry Impact of Big Data in the Cloud: An IBM Perspective Inhi Cho Suh IBM Software Group, Information Management Vice President, Product Management and Strategy email: inhicho@us.ibm.com twitter: @inhicho

More information

HP Vertica at MIT Sloan Sports Analytics Conference March 1, 2013 Will Cairns, Senior Data Scientist, HP Vertica

HP Vertica at MIT Sloan Sports Analytics Conference March 1, 2013 Will Cairns, Senior Data Scientist, HP Vertica HP Vertica at MIT Sloan Sports Analytics Conference March 1, 2013 Will Cairns, Senior Data Scientist, HP Vertica So What s the market s definition of Big Data? Datasets whose volume, velocity, variety

More information

BIG DATA: BIG BOOST TO BIG TECH

BIG DATA: BIG BOOST TO BIG TECH BIG DATA: BIG BOOST TO BIG TECH Ms. Tosha Joshi Department of Computer Applications, Christ College, Rajkot, Gujarat (India) ABSTRACT Data formation is occurring at a record rate. A staggering 2.9 billion

More information

CAP4773/CIS6930 Projects in Data Science, Fall 2014 [Review] Overview of Data Science

CAP4773/CIS6930 Projects in Data Science, Fall 2014 [Review] Overview of Data Science CAP4773/CIS6930 Projects in Data Science, Fall 2014 [Review] Overview of Data Science Dr. Daisy Zhe Wang CISE Department University of Florida August 25th 2014 20 Review Overview of Data Science Why Data

More information

Big Data Analytics: 14 November 2013

Big Data Analytics: 14 November 2013 www.pwc.com CSM-ACE 2013 Big Data Analytics: Take it to the next level in building innovation, differentiation and growth 14 About me Data analytics in the UK Forensic technology and data analytics in

More information

Keywords Big Data; OODBMS; RDBMS; hadoop; EDM; learning analytics, data abundance.

Keywords Big Data; OODBMS; RDBMS; hadoop; EDM; learning analytics, data abundance. Volume 4, Issue 11, November 2014 ISSN: 2277 128X International Journal of Advanced Research in Computer Science and Software Engineering Research Paper Available online at: www.ijarcsse.com Analytics

More information

An analysis of Big Data ecosystem from an HCI perspective.

An analysis of Big Data ecosystem from an HCI perspective. An analysis of Big Data ecosystem from an HCI perspective. Jay Sanghvi Rensselaer Polytechnic Institute For: Theory and Research in Technical Communication and HCI Rensselaer Polytechnic Institute Wednesday,

More information

BIG Data. An Introductory Overview. IT & Business Management Solutions

BIG Data. An Introductory Overview. IT & Business Management Solutions BIG Data An Introductory Overview IT & Business Management Solutions What is Big Data? Having been a dominating industry buzzword for the past few years, there is no contesting that Big Data is attracting

More information

Smarter Analytics. Barbara Cain. Driving Value from Big Data

Smarter Analytics. Barbara Cain. Driving Value from Big Data Smarter Analytics Driving Value from Big Data Barbara Cain Vice President Product Management - Business Intelligence and Advanced Analytics Business Analytics IBM Software Group 1 Agenda for today 1 Big

More information

The emergence of big data technology and analytics

The emergence of big data technology and analytics ABSTRACT The emergence of big data technology and analytics Bernice Purcell Holy Family University The Internet has made new sources of vast amount of data available to business executives. Big data is

More information

Impact of Big Data in Oil & Gas Industry. Pranaya Sangvai Reliance Industries Limited 04 Feb 15, DEJ, Mumbai, India.

Impact of Big Data in Oil & Gas Industry. Pranaya Sangvai Reliance Industries Limited 04 Feb 15, DEJ, Mumbai, India. Impact of Big Data in Oil & Gas Industry Pranaya Sangvai Reliance Industries Limited 04 Feb 15, DEJ, Mumbai, India. New Age Information 2.92 billions Internet Users in 2014 Twitter processes 7 terabytes

More information

Big Data and Analytics: Challenges and Opportunities

Big Data and Analytics: Challenges and Opportunities Big Data and Analytics: Challenges and Opportunities Dr. Amin Beheshti Lecturer and Senior Research Associate University of New South Wales, Australia (Service Oriented Computing Group, CSE) Talk: Sharif

More information

BIG DATA I N B A N K I N G

BIG DATA I N B A N K I N G $ BIG DATA IN BANKING Table of contents What is Big Data?... How data science creates value in Banking... Best practices for Banking. Case studies... 3 7 10 1. Fraud detection... 2. Contact center efficiency

More information

Transforming the Telecoms Business using Big Data and Analytics

Transforming the Telecoms Business using Big Data and Analytics Transforming the Telecoms Business using Big Data and Analytics Event: ICT Forum for HR Professionals Venue: Meikles Hotel, Harare, Zimbabwe Date: 19 th 21 st August 2015 AFRALTI 1 Objectives Describe

More information

Healthcare Measurement Analysis Using Data mining Techniques

Healthcare Measurement Analysis Using Data mining Techniques www.ijecs.in International Journal Of Engineering And Computer Science ISSN:2319-7242 Volume 03 Issue 07 July, 2014 Page No. 7058-7064 Healthcare Measurement Analysis Using Data mining Techniques 1 Dr.A.Shaik

More information

One thing everyone seems to agree with is that Big Data reflects the geometric growth of captured data and our intent to take advantage of it.

One thing everyone seems to agree with is that Big Data reflects the geometric growth of captured data and our intent to take advantage of it. 1. CONTEXT Everywhere you turn these days, you will hear three buzzwords: SaaS (Software as a Service), cloud computing and Big Data. The first is a business model, the second a capacity model. The last

More information

Big Data and utility function in bank services. Nikolay K. Vitanov 1

Big Data and utility function in bank services. Nikolay K. Vitanov 1 Big Data and utility function in bank services Selected aspects Nikolay K. Vitanov 1 1 Institute of Mechanics, Bulgarian Academy of Sciences Sofia, 16. 06. 2015 Vitanov (BAS) Big Data and utility function

More information

Grabbing Value from Big Data: The New Game Changer for Financial Services

Grabbing Value from Big Data: The New Game Changer for Financial Services Financial Services Grabbing Value from Big Data: The New Game Changer for Financial Services How financial services companies can harness the innovative power of big data 2 Grabbing Value from Big Data:

More information

Big Data & Analytics: Your concise guide (note the irony) Wednesday 27th November 2013

Big Data & Analytics: Your concise guide (note the irony) Wednesday 27th November 2013 Big Data & Analytics: Your concise guide (note the irony) Wednesday 27th November 2013 Housekeeping 1. Any questions coming out of today s presentation can be discussed in the bar this evening 2. OCF is

More information

Big Trouble. Does Big Data spell. for Lawyers? Presented to Colorado Bar Association, Communications & Technology Law Section Denver, Colorado

Big Trouble. Does Big Data spell. for Lawyers? Presented to Colorado Bar Association, Communications & Technology Law Section Denver, Colorado Does Big Data spell Big Trouble for Lawyers? Paul Karlzen Director HR Information & Analytics April 1, 2015 Presented to Colorado Bar Association, Communications & Technology Law Section Denver, Colorado

More information

BIG DATA: ARE YOU READY? Andy Kyiet Demand Flow Intelligence May, 2013

BIG DATA: ARE YOU READY? Andy Kyiet Demand Flow Intelligence May, 2013 BIG DATA: ARE YOU READY? Andy Kyiet Demand Flow Intelligence May, 2013 PERSONAL BACKGROUND Founder of the first specialist Service Management & Helpdesk System provider in Europe Past President of AFSMI

More information

Smarter Planet evolution

Smarter Planet evolution Smarter Planet evolution 13/03/2012 2012 IBM Corporation Ignacio Pérez González Enterprise Architect ignacio.perez@es.ibm.com @ignaciopr Mike May Technologies of the Change Capabilities Tendencies Vision

More information

IJRCS - International Journal of Research in Computer Science ISSN: 2349-3828

IJRCS - International Journal of Research in Computer Science ISSN: 2349-3828 ISSN: 2349-3828 Implementing Big Data for Intelligent Business Decisions Dr. V. B. Aggarwal Deepshikha Aggarwal 1(Jagan Institute of Management Studies, Delhi, India, vbaggarwal@jimsindia.org) 2(Jagan

More information

Big Data Analytics. Prof. Dr. Lars Schmidt-Thieme

Big Data Analytics. Prof. Dr. Lars Schmidt-Thieme Big Data Analytics Prof. Dr. Lars Schmidt-Thieme Information Systems and Machine Learning Lab (ISMLL) Institute of Computer Science University of Hildesheim, Germany 33. Sitzung des Arbeitskreises Informationstechnologie,

More information

BIG DATA & ANALYTICS. Transforming the business and driving revenue through big data and analytics

BIG DATA & ANALYTICS. Transforming the business and driving revenue through big data and analytics BIG DATA & ANALYTICS Transforming the business and driving revenue through big data and analytics Collection, storage and extraction of business value from data generated from a variety of sources are

More information

BIG DATA IN SUPPLY CHAIN MANAGEMENT: AN EXPLORATORY STUDY

BIG DATA IN SUPPLY CHAIN MANAGEMENT: AN EXPLORATORY STUDY Gheorghe MILITARU Politehnica University of Bucharest, Romania Massimo POLLIFRONI University of Turin, Italy Alexandra IOANID Politehnica University of Bucharest, Romania BIG DATA IN SUPPLY CHAIN MANAGEMENT:

More information

Big Data: Tools and Technologies in Big Data

Big Data: Tools and Technologies in Big Data Big Data: Tools and Technologies in Big Data Jaskaran Singh Student Lovely Professional University, Punjab Varun Singla Assistant Professor Lovely Professional University, Punjab ABSTRACT Big data can

More information

Hadoop for Enterprises:

Hadoop for Enterprises: Hadoop for Enterprises: Overcoming the Major Challenges Introduction to Big Data Big Data are information assets that are high volume, velocity, and variety. Big Data demands cost-effective, innovative

More information

Introduction to Engineering Using Robotics Experiments Lecture 17 Big Data

Introduction to Engineering Using Robotics Experiments Lecture 17 Big Data Introduction to Engineering Using Robotics Experiments Lecture 17 Big Data Yinong Chen 2 Big Data Big Data Technologies Cloud Computing Service and Web-Based Computing Applications Industry Control Systems

More information

We are Big Data A Sonian Whitepaper

We are Big Data A Sonian Whitepaper EXECUTIVE SUMMARY Big Data is not an uncommon term in the technology industry anymore. It s of big interest to many leading IT providers and archiving companies. But what is Big Data? While many have formed

More information

Big Data: Study in Structured and Unstructured Data

Big Data: Study in Structured and Unstructured Data Big Data: Study in Structured and Unstructured Data Motashim Rasool 1, Wasim Khan 2 mail2motashim@gmail.com, khanwasim051@gmail.com Abstract With the overlay of digital world, Information is available

More information

Generating the Business Value of Big Data:

Generating the Business Value of Big Data: Leveraging People, Processes, and Technology Generating the Business Value of Big Data: Analyzing Data to Make Better Decisions Authors: Rajesh Ramasubramanian, MBA, PMP, Program Manager, Catapult Technology

More information

Secure Data Transmission Solutions for the Management and Control of Big Data

Secure Data Transmission Solutions for the Management and Control of Big Data Secure Data Transmission Solutions for the Management and Control of Big Data Get the security and governance capabilities you need to solve Big Data challenges with Axway and CA Technologies. EXECUTIVE

More information

CIS492 Special Topics: Cloud Computing د. منذر الطزاونة

CIS492 Special Topics: Cloud Computing د. منذر الطزاونة CIS492 Special Topics: Cloud Computing د. منذر الطزاونة Big Data Definition No single standard definition Big Data is data whose scale, diversity, and complexity require new architecture, techniques, algorithms,

More information

Department of Computer Science University of Cyprus EPL646 Advanced Topics in Databases. Lecture 12

Department of Computer Science University of Cyprus EPL646 Advanced Topics in Databases. Lecture 12 Department of Computer Science University of Cyprus EPL646 Advanced Topics in Databases Lecture 12 Big Data Management II (NoSQL Databases / CouchDB) Chapter 20: Abiteboul et. Al. + http://guide.couchdb.org/

More information

Delivering new insights and value to consumer products companies through big data

Delivering new insights and value to consumer products companies through big data IBM Software White Paper Consumer Products Delivering new insights and value to consumer products companies through big data 2 Delivering new insights and value to consumer products companies through big

More information

ISSN:2321-1156 International Journal of Innovative Research in Technology & Science(IJIRTS)

ISSN:2321-1156 International Journal of Innovative Research in Technology & Science(IJIRTS) Nguyễn Thị Thúy Hoài, College of technology _ Danang University Abstract The threading development of IT has been bringing more challenges for administrators to collect, store and analyze massive amounts

More information

The New Normal: Get Ready for the Era of Extreme Information Management. John Mancini President, AIIM @jmancini77 DigitalLandfill.

The New Normal: Get Ready for the Era of Extreme Information Management. John Mancini President, AIIM @jmancini77 DigitalLandfill. The New Normal: Get Ready for the Era of Extreme Information Management John Mancini President, AIIM @jmancini77 DigitalLandfill.org Giving Credit Where Credit is Due I didn t make up the term Extreme

More information

Big Data-Challenges and Opportunities

Big Data-Challenges and Opportunities Big Data-Challenges and Opportunities White paper - August 2014 User Acceptance Tests Test Case Execution Quality Definition Test Design Test Plan Test Case Development Table of Contents Introduction 1

More information

Research Note What is Big Data?

Research Note What is Big Data? Research Note What is Big Data? By: Devin Luco Copyright 2012, ASA Institute for Risk & Innovation Keywords: Big Data, Database Management, Data Variety, Data Velocity, Data Volume, Structured Data, Unstructured

More information

A New Era Of Analytic

A New Era Of Analytic Penang egovernment Seminar 2014 A New Era Of Analytic Megat Anuar Idris Head, Project Delivery, Business Analytics & Big Data Agenda Overview of Big Data Case Studies on Big Data Big Data Technology Readiness

More information

Problems to store, transfer and process the Big Data 6/2/2016 GIANG TRAN - TTTGIANG2510@GMAIL.COM 1

Problems to store, transfer and process the Big Data 6/2/2016 GIANG TRAN - TTTGIANG2510@GMAIL.COM 1 Problems to store, transfer and process the Big Data COURSE: COMPUTING CLUSTERS, GRIDS, AND CLOUDS LECTURER: ANDREY SHEVEL ITMO UNIVERSITY SAINT PETERSBURG 6/2/2016 GIANG TRAN - TTTGIANG2510@GMAIL.COM

More information

AGENDA. What is BIG DATA? What is Hadoop? Why Microsoft? The Microsoft BIG DATA story. Our BIG DATA Roadmap. Hadoop PDW

AGENDA. What is BIG DATA? What is Hadoop? Why Microsoft? The Microsoft BIG DATA story. Our BIG DATA Roadmap. Hadoop PDW AGENDA What is BIG DATA? What is Hadoop? Why Microsoft? The Microsoft BIG DATA story Hadoop PDW Our BIG DATA Roadmap BIG DATA? Volume 59% growth in annual WW information 1.2M Zetabytes (10 21 bytes) this

More information

Big Data. White Paper. Big Data Executive Overview WP-BD-10312014-01. Jafar Shunnar & Dan Raver. Page 1 Last Updated 11-10-2014

Big Data. White Paper. Big Data Executive Overview WP-BD-10312014-01. Jafar Shunnar & Dan Raver. Page 1 Last Updated 11-10-2014 White Paper Big Data Executive Overview WP-BD-10312014-01 By Jafar Shunnar & Dan Raver Page 1 Last Updated 11-10-2014 Table of Contents Section 01 Big Data Facts Page 3-4 Section 02 What is Big Data? Page

More information

BIG DATA TRENDS AND TECHNOLOGIES

BIG DATA TRENDS AND TECHNOLOGIES BIG DATA TRENDS AND TECHNOLOGIES THE WORLD OF DATA IS CHANGING Cloud WHAT IS BIG DATA? Big data are datasets that grow so large that they become awkward to work with using onhand database management tools.

More information

BUY BIG DATA IN RETAIL

BUY BIG DATA IN RETAIL BUY BIG DATA IN RETAIL Table of contents What is Big Data?... How Data Science creates value in Retail... Best practices for Retail. Case studies... 3 7 11 1. Social listening... 2. Cross-selling... 3.

More information

The Next Wave of Data Management. Is Big Data The New Normal?

The Next Wave of Data Management. Is Big Data The New Normal? The Next Wave of Data Management Is Big Data The New Normal? Table of Contents Introduction 3 Separating Reality and Hype 3 Why Are Firms Making IT Investments In Big Data? 4 Trends In Data Management

More information

Exploiting Data at Rest and Data in Motion with a Big Data Platform

Exploiting Data at Rest and Data in Motion with a Big Data Platform Exploiting Data at Rest and Data in Motion with a Big Data Platform Sarah Brader, sarah_brader@uk.ibm.com What is Big Data? Where does it come from? 12+ TBs of tweet data every day 30 billion RFID tags

More information

The 3 questions to ask yourself about BIG DATA

The 3 questions to ask yourself about BIG DATA The 3 questions to ask yourself about BIG DATA Do you have a big data problem? Companies looking to tackle big data problems are embarking on a journey that is full of hype, buzz, confusion, and misinformation.

More information

Value from Big Data really?

Value from Big Data really? Value from Big Data really? DAMA SA Chapter Meeting: Johannesburg 24 June 2014 Let s talk about Big Data! Page 2 Is Digital Transformation really happening? 1993 2013 Page 3 But before we do that; where

More information

International Journal of Advancements in Research & Technology, Volume 3, Issue 5, May-2014 18 ISSN 2278-7763. BIG DATA: A New Technology

International Journal of Advancements in Research & Technology, Volume 3, Issue 5, May-2014 18 ISSN 2278-7763. BIG DATA: A New Technology International Journal of Advancements in Research & Technology, Volume 3, Issue 5, May-2014 18 BIG DATA: A New Technology Farah DeebaHasan Student, M.Tech.(IT) Anshul Kumar Sharma Student, M.Tech.(IT)

More information

Investigative Research on Big Data: An Analysis

Investigative Research on Big Data: An Analysis Investigative Research on Big Data: An Analysis Shyam J. Dhoble 1, Prof. Nitin Shelke 2 ME Scholar, Department of CSE, Raisoni college of Engineering and Management Amravati, Maharashtra, India. 1 Assistant

More information

Improving Data Processing Speed in Big Data Analytics Using. HDFS Method

Improving Data Processing Speed in Big Data Analytics Using. HDFS Method Improving Data Processing Speed in Big Data Analytics Using HDFS Method M.R.Sundarakumar Assistant Professor, Department Of Computer Science and Engineering, R.V College of Engineering, Bangalore, India

More information

Understanding Your Customer Journey by Extending Adobe Analytics with Big Data

Understanding Your Customer Journey by Extending Adobe Analytics with Big Data SOLUTION BRIEF Understanding Your Customer Journey by Extending Adobe Analytics with Big Data Business Challenge Today s digital marketing teams are overwhelmed by the volume and variety of customer interaction

More information

Systems of Discovery The Perfect Storm of Big Data, Cloud and Internet-of-Things

Systems of Discovery The Perfect Storm of Big Data, Cloud and Internet-of-Things Systems of Discovery The Perfect Storm of Big Data, Cloud and Internet-of-Things Mac Devine CTO, IBM Cloud Services Division IBM Distinguished Engineer wdevine@us.ibm.com twitter: mac_devine Forecast for

More information

How Big Data is Different

How Big Data is Different FALL 2012 VOL.54 NO.1 Thomas H. Davenport, Paul Barth and Randy Bean How Big Data is Different Brought to you by Please note that gray areas reflect artwork that has been intentionally removed. The substantive

More information

Keywords Big Data, NoSQL, Relational Databases, Decision Making using Big Data, Hadoop

Keywords Big Data, NoSQL, Relational Databases, Decision Making using Big Data, Hadoop Volume 4, Issue 1, January 2014 ISSN: 2277 128X International Journal of Advanced Research in Computer Science and Software Engineering Research Paper Available online at: www.ijarcsse.com Transitioning

More information

SECURITY MEETS BIG DATA. Achieve Effectiveness And Efficiency. Copyright 2012 EMC Corporation. All rights reserved.

SECURITY MEETS BIG DATA. Achieve Effectiveness And Efficiency. Copyright 2012 EMC Corporation. All rights reserved. SECURITY MEETS BIG DATA Achieve Effectiveness And Efficiency 1 IN 2010 THE DIGITAL UNIVERSE WAS 1.2 ZETTABYTES 1,000,000,000,000,000,000,000 Zetta Exa Peta Tera Giga Mega Kilo Byte Source: 2010 IDC Digital

More information

Turning Big Data into Big Decisions Delivering on the High Demand for Data

Turning Big Data into Big Decisions Delivering on the High Demand for Data Turning Big Data into Big Decisions Delivering on the High Demand for Data Michael Ho, Vice President of Professional Services Digital Government Institute s Government Big Data Conference, October 31,

More information

Statistics for BIG data

Statistics for BIG data Statistics for BIG data Statistics for Big Data: Are Statisticians Ready? Dennis Lin Department of Statistics The Pennsylvania State University John Jordan and Dennis K.J. Lin (ICSA-Bulletine 2014) Before

More information

Big Data Challenges and Success Factors. Deloitte Analytics Your data, inside out

Big Data Challenges and Success Factors. Deloitte Analytics Your data, inside out Big Data Challenges and Success Factors Deloitte Analytics Your data, inside out Big Data refers to the set of problems and subsequent technologies developed to solve them that are hard or expensive to

More information

Doing Multidisciplinary Research in Data Science

Doing Multidisciplinary Research in Data Science Doing Multidisciplinary Research in Data Science Assoc.Prof. Abzetdin ADAMOV CeDAWI - Center for Data Analytics and Web Insights Qafqaz University aadamov@qu.edu.az http://ce.qu.edu.az/~aadamov 16 May

More information

1. Understanding Big Data

1. Understanding Big Data Big Data and its Real Impact on Your Security & Privacy Framework: A Pragmatic Overview Erik Luysterborg Partner, Deloitte EMEA Data Protection & Privacy leader Prague, SCCE, March 22 nd 2016 1. 2016 Deloitte

More information

Hadoop Beyond Hype: Complex Adaptive Systems Conference Nov 16, 2012. Viswa Sharma Solutions Architect Tata Consultancy Services

Hadoop Beyond Hype: Complex Adaptive Systems Conference Nov 16, 2012. Viswa Sharma Solutions Architect Tata Consultancy Services Hadoop Beyond Hype: Complex Adaptive Systems Conference Nov 16, 2012 Viswa Sharma Solutions Architect Tata Consultancy Services 1 Agenda What is Hadoop Why Hadoop? The Net Generation is here Sizing the

More information

How to make BIG DATA work for you. Faster results with Microsoft SQL Server PDW

How to make BIG DATA work for you. Faster results with Microsoft SQL Server PDW How to make BIG DATA work for you. Faster results with Microsoft SQL Server PDW Roger Breu PDW Solution Specialist Microsoft Western Europe Marcus Gullberg PDW Partner Account Manager Microsoft Sweden

More information

BIG DATA: BIG CHALLENGE FOR SOFTWARE TESTERS

BIG DATA: BIG CHALLENGE FOR SOFTWARE TESTERS BIG DATA: BIG CHALLENGE FOR SOFTWARE TESTERS Megha Joshi Assistant Professor, ASM s Institute of Computer Studies, Pune, India Abstract: Industry is struggling to handle voluminous, complex, unstructured

More information

Volume 3, Issue 8, August 2015 International Journal of Advance Research in Computer Science and Management Studies

Volume 3, Issue 8, August 2015 International Journal of Advance Research in Computer Science and Management Studies Volume 3, Issue 8, August 2015 International Journal of Advance Research in Computer Science and Management Studies Research Article / Survey Paper / Case Study Available online at: www.ijarcsms.com An

More information

BEYOND BI: Big Data Analytic Use Cases

BEYOND BI: Big Data Analytic Use Cases BEYOND BI: Big Data Analytic Use Cases Big Data Analytics Use Cases This white paper discusses the types and characteristics of big data analytics use cases, how they differ from traditional business intelligence

More information