ESS event: Big Data in Official Statistics

Size: px
Start display at page:

Download "ESS event: Big Data in Official Statistics"

Transcription

1 ESS event: Big Data in Official Statistics v erbi v is 1

2 Parallel sessions 2A and 2B LEARNING AND DEVELOPMENT: CAPACITY BUILDING AND TRAINING FOR ESS HUMAN RESOURCES FACILITATOR: JOSÉ CERVERA- FERRI 2

3 Session 2 Related Scheveningen challenges [SCH5] Short-term Human Resources needs: recruitment, professional training, secondment/re-deployment [SCH5] Long-term needs: academic curricula for Data Scientists [SCH6] Collaboration with academia for training Data Scientists for official statistics 3

4 Session 2: Topics for discussion Skills for Big Data Opportunities for building skills Proposal for a key input to the roadmap to be established by the ESS Task Force Cross-cutting: short-term vs long-term 4

5 Session 2: Organization Short-term Long-term Skills for Big Data Opportunities for acquiring skills Proposal for a roadmap to acquire skills for Big Data in the ESS Session 2A Session 2A Session 2B Session 2B 5

6 Parallel session 2A SKILLS FOR BIG DATA OPPORTUNITIES FOR ACQUIRING SKILLS 6

7 Session 2A Preliminary considerations (1): Can NSIs rely on existing skills? Non-traditional set of skills to develop Trained statisticians and IT staff in statistics are already close to the data science skills required for Big Data (data cleaning, cubes, analytical software, data mining, etc.). Staff well-trained in methodology and statistical domains (UNECE Sprint paper, SWOT analysis strength). The Official Statistics Community has less knowledge of Big Data than many important players like Google. The Official Statistics Community has limited skills and limited IT resources when it comes to the new, nontraditional, technologies used to gather, process and analyse Big Data (UNECE Sprint paper, SWOT analysis weakness). 7

8 Session 2A Preliminary considerations (1): Can NSIs rely on existing skills? (cont.) Young staff coming in from universities may be very innovative and already have a personal relationship with Big Data (Facebook, Google, Twitter trends) and less constrained by traditional IT and analysis (UNECE Sprint paper, SWOT analysis opportunity). Failure to permit innovative methods might render OSC organizations less attractive workplaces for top talent (UNECE Sprint paper, SWOT analysis threats). Cultural change: a culture that values high quality and accurate information and regards the best way to achieve this through use of methods where the design can be controlled. Big Data doesn't allow this luxury Innovative thinking, risk-taking (is it the realm of Civil Servants??) 8

9 Session 2A Preliminary considerations (2): Learning methods Learning by doing in OS Training individuals, or teams? The business analyst and project manager The mathematician who builds algorithms The data architect The statistician (data collection, editing, processing) The communicator (visualization) Data analyst Data scientist Data engineer Data integrator System manager 9

10 Session 2A Preliminary considerations (3): Competition Competition with the Industry: better salaries in the private sector for Data Scientists? How to retain the talent? 10

11 Session 2A Skills for Big Data Data Scientist vs. Statistician Data Scientist as the connective tissue between data-processing technologies and datadriven decision making Necessary skills: math/statistics, IT, visualization, subject matter specialization Math/stat: data mining techniques IT: Hadoop, MongoDB, NoSQL, 11

12 Session 2A: IT Skills for Big Data R-SAS-SPSS Business Intelligence, Visual Analytics, Excel MapReduce Pig, Java SQL ETL (Extract, transform, load) Linux Which are the priorities? 12

13 Session 2A Statistical Skills for Big Data Computational statistics Analytical methods: correlations & causality, modelling, network analysis, information reduction Dissemination: data visualization Which are the priorities? 13

14 Session 2A Opportunities in the ESS ESS Learning and Development Framework ESTP 2014 course Big Data: Effective Processing and Analysis of Very Large and Unstructured Data for Official Statistics Contents: classification of various massive data sets, ETL (extract, transform, load), specific challenges, Privacy and statistical disclosure issues, comuting base, overview of statistical methods. Focus on concrete examples. Course requirements: Database fundamentals and data manipulation languages Data collection and integration tools Data mining techniques for large data sets Object-oriented design and programming Probablity and random variables Is there anyone with such a complete background in Official Statistics??? European Masters in Official Statistics (EMOS): ESS certification of programmes offered by Universities EMOS workshop 2014 (Helsinki, June 2014) Other methods for transfer of know-how within the ESS? 14

15 Parallel session 2B OPPORTUNITIES FOR ACQUIRING SKILLS (CONT.) KEY INPUT TO THE ROADMAP TO BE ESTABLISHED BY THE ESSTASK FORCE 15

16 Sessions 2B Opportunities outside the ESS Grasping the opportunities outside: Diversity of academic programmes on Big Data, Business Analytics, Data Science (certification?) Training offer from private companies (certification?) Opportunities within Horizon

17 Session 2B [SH6] Collaboration with Academia Academic collaborators: use of existing expertise in statistical analysis of large sets of data: astronomy, remote sensing, genetics, image processing. Source of training: need for mapping academic programmes on Big Data How can academics be integrated with NSI staff? How can training be financed? National or ESS level? 17

18 Session 2B Horizon 2020 Marie Sklodowska-Curie actions: support for innovative training networks, mobility of researchers, inter-sectoral cooperation ICT : Big data and Open Data Innovation and take-up: Objective: To contribute to capacity-building by designing and coordinating a network of European skills centres for big data analytics technologies and business development. The network is expected to identify knowledge/skills gaps in the European industrial landscape and produce effective learning curricula and documentation to train large numbers of European data analysts and business developers, capable of (co)operating across national borders on the basis of a common vision and methodology Expected impact: Availability of deployable educational material for data scientists and data workers and thousands of European data professionals trained in state-ofthe-art data analytics technologies and capable of (co)operating in cross-border, cross-lingual and cross-sector European data supply chains. Call on Training and educating Data Scientists More detailed linkages in Horizon 2020?? 18

19 Session 2B Input to the Roadmap: The actions Ideas for actions (which term?): Identify existing skills in the ESS Recruit Data Scientist with the missing skills Establish a network of providers of Big Data skills within the ESS Map the offer of Data Science training programmes in the private sector and their applicability to OS Establish a repository of assessed training materials Establish agreements with private sector and academia as providers of training, Who? NSIs, Eurostat, International organizations, private sector, Academia? Working Groups? Gexp (EMOS), HLG, ESTP,??? Which source of financing? Horizon 2020? Eurostat? National budgets? 19

20 Session 2B Input to the Roadmap: The actors Ideas for actors : NSIs Eurostat International organizations Universities Private sector 20

21 Session 2B Input to the Roadmap for Big Data training Brainstorming of ideas for building skills Assessment: sort by impact and ease of implementation Discussion of term, actors and level (national/eu/global), Proposal of responsibilities and time frame for the Input Rome Roadmap 21

ESS Big Data Event Rome 2014

ESS Big Data Event Rome 2014 ESS Big Data Event Rome 2014 Technical Workshop Report ESS Big Data Event Rome2014 Technical Event Report 2 Editor: José L. CERVERA (DevStat) Authors: José L. CERVERA and Paola VOTTA (DevStat) Donatella

More information

Modernization of European Official Statistics through Big Data methodologies and best practices: ESS Big Data Event Roma 2014

Modernization of European Official Statistics through Big Data methodologies and best practices: ESS Big Data Event Roma 2014 Modernization of European Official Statistics through Big Data methodologies and best practices: ESS Big Data Event Roma 2014 CONCEPT PAPER (DRAFT VERSION v0.3) Big Data for Official Statistics: recognition

More information

ESS event: Big Data in Official Statistics. Antonino Virgillito, Istat

ESS event: Big Data in Official Statistics. Antonino Virgillito, Istat ESS event: Big Data in Official Statistics Antonino Virgillito, Istat v erbi v is 1 About me Head of Unit Web and BI Technologies, IT Directorate of Istat Project manager and technical coordinator of Web

More information

15.00 15.30 30 XML enabled databases. Non relational databases. Guido Rotondi

15.00 15.30 30 XML enabled databases. Non relational databases. Guido Rotondi Programme of the ESTP training course on BIG DATA EFFECTIVE PROCESSING AND ANALYSIS OF VERY LARGE AND UNSTRUCTURED DATA FOR OFFICIAL STATISTICS Rome, 5 9 May 2014 Istat Piazza Indipendenza 4, Room Vanoni

More information

Integrating a Big Data Platform into Government:

Integrating a Big Data Platform into Government: Integrating a Big Data Platform into Government: Drive Better Decisions for Policy and Program Outcomes John Haddad, Senior Director Product Marketing, Informatica Digital Government Institute s Government

More information

BIG DATA & DATA SCIENCE

BIG DATA & DATA SCIENCE BIG DATA & DATA SCIENCE ACADEMY PROGRAMS IN-COMPANY TRAINING PORTFOLIO 2 TRAINING PORTFOLIO 2016 Synergic Academy Solutions BIG DATA FOR LEADING BUSINESS Big data promises a significant shift in the way

More information

W H I T E P A P E R. Deriving Intelligence from Large Data Using Hadoop and Applying Analytics. Abstract

W H I T E P A P E R. Deriving Intelligence from Large Data Using Hadoop and Applying Analytics. Abstract W H I T E P A P E R Deriving Intelligence from Large Data Using Hadoop and Applying Analytics Abstract This white paper is focused on discussing the challenges facing large scale data processing and the

More information

The use of Big Data for statistics

The use of Big Data for statistics Workshop on the use of mobile positioning data for tourism statistics Prague (CZ), 14 May 2014 The use of Big Data for statistics EUROSTAT, Unit G-3 "Short-term statistics; tourism" What is the role of

More information

Questionnaire about the skills necessary for people. working with Big Data in the Statistical Organisations

Questionnaire about the skills necessary for people. working with Big Data in the Statistical Organisations Questionnaire about the skills necessary for people working with Big Data in the Statistical Organisations Preliminary results of the survey (19.08 2014) More detailed analysis will be prepared by October

More information

big data in the European Statistical System

big data in the European Statistical System Conference by STATEC and EUROSTAT Savoir pour agir: la statistique publique au service des citoyens big data in the European Statistical System Michail SKALIOTIS EUROSTAT, Head of Task Force 'Big Data'

More information

You should have a working knowledge of the Microsoft Windows platform. A basic knowledge of programming is helpful but not required.

You should have a working knowledge of the Microsoft Windows platform. A basic knowledge of programming is helpful but not required. What is this course about? This course is an overview of Big Data tools and technologies. It establishes a strong working knowledge of the concepts, techniques, and products associated with Big Data. Attendees

More information

This Symposium brought to you by www.ttcus.com

This Symposium brought to you by www.ttcus.com This Symposium brought to you by www.ttcus.com Linkedin/Group: Technology Training Corporation @Techtrain Technology Training Corporation www.ttcus.com Big Data Analytics as a Service (BDAaaS) Big Data

More information

The Sandbox Task Team

The Sandbox Task Team United Nations Economic Commission for Europe Statistical Division Workshop on the Modernisation of Statistical Production and Services November 19-20, 2014 The Role of Big Data in the Modernisation of

More information

New Frontiers for Official Statistics

New Frontiers for Official Statistics European Data Forum 2015 November 16-17, 2015, Luxembourg New Frontiers for Official Statistics Mariana KOTZEVA EUROSTAT, Deputy Director General Key issues 1. A dynamically changing data ecosystem 2.

More information

The? Data: Introduction and Future

The? Data: Introduction and Future The? Data: Introduction and Future Husnu Sensoy Global Maksimum Data & Information Technologies Global Maksimum Data & Information Technologies The Data Company Massive Data Unstructured Data Insight Information

More information

Implement Hadoop jobs to extract business value from large and varied data sets

Implement Hadoop jobs to extract business value from large and varied data sets Hadoop Development for Big Data Solutions: Hands-On You Will Learn How To: Implement Hadoop jobs to extract business value from large and varied data sets Write, customize and deploy MapReduce jobs to

More information

International collaboration to understand the relevance of Big Data for official statistics

International collaboration to understand the relevance of Big Data for official statistics Statistical Journal of the IAOS 31 (2015) 159 163 159 DOI 10.3233/SJI-150889 IOS Press International collaboration to understand the relevance of Big Data for official statistics Steven Vale United Nations

More information

Building Your Big Data Team

Building Your Big Data Team Building Your Big Data Team With all the buzz around Big Data, many companies have decided they need some sort of Big Data initiative in place to stay current with modern data management requirements.

More information

III Big Data Technologies

III Big Data Technologies III Big Data Technologies Today, new technologies make it possible to realize value from Big Data. Big data technologies can replace highly customized, expensive legacy systems with a standard solution

More information

POSTGRAD PLACEMENTS. Placements are an integral part of the Masters programmes, so international students will not require additional work visas.

POSTGRAD PLACEMENTS. Placements are an integral part of the Masters programmes, so international students will not require additional work visas. POSTGRAD PLACEMENTS COMPUTATIONAL FINANCE DATA SCIENCE AND ANALYTICS MACHINE LEARNING KEY INFORMATION Placements can start in the middle of June 2015 or later and must finish by the middle of June 2016

More information

Bigg-Data LLC, Data Scientists Hadoop Developers/Administrators

Bigg-Data LLC, Data Scientists Hadoop Developers/Administrators Bigg-Data LLC, is a Software Solutions Technical training and resources firm. We are the first Professional Solutions Company in the country that specializes in providing big data training and resources.

More information

Advanced Big Data Analytics with R and Hadoop

Advanced Big Data Analytics with R and Hadoop REVOLUTION ANALYTICS WHITE PAPER Advanced Big Data Analytics with R and Hadoop 'Big Data' Analytics as a Competitive Advantage Big Analytics delivers competitive advantage in two ways compared to the traditional

More information

European Master in Official Statistics

European Master in Official Statistics Eurostat Luxembourg, 20 May 2014 European Master in Official Statistics At the 21 st Meeting of the European Statistical System Committee (ESSC) 1 on 14 May 2014 the ESSC agreed to following opinions:

More information

BIG DATA. Value 8/14/2014 WHAT IS BIG DATA? THE 5 V'S OF BIG DATA WHAT IS BIG DATA?

BIG DATA. Value 8/14/2014 WHAT IS BIG DATA? THE 5 V'S OF BIG DATA WHAT IS BIG DATA? WHAT IS BIG DATA? BIG DATA DR. KLARA NELSON THE UNIVERSITY OF TAMPA "Volumes of data that are unusually large, or types of data that are unstructured" Thomas Davenport, Keeping Up with the Quants, 2013,

More information

The 4 Pillars of Technosoft s Big Data Practice

The 4 Pillars of Technosoft s Big Data Practice beyond possible Big Use End-user applications Big Analytics Visualisation tools Big Analytical tools Big management systems The 4 Pillars of Technosoft s Big Practice Overview Businesses have long managed

More information

Reference Architecture, Requirements, Gaps, Roles

Reference Architecture, Requirements, Gaps, Roles Reference Architecture, Requirements, Gaps, Roles The contents of this document are an excerpt from the brainstorming document M0014. The purpose is to show how a detailed Big Data Reference Architecture

More information

Consulting and Systems Integration (1) Networks & Cloud Integration Engineer

Consulting and Systems Integration (1) Networks & Cloud Integration Engineer Ericsson is a world-leading provider of telecommunications equipment & services to mobile & fixed network operators. Over 1,000 networks in more than 180 countries use Ericsson equipment, & more than 40

More information

Manifest for Big Data Pig, Hive & Jaql

Manifest for Big Data Pig, Hive & Jaql Manifest for Big Data Pig, Hive & Jaql Ajay Chotrani, Priyanka Punjabi, Prachi Ratnani, Rupali Hande Final Year Student, Dept. of Computer Engineering, V.E.S.I.T, Mumbai, India Faculty, Computer Engineering,

More information

Hadoop for Enterprises:

Hadoop for Enterprises: Hadoop for Enterprises: Overcoming the Major Challenges Introduction to Big Data Big Data are information assets that are high volume, velocity, and variety. Big Data demands cost-effective, innovative

More information

Redesigning Data System Technology Curricula. IBM BDAEdCon 2014 Las Vegas Dr. Elena Gortcheva, Program Chair for MSc Data Systems Technology, UMUC

Redesigning Data System Technology Curricula. IBM BDAEdCon 2014 Las Vegas Dr. Elena Gortcheva, Program Chair for MSc Data Systems Technology, UMUC Redesigning Data System Technology Curricula in the Big Data World IBM BDAEdCon 2014 Las Vegas Dr. Elena Gortcheva, Program Chair for MSc Data Systems Technology, UMUC Presentation Outline Challenges Phase

More information

BIG DATA IN BUSINESS ENVIRONMENT

BIG DATA IN BUSINESS ENVIRONMENT Scientific Bulletin Economic Sciences, Volume 14/ Issue 1 BIG DATA IN BUSINESS ENVIRONMENT Logica BANICA 1, Alina HAGIU 2 1 Faculty of Economics, University of Pitesti, Romania olga.banica@upit.ro 2 Faculty

More information

Sunnie Chung. Cleveland State University

Sunnie Chung. Cleveland State University Sunnie Chung Cleveland State University Data Scientist Big Data Processing Data Mining 2 INTERSECT of Computer Scientists and Statisticians with Knowledge of Data Mining AND Big data Processing Skills:

More information

BEYOND POINT AND CLICK THE EXPANDING DEMAND FOR CODING SKILLS BURNING GLASS TECHNOLOGIES JUNE 2016

BEYOND POINT AND CLICK THE EXPANDING DEMAND FOR CODING SKILLS BURNING GLASS TECHNOLOGIES JUNE 2016 BEYOND POINT AND CLICK THE EXPANDING DEMAND FOR CODING SKILLS BURNING GLASS TECHNOLOGIES JUNE 2016 1 EXECUTIVE SUMMARY BEYOND POINT AND CLICK BEYOND POINT AND CLICK THE EXPANDING DEMAND FOR CODING SKILLS

More information

UN Global Working Group on Big Data

UN Global Working Group on Big Data UN Global Working Group on Big Data UNECE Workshop on Statistical Data Collection Washington, DC 29 April 1 May 2015 United Nations Statistics Division Nancy Snyder, Statistician, International Merchandise

More information

Big Data & Analytics @ Netflix. Paul Ellwood February 9th, 2015

Big Data & Analytics @ Netflix. Paul Ellwood February 9th, 2015 Big Data & Analytics @ Netflix Paul Ellwood February 9th, 2015 Who Am I? Director, Data Science & Engineering Also Leader, DataKind San Francisco chapter Formerly: Director, Product Analytics @ Netflix

More information

Data Science Certificate Program

Data Science Certificate Program Information Technologies Programs Data Science Certificate Program Accelerate Your Career extension.uci.edu/datascience Offered in partnership with University of California, Irvine Extension s professional

More information

Educational Opportunities in Big Data

Educational Opportunities in Big Data Educational Opportunities in Big Data Could current Big Gaps in Talent fill the void and Big Market Demand? Dr. KRS Murthy Dr.Sri.Murthy@Gmail.Com BigDataExpert@Gmail.Com (408)-464-3333 Big Gaps in Big

More information

Big Data: calling for a new scope in the curricula of Computer Science. Dr. Luis Alfonso Villa Vargas

Big Data: calling for a new scope in the curricula of Computer Science. Dr. Luis Alfonso Villa Vargas Big Data: calling for a new scope in the curricula of Computer Science Dr. Luis Alfonso Villa Vargas 23 de Abril, 2015, Puerto Vallarta, Jalisco, México Big Data: beyond my project } This talk is not about

More information

HLG - Big Data Sandbox for Statistical Production

HLG - Big Data Sandbox for Statistical Production HLG - Big Data Sandbox for Statistical Production Learning to produce meaningful statistics from big data Tatiana Yarmola (ex) Intern at the UNECE Statistical Division INEGI, December 3, 2013 Big Data:

More information

European Archival Records and Knowledge Preservation Database Archiving in the E-ARK Project

European Archival Records and Knowledge Preservation Database Archiving in the E-ARK Project European Archival Records and Knowledge Preservation Database Archiving in the E-ARK Project Janet Delve, University of Portsmouth Kuldar Aas, National Archives of Estonia Rainer Schmidt, Austrian Institute

More information

Big Data and Data Science. The globally recognised training program

Big Data and Data Science. The globally recognised training program Big Data and Data Science The globally recognised training program Certificate in Big Data Analytics Duration 5 days Big Data and Data Science enables value creation from data, through the use of calculative

More information

Machine Learning and Cloud Computing. trends, issues, solutions. EGI-InSPIRE RI-261323

Machine Learning and Cloud Computing. trends, issues, solutions. EGI-InSPIRE RI-261323 Machine Learning and Cloud Computing trends, issues, solutions Daniel Pop HOST Workshop 2012 Future plans // Tools and methods Develop software package(s)/libraries for scalable, intelligent algorithms

More information

Research trends relevant to data warehousing and OLAP include [Cuzzocrea et al.]: Combining the benefits of RDBMS and NoSQL database systems

Research trends relevant to data warehousing and OLAP include [Cuzzocrea et al.]: Combining the benefits of RDBMS and NoSQL database systems DATA WAREHOUSING RESEARCH TRENDS Research trends relevant to data warehousing and OLAP include [Cuzzocrea et al.]: Data source heterogeneity and incongruence Filtering out uncorrelated data Strongly unstructured

More information

Hadoop Beyond Hype: Complex Adaptive Systems Conference Nov 16, 2012. Viswa Sharma Solutions Architect Tata Consultancy Services

Hadoop Beyond Hype: Complex Adaptive Systems Conference Nov 16, 2012. Viswa Sharma Solutions Architect Tata Consultancy Services Hadoop Beyond Hype: Complex Adaptive Systems Conference Nov 16, 2012 Viswa Sharma Solutions Architect Tata Consultancy Services 1 Agenda What is Hadoop Why Hadoop? The Net Generation is here Sizing the

More information

CSPA. Common Statistical Production Architecture International activities on Big Data in Official Statistics. Carlo Vaccari Istat (vaccari@istat.

CSPA. Common Statistical Production Architecture International activities on Big Data in Official Statistics. Carlo Vaccari Istat (vaccari@istat. CSPA Common Statistical Production Architecture International activities on Big Data in Official Statistics Carlo Vaccari Istat (vaccari@istat.it) Data deluge Big Data definitions Data Characteristics:

More information

BIG DATA IN THE CLOUD : CHALLENGES AND OPPORTUNITIES MARY- JANE SULE & PROF. MAOZHEN LI BRUNEL UNIVERSITY, LONDON

BIG DATA IN THE CLOUD : CHALLENGES AND OPPORTUNITIES MARY- JANE SULE & PROF. MAOZHEN LI BRUNEL UNIVERSITY, LONDON BIG DATA IN THE CLOUD : CHALLENGES AND OPPORTUNITIES MARY- JANE SULE & PROF. MAOZHEN LI BRUNEL UNIVERSITY, LONDON Overview * Introduction * Multiple faces of Big Data * Challenges of Big Data * Cloud Computing

More information

Big Data Terminology - Key to Predictive Analytics Success. Mark E. Johnson Department of Statistics University of Central Florida F2: Statistics

Big Data Terminology - Key to Predictive Analytics Success. Mark E. Johnson Department of Statistics University of Central Florida F2: Statistics Big Data Terminology - Key to Predictive Analytics Success Mark E. Johnson Department of Statistics University of Central Florida F2: Statistics Outline Big Data Phenomena Terminology Role Background on

More information

Big Data Explained. An introduction to Big Data Science.

Big Data Explained. An introduction to Big Data Science. Big Data Explained An introduction to Big Data Science. 1 Presentation Agenda What is Big Data Why learn Big Data Who is it for How to start learning Big Data When to learn it Objective and Benefits of

More information

Global IDs gets big into 'big data' management

Global IDs gets big into 'big data' management Global IDs gets big into 'big data' management Analyst: Krishna Roy 29 May, 2013 Global IDs has so far largely focused on automating a range of tasks such as scanning, integrating, profiling, cleansing,

More information

FP7-ICT-2013-11-4.2. Scalable Data Analytics. Deadline: 16 April 2013 at 17:00:00 (Brussels local time)

FP7-ICT-2013-11-4.2. Scalable Data Analytics. Deadline: 16 April 2013 at 17:00:00 (Brussels local time) Scalable Data Analytics Deadline: 16 April 2013 at 17:00:00 (Brussels local time) Agenda Time 14H30 Programme Overview of Objective 4.2 Scalable Data Analytics By Carola Carstens, European Commission,

More information

SEYMOUR SLOAN IDEAS THAT MATTER

SEYMOUR SLOAN IDEAS THAT MATTER SEYMOUR SLOAN IDEAS THAT MATTER The value of Big Data: How analytics differentiate winners A DATA DRIVEN FUTURE Big data is fast becoming the term keeping senior executives up at night. The promise of

More information

BIG DATA: STORAGE, ANALYSIS AND IMPACT GEDIMINAS ŽYLIUS

BIG DATA: STORAGE, ANALYSIS AND IMPACT GEDIMINAS ŽYLIUS BIG DATA: STORAGE, ANALYSIS AND IMPACT GEDIMINAS ŽYLIUS WHAT IS BIG DATA? describes any voluminous amount of structured, semi-structured and unstructured data that has the potential to be mined for information

More information

AGENDA. What is BIG DATA? What is Hadoop? Why Microsoft? The Microsoft BIG DATA story. Our BIG DATA Roadmap. Hadoop PDW

AGENDA. What is BIG DATA? What is Hadoop? Why Microsoft? The Microsoft BIG DATA story. Our BIG DATA Roadmap. Hadoop PDW AGENDA What is BIG DATA? What is Hadoop? Why Microsoft? The Microsoft BIG DATA story Hadoop PDW Our BIG DATA Roadmap BIG DATA? Volume 59% growth in annual WW information 1.2M Zetabytes (10 21 bytes) this

More information

Getting Started Practical Input For Your Roadmap

Getting Started Practical Input For Your Roadmap Getting Started Practical Input For Your Roadmap Mike Ferguson Managing Director, Intelligent Business Strategies BA4ALL Big Data & Analytics Insight Conference Stockholm, May 2015 About Mike Ferguson

More information

Transforming the Telecoms Business using Big Data and Analytics

Transforming the Telecoms Business using Big Data and Analytics Transforming the Telecoms Business using Big Data and Analytics Event: ICT Forum for HR Professionals Venue: Meikles Hotel, Harare, Zimbabwe Date: 19 th 21 st August 2015 AFRALTI 1 Objectives Describe

More information

Collaborations between Official Statistics and Academia in the Era of Big Data

Collaborations between Official Statistics and Academia in the Era of Big Data Collaborations between Official Statistics and Academia in the Era of Big Data World Statistics Day October 20-21, 2015 Budapest Vijay Nair University of Michigan Past-President of ISI vnn@umich.edu What

More information

COMP9321 Web Application Engineering

COMP9321 Web Application Engineering COMP9321 Web Application Engineering Semester 2, 2015 Dr. Amin Beheshti Service Oriented Computing Group, CSE, UNSW Australia Week 11 (Part II) http://webapps.cse.unsw.edu.au/webcms2/course/index.php?cid=2411

More information

22 nd Meeting of the European Statistical System Committee

22 nd Meeting of the European Statistical System Committee 22 nd Meeting of the European Statistical System Committee Riga (Latvia), 26 September 2014 Item 8 of the agenda ESS Big Data Action Plan and Roadmap 1.0 Work Programme Objective 11.1 Eurostat Big Data

More information

Annex: Concept Note. Big Data for Policy, Development and Official Statistics New York, 22 February 2013

Annex: Concept Note. Big Data for Policy, Development and Official Statistics New York, 22 February 2013 Annex: Concept Note Friday Seminar on Emerging Issues Big Data for Policy, Development and Official Statistics New York, 22 February 2013 How is Big Data different from just very large databases? 1 Traditionally,

More information

INTRODUCTION TO APACHE HADOOP MATTHIAS BRÄGER CERN GS-ASE

INTRODUCTION TO APACHE HADOOP MATTHIAS BRÄGER CERN GS-ASE INTRODUCTION TO APACHE HADOOP MATTHIAS BRÄGER CERN GS-ASE AGENDA Introduction to Big Data Introduction to Hadoop HDFS file system Map/Reduce framework Hadoop utilities Summary BIG DATA FACTS In what timeframe

More information

ONS Big Data Project Progress report: Qtr 1 Jan to Mar 2014

ONS Big Data Project Progress report: Qtr 1 Jan to Mar 2014 Official ONS Big Data Project Qtr 1 Report May 2014 ONS Big Data Project Progress report: Qtr 1 Jan to Mar 2014 Jane Naylor, Nigel Swier, Susan Williams Office for National Statistics Background The amount

More information

BIG DATA AND ANALYTICS

BIG DATA AND ANALYTICS BIG DATA AND ANALYTICS Björn Bjurling, bgb@sics.se Daniel Gillblad, dgi@sics.se Anders Holst, aho@sics.se Swedish Institute of Computer Science AGENDA What is big data and analytics? and why one must bother

More information

Big Data Executive Survey

Big Data Executive Survey Big Data Executive Full Questionnaire Big Date Executive Full Questionnaire Appendix B Questionnaire Welcome The survey has been designed to provide a benchmark for enterprises seeking to understand the

More information

Strategies For Setting Up Your Organisation For Success With Big Data. Kevin Long Business Development Director Teradata

Strategies For Setting Up Your Organisation For Success With Big Data. Kevin Long Business Development Director Teradata Strategies For Setting Up Your Organisation For Success With Big Data Kevin Long Business Development Director Teradata Agenda Developing a big data strategy and plan that is aligned with your organisation

More information

Some Economics of Cultural PSI: the Micro Perspective

Some Economics of Cultural PSI: the Micro Perspective Some Economics of Cultural PSI: the Micro Perspective Massimiliano Nuccio Research Affiliate ASK Bocconi Research Centre Bocconi University Milan - 10 October 2014 1 Agenda Which new sources of data can

More information

Data Mining in the Swamp

Data Mining in the Swamp WHITE PAPER Page 1 of 8 Data Mining in the Swamp Taming Unruly Data with Cloud Computing By John Brothers Business Intelligence is all about making better decisions from the data you have. However, all

More information

PREDICTIVE MARKETING, DIGITAL ATTRIBUTION, OPTIMIZATION, AND DATA-DRIVEN PERSONALIZATION

PREDICTIVE MARKETING, DIGITAL ATTRIBUTION, OPTIMIZATION, AND DATA-DRIVEN PERSONALIZATION PREDICTIVE MARKETING, DIGITAL ATTRIBUTION, OPTIMIZATION, AND DATA-DRIVEN PERSONALIZATION A m a r t y a B h a t t a c h a r j y & S u n e e l G r o v e r P r i n c i p a l S o l u t i o n A r c h i t e

More information

An interdisciplinary model for analytics education

An interdisciplinary model for analytics education An interdisciplinary model for analytics education Raffaella Settimi, PhD School of Computing, DePaul University Drew Conway s Data Science Venn Diagram http://drewconway.com/zia/2013/3/26/the-data-science-venn-diagram

More information

Buyer s Guide to Big Data Integration

Buyer s Guide to Big Data Integration SEPTEMBER 2013 Buyer s Guide to Big Data Integration Sponsored by Contents Introduction 1 Challenges of Big Data Integration: New and Old 1 What You Need for Big Data Integration 3 Preferred Technology

More information

BIG DATA TOOLS. Top 10 open source technologies for Big Data

BIG DATA TOOLS. Top 10 open source technologies for Big Data BIG DATA TOOLS Top 10 open source technologies for Big Data We are in an ever expanding marketplace!!! With shorter product lifecycles, evolving customer behavior and an economy that travels at the speed

More information

Big Data Challenges and Success Factors. Deloitte Analytics Your data, inside out

Big Data Challenges and Success Factors. Deloitte Analytics Your data, inside out Big Data Challenges and Success Factors Deloitte Analytics Your data, inside out Big Data refers to the set of problems and subsequent technologies developed to solve them that are hard or expensive to

More information

What is Data Science? Girl Develop It! Meetup Renée M. P. Teate, March 2015

What is Data Science? Girl Develop It! Meetup Renée M. P. Teate, March 2015 What is Data Science? { Girl Develop It! Meetup Renée M. P. Teate, March 2015 Let s start with: What is Data? http://upload.wikimedia.org/wikipedia/commons/f/f0/darpa _Big_Data.jpg https://encryptedtbn2.gstatic.com/images?q=tbn:and9gcs9dku3_tzi-swwyaqee5y0ehuvoiznsya_raknubbd0jyxpx7pw

More information

Big Data Specialized Studies

Big Data Specialized Studies Information Technologies Programs Big Data Specialized Studies Accelerate Your Career extension.uci.edu/bigdata Offered in partnership with University of California, Irvine Extension s professional certificate

More information

Introduction to Big Data! with Apache Spark" UC#BERKELEY#

Introduction to Big Data! with Apache Spark UC#BERKELEY# Introduction to Big Data! with Apache Spark" UC#BERKELEY# So What is Data Science?" Doing Data Science" Data Preparation" Roles" This Lecture" What is Data Science?" Data Science aims to derive knowledge!

More information

Big data for official statistics

Big data for official statistics Big data for official statistics Strategies and some initial European applications Martin Karlberg and Michail Skaliotis, Eurostat 27 September 2013 Seminar on Statistical Data Collection WP 30 1 Big Data

More information

USING BIG DATA FOR INTELLIGENT BUSINESSES

USING BIG DATA FOR INTELLIGENT BUSINESSES HENRI COANDA AIR FORCE ACADEMY ROMANIA INTERNATIONAL CONFERENCE of SCIENTIFIC PAPER AFASES 2015 Brasov, 28-30 May 2015 GENERAL M.R. STEFANIK ARMED FORCES ACADEMY SLOVAK REPUBLIC USING BIG DATA FOR INTELLIGENT

More information

Big Data and Analytics: Challenges and Opportunities

Big Data and Analytics: Challenges and Opportunities Big Data and Analytics: Challenges and Opportunities Dr. Amin Beheshti Lecturer and Senior Research Associate University of New South Wales, Australia (Service Oriented Computing Group, CSE) Talk: Sharif

More information

Big Data (Adv. Analytics) in 15 Mins. Peter LePine Managing Director Sales Support IM & BI Practice

Big Data (Adv. Analytics) in 15 Mins. Peter LePine Managing Director Sales Support IM & BI Practice Big Data (Adv. Analytics) in 15 Mins. Peter LePine Managing Director Sales Support IM & BI Practice Agenda Big Data in 15 Mins. Goal: Provide a basic understanding of; What is Big Data; Why it s important

More information

Challenges of Analytics

Challenges of Analytics Challenges of Analytics Setting-up a Data Science Team BA4ALL Eindhoven November 2015 Laurent FAYET CEO @lbfayet www.artycs.eu 1 Agenda 1 About ARTYCS 2 Definitions 3 Data Value Creation 4 An Approach

More information

Big Data and Data Science: Behind the Buzz Words

Big Data and Data Science: Behind the Buzz Words Big Data and Data Science: Behind the Buzz Words Peggy Brinkmann, FCAS, MAAA Actuary Milliman, Inc. April 1, 2014 Contents Big data: from hype to value Deconstructing data science Managing big data Analyzing

More information

How to use Big Data in Industry 4.0 implementations. LAURI ILISON, PhD Head of Big Data and Machine Learning

How to use Big Data in Industry 4.0 implementations. LAURI ILISON, PhD Head of Big Data and Machine Learning How to use Big Data in Industry 4.0 implementations LAURI ILISON, PhD Head of Big Data and Machine Learning Big Data definition? Big Data is about structured vs unstructured data Big Data is about Volume

More information

BIG DATA ANALYTICS REFERENCE ARCHITECTURES AND CASE STUDIES

BIG DATA ANALYTICS REFERENCE ARCHITECTURES AND CASE STUDIES BIG DATA ANALYTICS REFERENCE ARCHITECTURES AND CASE STUDIES Relational vs. Non-Relational Architecture Relational Non-Relational Rational Predictable Traditional Agile Flexible Modern 2 Agenda Big Data

More information

NOS for Data Analysis (802) September 2014 V1.3

NOS for Data Analysis (802) September 2014 V1.3 NOS for Data Analysis (802) September 2014 V1.3 NOS Reference ESKITP802301 ESKITP802401 ESKITP802501 ESKITP802601 NOS Title Assist in Delivering Routine Data Analysis Studies Design and Implement Data

More information

The Data Engineer. Mike Tamir Chief Science Officer Galvanize. Steven Miller Global Leader Academic Programs IBM Analytics

The Data Engineer. Mike Tamir Chief Science Officer Galvanize. Steven Miller Global Leader Academic Programs IBM Analytics The Data Engineer Mike Tamir Chief Science Officer Galvanize Steven Miller Global Leader Academic Programs IBM Analytics Alessandro Gagliardi Lead Faculty Galvanize Businesses are quickly realizing that

More information

With the Emergence of Big Data, Where do Relational Technologies Fit? Donna Burbank President, DAMA Rocky Mountain Chapter

With the Emergence of Big Data, Where do Relational Technologies Fit? Donna Burbank President, DAMA Rocky Mountain Chapter With the Emergence of Big Data, Where do Relational Technologies Fit? Donna Burbank President, DAMA Rocky Mountain Chapter Agenda Big Data A Technical & Cultural Paradigm Shift (aka Donna s Rants/Musings)

More information

Building and Managing Analytics Teams

Building and Managing Analytics Teams Building and Managing Analytics Teams Stanford SC Forum Roundtable Event Creating Business Value with Analytics and Big Data Thomas Olavson Director, Operations Decision Support Google July 20, 2011 My

More information

Oracle Big Data Handbook

Oracle Big Data Handbook ORACLG Oracle Press Oracle Big Data Handbook Tom Plunkett Brian Macdonald Bruce Nelson Helen Sun Khader Mohiuddin Debra L. Harding David Segleau Gokula Mishra Mark F. Hornick Robert Stackowiak Keith Laker

More information

Big Data Systems and Interoperability

Big Data Systems and Interoperability Big Data Systems and Interoperability Emerging Standards for Systems Engineering David Boyd VP, Data Solutions Email: dboyd@incadencecorp.com Topics Shameless plugs and denials What is Big Data and Why

More information

Big data for the Masses The Unique Challenge of Big Data Integration

Big data for the Masses The Unique Challenge of Big Data Integration Big data for the Masses The Unique Challenge of Big Data Integration White Paper Table of contents Executive Summary... 4 1. Big Data: a Big Term... 4 1.1. The Big Data... 4 1.2. The Big Technology...

More information

Securing NoSQL Clusters

Securing NoSQL Clusters Presents Securing NoSQL Clusters Adrian Lane, CTO alane@securosis.com Twitter: @AdrianLane David Mortman dmortman@securosis.com Twitter: @ Independent analysts with backgrounds on both the user and vendor

More information

WHITE PAPER. Four Key Pillars To A Big Data Management Solution

WHITE PAPER. Four Key Pillars To A Big Data Management Solution WHITE PAPER Four Key Pillars To A Big Data Management Solution EXECUTIVE SUMMARY... 4 1. Big Data: a Big Term... 4 EVOLVING BIG DATA USE CASES... 7 Recommendation Engines... 7 Marketing Campaign Analysis...

More information

Data Science and Business Analytics Certificate Data Science and Business Intelligence Certificate

Data Science and Business Analytics Certificate Data Science and Business Intelligence Certificate Data Science and Business Analytics Certificate Data Science and Business Intelligence Certificate Description The Helzberg School of Management has launched two graduate-level certificates: one in Data

More information

Masters programmes in Big Data

Masters programmes in Big Data Computer Science Masters programmes in Big Data Computational Finance (with a Year in Industry) Data Science and Analytics (with a Year in Industry) Machine Learning (with a Year in Industry) Contents

More information

BIG DATA What it is and how to use?

BIG DATA What it is and how to use? BIG DATA What it is and how to use? Lauri Ilison, PhD Data Scientist 21.11.2014 Big Data definition? There is no clear definition for BIG DATA BIG DATA is more of a concept than precise term 1 21.11.14

More information

AppSymphony White Paper

AppSymphony White Paper AppSymphony White Paper Secure Self-Service Analytics for Curated Digital Collections Introduction Optensity, Inc. offers a self-service analytic app composition platform, AppSymphony, which enables data

More information

The value proposition

The value proposition The value proposition of the Official Statistics Research Community in the field of Big Data Martin Karlberg, Eurostat 1 The Horizon 2020 (H2020) research framework programme The biggest EU Research and

More information

Native Connectivity to Big Data Sources in MicroStrategy 10. Presented by: Raja Ganapathy

Native Connectivity to Big Data Sources in MicroStrategy 10. Presented by: Raja Ganapathy Native Connectivity to Big Data Sources in MicroStrategy 10 Presented by: Raja Ganapathy Agenda MicroStrategy supports several data sources, including Hadoop Why Hadoop? How does MicroStrategy Analytics

More information

Towards the Data-Driven Economy. Gabriella Cattaneo, IDC European Government Consulting European Data Forum 2015, Luxembourg, 17 November 2015

Towards the Data-Driven Economy. Gabriella Cattaneo, IDC European Government Consulting European Data Forum 2015, Luxembourg, 17 November 2015 Towards the Data-Driven Economy Gabriella Cattaneo, IDC European Government Consulting European Data Forum 2015, Luxembourg, 17 November 2015 The European Data Market Monitoring study on behalf of DG CONNECT

More information

BITKOM& NIK - Big Data Wo liegen die Chancen für den Mittelstand?

BITKOM& NIK - Big Data Wo liegen die Chancen für den Mittelstand? BITKOM& NIK - Big Data Wo liegen die Chancen für den Mittelstand? The Big Data Buzz big data is a collection of data sets so large and complex that it becomes difficult to process using on-hand database

More information

IBM: An Early Leader across the Big Data Security Analytics Continuum Date: June 2013 Author: Jon Oltsik, Senior Principal Analyst

IBM: An Early Leader across the Big Data Security Analytics Continuum Date: June 2013 Author: Jon Oltsik, Senior Principal Analyst ESG Brief IBM: An Early Leader across the Big Data Security Analytics Continuum Date: June 2013 Author: Jon Oltsik, Senior Principal Analyst Abstract: Many enterprise organizations claim that they already

More information