Big Data in the context of Preservation and Value Adding

Size: px
Start display at page:

Download "Big Data in the context of Preservation and Value Adding"

Transcription

1 Big Data in the context of Preservation and Value Adding R. Leone, R. Cosac, I. Maggio, D. Iozzino ESRIN 06/11/2013 ESA UNCLASSIFIED

2 Big Data Background ESA/ESRIN organized a 'Big Data from Space' event on 5-7 June 2013 in order to address the barriers that hamper an effective use of large volumes of Earth Observation (EO) data. This event aimed to stimulate discussion between the different communities involved in the business of providing and manipulating EO very large-scale data and complex analyses. The meeting involved some 250 science, industry and policymaking representatives and national delegates from Europe, the US, Australia, China and Africa. Over 50 presentations during the three-day conference stimulated discussion between the different communities in the business of providing and manipulating very large-scale data and complex analyses of satellite and in situ Earth observations

3 Big Data Event Günther Kohlhammer H/EO Ground Segment and Missions Operations Department Big Data from Space (Big?) Data and Earth Observation Kostas Glinos Head of Unit - e-infrastructure DG CONNECT European Commission e-infrastructures for big data Reinhard Schulte-Braucks Head GMES Unit, DG ENTR, EC Copernicus and Big Data Gilles Ollier Head of Sector Earth observation Directorate General Research & Innovation EUROPEAN COMMISSION Earth Observation data and the EC Environmental Research and Innovation program

4 Big Data Technology Mosaic

5 Big Data Agencies and Research Institution

6 Big Data Industry

7 Big Data Background The event covered diverse aspects of handling large-scale data and complex analysis of Earth observation data products, including: Typical order of data volumes involved and their trends, primarily with respect to the utilization of streaming of data from presently available and upcoming satellite capabilities, and from ubiquitous ground devices. Challenges of data access, including timeliness, needs and policies for their dissemination, data capture, storage, search, sharing (including use of interoperability standards), transfer capacity, mining and analysis (including identification of representative samples), fusion, systematic and peak processing and visualization. The cost and weighing factors against identified challenges and in support of continuous evolution of techniques and technologies, in the short and longterm.

8 Big Data Topics Event papers were mostly, but not exclusively, devoted to the following topics: applied multivariate analysis, data mining computing power and storage scalability costs and weighting factors data access and use policies, licensing of derivative work data capturing and description data interoperability, retrieval, navigation data protection, and trustworthiness data delivery timeliness, distribution services, network capacity data slicing, sub-setting, extraction data variety, fusion, correlation data visualization, rendering, video streaming peak data processing performance indicators for big Earth data services systematic data processing spatial on-line analytical processing systems sustainability of big Earth data services.

9 Big Data and The Fourth V Paradigm Big Data consist of: Volume size of the data is increasing fast Value amount of value that can be derived from the data (through innovative analysis techniques, through combined use of diverse EO data and long term data series analysis (old data gaining value from new ones) Variety diversity and complexity of the data (format, type and storage medium) Velocity data is arriving at a faster rate and technology is always advancing

10 Big Data Current Challenges At this point in time main challenge is not only the volume of data, but its diversity (in term of data product content, format and type). Older EO data is recorded on various media, in different formats. A huge task represents the recovery, reformatting, reprocessing of such data, as well as the transcription of various associated information which is necessary to understand and use the data. Challenges include capture, curation, storage, search, sharing, transfer, analysis, and visualization. A large proportion of users are not domain experts anymore data discovery tools, documentation and support are needed

11 Big Data Solutions Digital storytelling to open up to a larger audience and ease the understanding of users which may not be domain experts. Improved algorithms to lower processing time. Cloud services to support the Big data and ease accessibility EO Platforms where data can be easily accessed, shared and manipulated (e.g. Google Earth Engine, Nephelae). Cloud services and platforms could increase revisiting rate and enhancement of EO data archives.

12 Big Data Presentations Examples From Google Google Earth Engine: A Global Scale Geospatial Analysis Platform NOAA Big Data for a Big Ocean ECMWF Experience with managing a Multi-Petabyte Meteorological Archive DLR (GeoFarm) Enabling EO Data Exploitation CloudEO An Open Cloud Based EO-Services Production Platform and Marketplace AVHRR TIMELINE Project.. To Spacemetric- ESA LDCM repository, data processing and dissemination

13 Big Data Future Challenges Are we prepared to satisfy increasing user communities requirements (from policy maker to scientist?) Are we prepared to accommodate and process "Big data" EO data? Are we ready to share knowledge and tools on collaborative platforms? What are the security issues with regards to cloud services? Can we trust the cloud? How can we protect sensitive data? One idea could be to break Big data into pieces. While this could make it more secure, how will it affect the data processing and analysis times?

14 Big Data Conclusions and Recommendations The Big Data from Space event has set a new paradigm and a more advanced perspective to Big Data issues. The event ended with a strong call by all parties for the ability to handle and use big EO data. This could potentially open new opportunities for research and international cooperation schemes such as programmatic and industrial coordination. There was also unanimous support to promote the development of processing capabilities, and making data more accessible to users, complementing more traditional web service approaches. The excellent feedback and contributions received during the event paved the way to ESA for managing future EO data and form the basis for discussion among Earth observation data owners and suppliers.

15 Big Data Conclusions and Recommendations Scientists regularly encounter limitations due to large data sets in many areas. One of the possible ways to solve this issue would be that of bringing data processing directly to the data collection devices and/or facilities. This way the dataset to be archived, maybe long-term, would be orders of magnitudes smaller than the actual data amount collected by the sensors, by storing only the meaningful information extracted from the huge data collection. Big Data can also mean big changes to storage infrastructure, or working smarter with the available ones (object storage systems vs. cloud). Big Data activities boost private business entrepreneurial efforts, also at a huge level of infrastructure and investments (see, e.g., the papers by Google, Microsoft representatives). A new professional figure is emerging, namely that of the Data Scientist. This is a new type of specialist, with a solid foundation typically in computer science and applications, but as well as in modeling, statistics, analytics and math. A Data Scientist is a practitioner of data science. He is able to extract the meaningful information from the data deluge.

16 Big Data versus Long Term Data Preservation and Value The event addressed indirectly Long Term Data Preservation and Value issues,. Very large data sets data handling, their curation, valorization, retrieval, manipulation and finally visualization All issues will bring a contribution to the solution of problems always arising when dealing with such large data sets such as those considered when carrying out LTDP activities. One of the most relevant points was a new way of carrying out scientific research. Increasingly, scientific breakthroughs will be powered by advanced computing capabilities that help researchers manipulate and explore massive datasets. After experimental, theoretical, and computational science, a Fourth Paradigm, emerging in scientific research, refers to the data management techniques and the computational system needed to manipulate, visualize, and manage those large amounts of scientific data.

17 Next Steps from LTDP to LTDP4V Valorize the past being inspired by the future 1. Foster interactions with user communities to gather requirements and collect feedback on data need and discoverability requirements 2. Review LTDP operations concept and scenarios to accommodate the 4V paradigm 3. Reinforce cooperation and federation with all involved parties (data producer, archive and consumer) 4. Identify technological areas for innovation for data exploitation 5. Improve communication to strengthen the value of data preservation, understandability and usage 6. Create center of excellence for thematic sensors applications and Fundamental Data Records exploitation

18

Summary of 1 st ESA symposium on big Earth observing data

Summary of 1 st ESA symposium on big Earth observing data Summary of 1 st ESA symposium on big Earth observing data Veronica Guidetti European Space Agency Copernicus Big Data Workshop European Commission Brussels 13-14 Mar 2014 NOT IN SCOPE OF MY TODAY S TALK

More information

How To Work With Big Data From Space In Europe

How To Work With Big Data From Space In Europe Industry & SMEs Round Table 2014 Conference on Big Data from Space (BiDS '14) Dr. Florin Serban Dr. Catalin Cucu-Dumitrescu 12-14 November 2014, ESRIN, Frascati, Italy Main aspects to be presented: Status

More information

A Future Scenario of interconnected EO Platforms How will EO data be used in 2025?

A Future Scenario of interconnected EO Platforms How will EO data be used in 2025? A Future Scenario of interconnected EO Platforms How will EO data be used in 2025? ESA UNCLASSIFIED For Official Use European EO data asset Heritage missions Heritage Core GS (data preservation, curation

More information

ESA Earth Observation Big Data R&D Past, Present, & Future Activities

ESA Earth Observation Big Data R&D Past, Present, & Future Activities ESA Earth Observation Big Data R&D Past, Present, & Future Activities [Sveinung.Loekken, Jordi.Farres]@esa.int Ground Segment and Mission Operations Department, Earth Observation Programmes Directorate,

More information

This document provides the report of the BIG DATA FROM SPACE event, held at ESA- ESRIN on 5-7 June 2013.

This document provides the report of the BIG DATA FROM SPACE event, held at ESA- ESRIN on 5-7 June 2013. 1 INTRODUCTION Big Earth observing data can be defined in terms of volumes, their degree of diversity and complexity - including streaming of data from presently available and upcoming satellite capabilities,

More information

Long Term Preservation of Earth Observation Data

Long Term Preservation of Earth Observation Data Long Term Preservation of Earth Observation Data QA4EO Workshop RAL, October 18-20 th 2011 Mirko Albani and Bojan Bojkov* (ESA/ESRIN) Page 1 Outline Earth Observation data preservation: the need and the

More information

Long Term Preservation of Earth Observation Space Data. Preservation Workflow

Long Term Preservation of Earth Observation Space Data. Preservation Workflow Long Term Preservation of Earth Observation Space Data Preservation Workflow CEOS-WGISS Doc. Ref.: CEOS/WGISS/DSIG/PW Data Stewardship Interest Group Date: March 2015 Issue: Version 1.0 Preservation Workflow

More information

Helix Nebula, the Science Cloud

Helix Nebula, the Science Cloud Helix Nebula, the Science Cloud e-irg strategy workshop 11 & 12 June 2012 Maryline Lengert, ESA From Requirement Collection to Strategic Plan & Proof of Concept End 2010: ESA started collecting Cloud Computing

More information

Copernicus Space Component ESA Data Access Overview J. Martin (ESA), R. Knowelden (Airbus D&S)

Copernicus Space Component ESA Data Access Overview J. Martin (ESA), R. Knowelden (Airbus D&S) Copernicus Space Component ESA Data Access Overview J. Martin (ESA), R. Knowelden (Airbus D&S) Introduction and Context Operational Scenarios Translation into interfaces Translation into services Current

More information

How To Use Data From Copernicus And Big Data To Help The Environment

How To Use Data From Copernicus And Big Data To Help The Environment Copernicus and Big Data: Challenges and Opportunities Alessandro Annoni European Commission Joint Research Centre www.jrc.ec.europa.eu Serving society Stimulating innovation Supporting legislation Big

More information

Kimmo Rossi. European Commission DG CONNECT

Kimmo Rossi. European Commission DG CONNECT Kimmo Rossi European Commission DG CONNECT Unit G.3 - Data Value Chain SC1 info day, Brussels 5/12/2014 1 What we do Unit CNECT.G3 Data Value Chain FP7/CIP/H2020 project portfolio: Big Data, analytics,

More information

Global Scientific Data Infrastructures: The Big Data Challenges. Capri, 12 13 May, 2011

Global Scientific Data Infrastructures: The Big Data Challenges. Capri, 12 13 May, 2011 Global Scientific Data Infrastructures: The Big Data Challenges Capri, 12 13 May, 2011 Data-Intensive Science Science is, currently, facing from a hundred to a thousand-fold increase in volumes of data

More information

Mission Operations and Ground Segment

Mission Operations and Ground Segment ESA Earth Observation Info Days Mission Operations and Ground Segment ESA EO Ground Segment and Mission Operations department (EOP-G) May 2013 EOEP 2013 Page 1 ESA Unclassified For Official Use MISSION

More information

Cloud Computing and Content Delivery Network use within Earth Observation Ground Segments: experiences and lessons learnt

Cloud Computing and Content Delivery Network use within Earth Observation Ground Segments: experiences and lessons learnt Cloud Computing and Content Delivery Network use within Earth Observation Ground Segments: experiences and lessons learnt J.Farres EOP-GS ESRIN 6/6/2012 Page 1 Agenda 1. Introduction 2. ESA Experiences

More information

Information Visualization WS 2013/14 11 Visual Analytics

Information Visualization WS 2013/14 11 Visual Analytics 1 11.1 Definitions and Motivation Lot of research and papers in this emerging field: Visual Analytics: Scope and Challenges of Keim et al. Illuminating the path of Thomas and Cook 2 11.1 Definitions and

More information

ICT Perspectives on Big Data: Well Sorted Materials

ICT Perspectives on Big Data: Well Sorted Materials ICT Perspectives on Big Data: Well Sorted Materials 3 March 2015 Contents Introduction 1 Dendrogram 2 Tree Map 3 Heat Map 4 Raw Group Data 5 For an online, interactive version of the visualisations in

More information

EGI services for distribution and federation of data and computing

EGI services for distribution and federation of data and computing EGI services for distribution and federation of data and computing Tiziana Ferrari Technical Director, EGI.eu tiziana.ferrari@egi.eu March 2014 EGI-InSPIRE RI-261323 1 Accelerating Excellent Science MISSION.

More information

EARSC Views on the. Procurement of the Copernicus Services

EARSC Views on the. Procurement of the Copernicus Services EARSC Views on the Procurement of the Copernicus Services EARSC, the European Association of Remote Sensing Companies represents the Earth Observation geoinformation services sector in Europe. Today EARSC

More information

Government Technology Trends to Watch in 2014: Big Data

Government Technology Trends to Watch in 2014: Big Data Government Technology Trends to Watch in 2014: Big Data OVERVIEW The federal government manages a wide variety of civilian, defense and intelligence programs and services, which both produce and require

More information

Standards for Big Data in the Cloud

Standards for Big Data in the Cloud Standards for Big Data in the Cloud International Cloud Symposium 15/10/2013 Carola Carstens (Project Officer) DG CONNECT, Unit G3 Data Value Chain European Commission Outline 1) Data Value Chain Unit

More information

可 视 化 与 可 视 计 算 概 论. Introduction to Visualization and Visual Computing 袁 晓 如 北 京 大 学 2015.12.23

可 视 化 与 可 视 计 算 概 论. Introduction to Visualization and Visual Computing 袁 晓 如 北 京 大 学 2015.12.23 可 视 化 与 可 视 计 算 概 论 Introduction to Visualization and Visual Computing 袁 晓 如 北 京 大 学 2015.12.23 2 Visual Analytics Adapted from Jim Thomas s slides 3 Visual Analytics Definition Visual Analytics is the

More information

The distribution of marine OpenData via distributed data networks and Web APIs. The example of ERDDAP, the message broker and data mediator from NOAA

The distribution of marine OpenData via distributed data networks and Web APIs. The example of ERDDAP, the message broker and data mediator from NOAA The distribution of marine OpenData via distributed data networks and Web APIs. The example of ERDDAP, the message broker and data mediator from NOAA Dr. Conor Delaney 9 April 2014 GeoMaritime, London

More information

Workprogramme 2014-15

Workprogramme 2014-15 Workprogramme 2014-15 e-infrastructures DCH-RP final conference 22 September 2014 Wim Jansen einfrastructure DG CONNECT European Commission DEVELOPMENT AND DEPLOYMENT OF E-INFRASTRUCTURES AND SERVICES

More information

Copernicus Space Component Data Access Architecture. Meeting with Austria 27 May 2014, Vienna

Copernicus Space Component Data Access Architecture. Meeting with Austria 27 May 2014, Vienna Copernicus Space Component Data Access Architecture Meeting with Austria 27, Vienna Copernicus Data Policy Users shall have free, full and open access to Copernicus dedicated Sentinel data and Copernicus

More information

Big Data and Cloud Computing for GHRSST

Big Data and Cloud Computing for GHRSST Big Data and Cloud Computing for GHRSST Jean-Francois Piollé (jfpiolle@ifremer.fr) Frédéric Paul, Olivier Archer CERSAT / Institut Français de Recherche pour l Exploitation de la Mer Facing data deluge

More information

Overview of state of art in Data management. Stefano Cozzini CNR/IOM and exact lab srl

Overview of state of art in Data management. Stefano Cozzini CNR/IOM and exact lab srl Overview of state of art in Data management Stefano Cozzini CNR/IOM and exact lab srl AIM of this short talk Frame the problem and the discussion around DATA: What are big data? Which kind of challenges

More information

Helix Nebula, the Science Cloud: Potential for Earth Science Franco-British Workshop on Big Data in Science 6-7 November 2012

Helix Nebula, the Science Cloud: Potential for Earth Science Franco-British Workshop on Big Data in Science 6-7 November 2012 Helix Nebula, the Science Cloud: Potential for Earth Science 6-7 November 2012 Strategic Goal Helix Nebula, the Science Cloud is a partnership that has been created to support the massive IT requirements

More information

locuz.com Big Data Services

locuz.com Big Data Services locuz.com Big Data Services Big Data At Locuz, we help the enterprise move from being a data-limited to a data-driven one, thereby enabling smarter, faster decisions that result in better business outcome.

More information

EO data hosting and processing core capabilities and emerging solutions

EO data hosting and processing core capabilities and emerging solutions EO data hosting and processing core capabilities and emerging solutions Andrew Groom 4 th March 2015 Contents An introduction to Airbus Defence and Space, Geo-Intelligence Elements of the C3S vision EO

More information

Collaborations between Official Statistics and Academia in the Era of Big Data

Collaborations between Official Statistics and Academia in the Era of Big Data Collaborations between Official Statistics and Academia in the Era of Big Data World Statistics Day October 20-21, 2015 Budapest Vijay Nair University of Michigan Past-President of ISI vnn@umich.edu What

More information

White Paper. Version 1.2 May 2015 RAID Incorporated

White Paper. Version 1.2 May 2015 RAID Incorporated White Paper Version 1.2 May 2015 RAID Incorporated Introduction The abundance of Big Data, structured, partially-structured and unstructured massive datasets, which are too large to be processed effectively

More information

CONCEPTUALIZING BUSINESS INTELLIGENCE ARCHITECTURE MOHAMMAD SHARIAT, Florida A&M University ROSCOE HIGHTOWER, JR., Florida A&M University

CONCEPTUALIZING BUSINESS INTELLIGENCE ARCHITECTURE MOHAMMAD SHARIAT, Florida A&M University ROSCOE HIGHTOWER, JR., Florida A&M University CONCEPTUALIZING BUSINESS INTELLIGENCE ARCHITECTURE MOHAMMAD SHARIAT, Florida A&M University ROSCOE HIGHTOWER, JR., Florida A&M University Given today s business environment, at times a corporate executive

More information

EUMETSAT DATA CENTRES AND ARCHIVE AND LONG-TERM DATA PRESERVATION

EUMETSAT DATA CENTRES AND ARCHIVE AND LONG-TERM DATA PRESERVATION Prepared by EUMETSAT Agenda Item: WGIV/5 EUMETSAT DATA CENTRES AND ARCHIVE AND LONG-TERM DATA PRESERVATION The importance of historic data and products derived from Meteorological Satellites, e.g. for

More information

Exploitation of ISS scientific data

Exploitation of ISS scientific data Cooperative ISS Research data Conservation and Exploitation Exploitation of ISS scientific data Luigi Carotenuto Telespazio s.p.a. Copernicus Big Data Workshop March 13-14 2014 European Commission Brussels

More information

Cloud Computing for Research Roger Barga Cloud Computing Futures, Microsoft Research

Cloud Computing for Research Roger Barga Cloud Computing Futures, Microsoft Research Cloud Computing for Research Roger Barga Cloud Computing Futures, Microsoft Research Trends: Data on an Exponential Scale Scientific data doubles every year Combination of inexpensive sensors + exponentially

More information

Space Work Programme 2015

Space Work Programme 2015 Space Work Programme 79 billion from 2014 to 2020 2 There is a place for SPACE everywhere 3 Space in Horizon 2020 Four objectives (specific programme) 1. Enhance competitiveness, non-dependence, and innovation

More information

NASA Earth Science Research in Data and Computational Science Technologies Report of the ESTO/AIST Big Data Study Roadmap Team September 2015

NASA Earth Science Research in Data and Computational Science Technologies Report of the ESTO/AIST Big Data Study Roadmap Team September 2015 NASA Earth Science Research in Data and Computational Science Technologies Report of the ESTO/AIST Big Data Study Roadmap Team September 2015 I. Background Over the next decade, the dramatic growth of

More information

A Professional Big Data Master s Program to train Computational Specialists

A Professional Big Data Master s Program to train Computational Specialists A Professional Big Data Master s Program to train Computational Specialists Anoop Sarkar, Fred Popowich, Alexandra Fedorova! School of Computing Science! Education for Employable Graduates: Critical Questions

More information

The Copernicus Master Prize and the ESA App Camps

The Copernicus Master Prize and the ESA App Camps The Copernicus Master Prize and the ESA App Camps EO Open Science 2.0 Frascati, 12 October 2015 Thomas Beer Copernicus Space Office, ESA- ESRIN The Copernicus Masters How it all started : In 2009 ESA-

More information

Is Big Data a Big Deal? What Big Data Does to Science

Is Big Data a Big Deal? What Big Data Does to Science Is Big Data a Big Deal? What Big Data Does to Science Netherlands escience Center Wilco Hazeleger Wilco Hazeleger Student @ Wageningen University and Reading University Meteorology PhD @ Utrecht University,

More information

Your door to future governance solutions

Your door to future governance solutions Your door to future governance solutions www.egovlab.eu 2 3 not just in theory but also in practice 4 5 www.egovlab.eu * Word from egovlab s director Vasilis Koulolias: The power of information and communication

More information

Worldwide Survey on Clouds for R&E

Worldwide Survey on Clouds for R&E Co-ordination & Harmonisation of Advanced e-infrastructures for Research and Education Data Sharing Worldwide Survey on Clouds for R&E Manuel Rodríguez, CIEMAT, on behalf of CHAIN-REDs project Rome, 27

More information

Question 3: Is cloud based SDI an opportunity or a threat for European SMEs?

Question 3: Is cloud based SDI an opportunity or a threat for European SMEs? Question 3: Is cloud based SDI an opportunity or a threat for European SMEs? Workshop: Open Data for stimulation of SME businesses in Agriculture, Transport, Tourism and Environment Miguel Ángel Esbrí,

More information

Internet of Things. Point of View. Turn your data into accessible, actionable insights for maximum business value.

Internet of Things. Point of View. Turn your data into accessible, actionable insights for maximum business value. Internet of Things Turn your data into accessible, actionable insights for maximum business value Executive Summary Use a connected ecosystem to create new levels of business value The Internet of Things

More information

USING BIG DATA FOR INTELLIGENT BUSINESSES

USING BIG DATA FOR INTELLIGENT BUSINESSES HENRI COANDA AIR FORCE ACADEMY ROMANIA INTERNATIONAL CONFERENCE of SCIENTIFIC PAPER AFASES 2015 Brasov, 28-30 May 2015 GENERAL M.R. STEFANIK ARMED FORCES ACADEMY SLOVAK REPUBLIC USING BIG DATA FOR INTELLIGENT

More information

Big Data Strategy Issues Paper

Big Data Strategy Issues Paper Big Data Strategy Issues Paper MARCH 2013 Contents 1. Introduction 3 1.1 Where are we now? 3 1.2 Why a big data strategy? 4 2. Opportunities for Australian Government agencies 5 2.1 What the future looks

More information

Big Data and Analytics: Challenges and Opportunities

Big Data and Analytics: Challenges and Opportunities Big Data and Analytics: Challenges and Opportunities Dr. Amin Beheshti Lecturer and Senior Research Associate University of New South Wales, Australia (Service Oriented Computing Group, CSE) Talk: Sharif

More information

Big Data and Healthcare Payers WHITE PAPER

Big Data and Healthcare Payers WHITE PAPER Knowledgent White Paper Series Big Data and Healthcare Payers WHITE PAPER Summary With the implementation of the Affordable Care Act, the transition to a more member-centric relationship model, and other

More information

Big Data and evolution of the Ground System EO ENG and the imarine case

Big Data and evolution of the Ground System EO ENG and the imarine case Big Data and evolution of the Ground System EO ENG and the imarine case Andrea Manieri Engineering R&D Lab. Rome, 26/11/2013 1 1 AGENDA The Big data challenges seen from the space Engineering and (some)

More information

ACCESS TO ERS AND ENVISAT DATA. CGMS is informed about the ESA Earth Observation data policy and data access, in particular in Near Real Time.

ACCESS TO ERS AND ENVISAT DATA. CGMS is informed about the ESA Earth Observation data policy and data access, in particular in Near Real Time. Prepared by ESA Agenda Item: III.3 Discussed in WG3 ACCESS TO ERS AND ENVISAT DATA CGMS is informed about the ESA Earth Observation data policy and data access, in particular in Near Real Time. ACCESS

More information

COMP9321 Web Application Engineering

COMP9321 Web Application Engineering COMP9321 Web Application Engineering Semester 2, 2015 Dr. Amin Beheshti Service Oriented Computing Group, CSE, UNSW Australia Week 11 (Part II) http://webapps.cse.unsw.edu.au/webcms2/course/index.php?cid=2411

More information

Big Data at ECMWF Providing access to multi-petabyte datasets Past, present and future

Big Data at ECMWF Providing access to multi-petabyte datasets Past, present and future Big Data at ECMWF Providing access to multi-petabyte datasets Past, present and future Baudouin Raoult Principal Software Strategist ECMWF Slide 1 ECMWF An independent intergovernmental organisation established

More information

Strategic Activities to Support Sustainability of Canada s Geospatial Data Infrastructure

Strategic Activities to Support Sustainability of Canada s Geospatial Data Infrastructure Strategic Activities to Support Sustainability of Canada s Geospatial Data Infrastructure Paula McLeod Canada Centre for Mapping and Earth Observation United Nations 10 th Regional Cartographic Conference

More information

Roadmapping Discussion Summary. Social Media and Linked Data for Emergency Response

Roadmapping Discussion Summary. Social Media and Linked Data for Emergency Response Roadmapping Discussion Summary Social Media and Linked Data for Emergency Response V. Lanfranchi 1, S. Mazumdar 1, E. Blomqvist 2, C. Brewster 3 1 OAK Group, Department of Computer Science University of

More information

A Binary Tree SMART CTO Webinar. Analyzing and Rationalizing Big Data in Messaging

A Binary Tree SMART CTO Webinar. Analyzing and Rationalizing Big Data in Messaging A Binary Tree SMART CTO Webinar Analyzing and Rationalizing Big Data in Messaging Who is Binary Tree? A Developer of Enterprise Email Migration Software Provides highest level of fidelity, performance

More information

Data Analytics, Management, Security and Privacy (Priority Area B)

Data Analytics, Management, Security and Privacy (Priority Area B) PRIORITY AREA B: DATA ANALYTICS, MANAGEMENT, SECURITY AND PRIVACY ACTION PLAN Data Analytics, Security and Privacy (Priority Area B) Context Data is growing at an exponential rate; information on the web

More information

How To Create A Data Science System

How To Create A Data Science System Enhance Collaboration and Data Sharing for Faster Decisions and Improved Mission Outcome Richard Breakiron Senior Director, Cyber Solutions Rbreakiron@vion.com Office: 571-353-6127 / Cell: 803-443-8002

More information

9360/15 FMA/AFG/cb 1 DG G 3 C

9360/15 FMA/AFG/cb 1 DG G 3 C Council of the European Union Brussels, 29 May 2015 (OR. en) 9360/15 OUTCOME OF PROCEEDINGS From: To: Council Delegations RECH 183 TELECOM 134 COMPET 288 IND 92 No. prev. doc.: 8970/15 RECH 141 TELECOM

More information

Data Centric Computing Revisited

Data Centric Computing Revisited Piyush Chaudhary Technical Computing Solutions Data Centric Computing Revisited SPXXL/SCICOMP Summer 2013 Bottom line: It is a time of Powerful Information Data volume is on the rise Dimensions of data

More information

Open Data, Open Innovation. magayler@microsoft.com

Open Data, Open Innovation. magayler@microsoft.com Open Data, Open Innovation and The Cloud magayler@microsoft.com Expectations are Changing Citizens expect Personalized, quality services wherever they are Engagement in issues that affect them Their privacy

More information

CYBERINFRASTRUCTURE FRAMEWORK FOR 21 st CENTURY SCIENCE AND ENGINEERING (CIF21)

CYBERINFRASTRUCTURE FRAMEWORK FOR 21 st CENTURY SCIENCE AND ENGINEERING (CIF21) CYBERINFRASTRUCTURE FRAMEWORK FOR 21 st CENTURY SCIENCE AND ENGINEERING (CIF21) Goal Develop and deploy comprehensive, integrated, sustainable, and secure cyberinfrastructure (CI) to accelerate research

More information

Exploring Big Data in Social Networks

Exploring Big Data in Social Networks Exploring Big Data in Social Networks virgilio@dcc.ufmg.br (meira@dcc.ufmg.br) INWEB National Science and Technology Institute for Web Federal University of Minas Gerais - UFMG May 2013 Some thoughts about

More information

Digital libraries of the future and the role of libraries

Digital libraries of the future and the role of libraries Digital libraries of the future and the role of libraries Donatella Castelli ISTI-CNR, Pisa, Italy Abstract Purpose: To introduce the digital libraries of the future, their enabling technologies and their

More information

Sentinels Operations Konzept und Prinzipien des Datenzugangs - Copernicus Space Component Data Access Overview

Sentinels Operations Konzept und Prinzipien des Datenzugangs - Copernicus Space Component Data Access Overview Sentinels Operations Konzept und Prinzipien des Datenzugangs - Copernicus Space Component Data Overview B. Hoersch Ground Segment and Mission Operations Department, Earth Observation Programmes Directorate,

More information

CYBERINFRASTRUCTURE FRAMEWORK FOR 21 ST CENTURY SCIENCE, ENGINEERING, AND EDUCATION (CIF21)

CYBERINFRASTRUCTURE FRAMEWORK FOR 21 ST CENTURY SCIENCE, ENGINEERING, AND EDUCATION (CIF21) CYBERINFRASTRUCTURE FRAMEWORK FOR 21 ST CENTURY SCIENCE, ENGINEERING, AND EDUCATION (CIF21) Overview The Cyberinfrastructure Framework for 21 st Century Science, Engineering, and Education (CIF21) investment

More information

Industry s view: How to match the services paradigm. by Markus Probeck, EARSC Director

Industry s view: How to match the services paradigm. by Markus Probeck, EARSC Director Industry s view: How to match the services paradigm by Markus Probeck, EARSC Director What is EARSC? EARSC is a trade association (NPO), founded in 1989, which represents European companies: offering and

More information

Concept and Project Objectives

Concept and Project Objectives 3.1 Publishable summary Concept and Project Objectives Proactive and dynamic QoS management, network intrusion detection and early detection of network congestion problems among other applications in the

More information

Industry 4.0 and Big Data

Industry 4.0 and Big Data Industry 4.0 and Big Data Marek Obitko, mobitko@ra.rockwell.com Senior Research Engineer 03/25/2015 PUBLIC PUBLIC - 5058-CO900H 2 Background Joint work with Czech Institute of Informatics, Robotics and

More information

Reference Architecture, Requirements, Gaps, Roles

Reference Architecture, Requirements, Gaps, Roles Reference Architecture, Requirements, Gaps, Roles The contents of this document are an excerpt from the brainstorming document M0014. The purpose is to show how a detailed Big Data Reference Architecture

More information

Big Data a threat or a chance?

Big Data a threat or a chance? Big Data a threat or a chance? Helwig Hauser University of Bergen, Dept. of Informatics Big Data What is Big Data? well, lots of data, right? we come back to this in a moment. certainly, a buzz-word but

More information

Data Analytics. SPAN White Paper. Turning information into insights

Data Analytics. SPAN White Paper. Turning information into insights SPAN White Paper Analytics Turning information into insights In today s business scenario, is defining a whole lot of organizational operations; it is not only a tool to assist a business strategy, but

More information

How to use Big Data in Industry 4.0 implementations. LAURI ILISON, PhD Head of Big Data and Machine Learning

How to use Big Data in Industry 4.0 implementations. LAURI ILISON, PhD Head of Big Data and Machine Learning How to use Big Data in Industry 4.0 implementations LAURI ILISON, PhD Head of Big Data and Machine Learning Big Data definition? Big Data is about structured vs unstructured data Big Data is about Volume

More information

European Data Infrastructure - EUDAT Data Services & Tools

European Data Infrastructure - EUDAT Data Services & Tools European Data Infrastructure - EUDAT Data Services & Tools Dr. Ing. Morris Riedel Research Group Leader, Juelich Supercomputing Centre Adjunct Associated Professor, University of iceland BDEC2015, 2015-01-28

More information

Outcomes of the CDS Technical Infrastructure Workshop

Outcomes of the CDS Technical Infrastructure Workshop Outcomes of the CDS Technical Infrastructure Workshop Baudouin Raoult Baudouin.raoult@ecmwf.int Funded by the European Union Implemented by Evaluation & QC function C3S architecture from European commission

More information

BIG DATA & ANALYTICS. Transforming the business and driving revenue through big data and analytics

BIG DATA & ANALYTICS. Transforming the business and driving revenue through big data and analytics BIG DATA & ANALYTICS Transforming the business and driving revenue through big data and analytics Collection, storage and extraction of business value from data generated from a variety of sources are

More information

Evolution of Chinese Research Data Policy

Evolution of Chinese Research Data Policy Bilateral US-China CODATA Workshop 2014 Evolution of Chinese Research Data Policy Jianhui Li(lijh@cnic.cn) Computer Network Information Center, CAS CODATA-China 25 Aug 2014 Outline Scientific Data Sharing

More information

Big Data Mining: Challenges and Opportunities to Forecast Future Scenario

Big Data Mining: Challenges and Opportunities to Forecast Future Scenario Big Data Mining: Challenges and Opportunities to Forecast Future Scenario Poonam G. Sawant, Dr. B.L.Desai Assist. Professor, Dept. of MCA, SIMCA, Savitribai Phule Pune University, Pune, Maharashtra, India

More information

11-12 June 2015, Bari-Italy. Stefano Nativi CNR-IIA

11-12 June 2015, Bari-Italy. Stefano Nativi CNR-IIA 11-12 June 2015, Bari-Italy Stefano Nativi CNR-IIA Coordinating an Observation Network of Networks EnCompassing satellite and IN-situ to fill the Gaps in European Observations GEOSS Information System

More information

W H I T E P A P E R. Deriving Intelligence from Large Data Using Hadoop and Applying Analytics. Abstract

W H I T E P A P E R. Deriving Intelligence from Large Data Using Hadoop and Applying Analytics. Abstract W H I T E P A P E R Deriving Intelligence from Large Data Using Hadoop and Applying Analytics Abstract This white paper is focused on discussing the challenges facing large scale data processing and the

More information

Kimmo Rossi. European Commission DG CONNECT

Kimmo Rossi. European Commission DG CONNECT Kimmo Rossi European Commission DG CONNECT Unit G.3 -Data Value Chain NCP training day, Brussels 18/9/2014 What we do Unit CNECT.G3 Data Value Chain FP7/CIP/H2020 project portfolio: Big Data, analytics,

More information

Self-Service Business Intelligence

Self-Service Business Intelligence Self-Service Business Intelligence BRIDGE THE GAP VISUALIZE DATA, DISCOVER TRENDS, SHARE FINDINGS Solgenia Analysis provides users throughout your organization with flexible tools to create and share meaningful

More information

Emerging Technologies CEOS/WGISS

Emerging Technologies CEOS/WGISS Emerging Technologies CEOS/WGISS CEOS Plenary 2014 Tromso, Norway Tuesday, October 28 th 16:00-17:00 Image Source: http://beck-technology.com/ Agenda WGISS Technology Exploration Interest Group Introduction

More information

BIG DATA: CHALLENGES AND OPPORTUNITIES IN LOGISTICS SYSTEMS

BIG DATA: CHALLENGES AND OPPORTUNITIES IN LOGISTICS SYSTEMS BIG DATA: CHALLENGES AND OPPORTUNITIES IN LOGISTICS SYSTEMS Branka Mikavica a*, Aleksandra Kostić-Ljubisavljević a*, Vesna Radonjić Đogatović a a University of Belgrade, Faculty of Transport and Traffic

More information

I. Justification and Program Goals

I. Justification and Program Goals MS in Data Science proposed by Department of Computer Science, B. Thomas Golisano College of Computing and Information Sciences Department of Information Sciences and Technologies, B. Thomas Golisano College

More information

BIG DATA: HELPING SCHOLARLY PUBLISHERS CUT THROUGH THE HYPE

BIG DATA: HELPING SCHOLARLY PUBLISHERS CUT THROUGH THE HYPE BIG DATA: HELPING SCHOLARLY PUBLISHERS CUT THROUGH THE HYPE Janice McCallum Health Content Advisors Association of American Publishers 2013 PSP Annual Conference, Washington, DC February 6-8, 2013 Focus

More information

ANALYTICS BUILT FOR INTERNET OF THINGS

ANALYTICS BUILT FOR INTERNET OF THINGS ANALYTICS BUILT FOR INTERNET OF THINGS Big Data Reporting is Out, Actionable Insights are In In recent years, it has become clear that data in itself has little relevance, it is the analysis of it that

More information

EO INSTITUTIONAL PERSPECTIVE

EO INSTITUTIONAL PERSPECTIVE EO INSTITUTIONAL PERSPECTIVE Emilio Vez Rodríguez CDTI 8 th September 2014 Agenda Global overview Figures, markets and main actors The European landscape: Development models Copernicus The role of ESA

More information

Hadoop in the Hybrid Cloud

Hadoop in the Hybrid Cloud Presented by Hortonworks and Microsoft Introduction An increasing number of enterprises are either currently using or are planning to use cloud deployment models to expand their IT infrastructure. Big

More information

IT Challenges for the Library and Information Studies Sector

IT Challenges for the Library and Information Studies Sector IT Challenges for the Library and Information Studies Sector This document is intended to facilitate and stimulate discussion at the e-science Scoping Study Expert Seminar for Library and Information Studies.

More information

Maximising economic growth and UK leadership in Earth Observation

Maximising economic growth and UK leadership in Earth Observation Maximising economic growth and UK leadership in Earth Observation Beth Greenaway, Head of Earth Observation 23 March 2015 http://www.bis.gov.uk/ukspaceagency Overview UK Space Agency EO Importance and

More information

Developing!EO!Services! Markets The!Importance!of!Data!Policy. Geoff!Sawyer!,!EARSC!Secretary! General

Developing!EO!Services! Markets The!Importance!of!Data!Policy. Geoff!Sawyer!,!EARSC!Secretary! General Developing!EO!Services! Markets The!Importance!of!Data!Policy Geoff!Sawyer!,!EARSC!Secretary! General What is EARSC? EARSC is a trade association (NGO), founded in 1989, which represents companies: offering

More information

Big Data R&D Initiative

Big Data R&D Initiative Big Data R&D Initiative Howard Wactlar CISE Directorate National Science Foundation NIST Big Data Meeting June, 2012 Image Credit: Exploratorium. The Landscape: Smart Sensing, Reasoning and Decision Environment

More information

Remote sensing information cloud service: research and practice

Remote sensing information cloud service: research and practice Remote sensing information cloud service: research and practice Yang Banghui Dr., Ren Fuhu Prof. and Wang jinnian Prof. yangbh@radi.ac.cn +8613810963452 Content 1 Background 2 Studying and Designing 3

More information

COMMISSION OF THE EUROPEAN COMMUNITIES COMMUNICATION FROM THE COMMISSION TO THE COUNCIL AND THE EUROPEAN PARLIAMENT

COMMISSION OF THE EUROPEAN COMMUNITIES COMMUNICATION FROM THE COMMISSION TO THE COUNCIL AND THE EUROPEAN PARLIAMENT COMMISSION OF THE EUROPEAN COMMUNITIES Brussels, 23.05.2005 COM(2005) 208 final COMMUNICATION FROM THE COMMISSION TO THE COUNCIL AND THE EUROPEAN PARLIAMENT European Space Policy - Preliminary Elements

More information

HARNESSING BIG DATA WITHIN THE FEDERAL GOVERNMENT FINDINGS AND RECOMMENDATIONS OF ATARC S BIG DATA INNOVATION LAB DECEMBER, 2015

HARNESSING BIG DATA WITHIN THE FEDERAL GOVERNMENT FINDINGS AND RECOMMENDATIONS OF ATARC S BIG DATA INNOVATION LAB DECEMBER, 2015 HARNESSING BIG DATA WITHIN THE FEDERAL GOVERNMENT FINDINGS AND RECOMMENDATIONS OF ATARC S BIG DATA INNOVATION LAB DECEMBER, 2015 ATARC Big Data Innovation Lab Sponsors ATARC Big Data Innovation Lab Objective

More information

Anuradha Bhatia, Faculty, Computer Technology Department, Mumbai, India

Anuradha Bhatia, Faculty, Computer Technology Department, Mumbai, India Volume 3, Issue 9, September 2013 ISSN: 2277 128X International Journal of Advanced Research in Computer Science and Software Engineering Research Paper Available online at: www.ijarcsse.com A Real Time

More information

AppSymphony White Paper

AppSymphony White Paper AppSymphony White Paper Secure Self-Service Analytics for Curated Digital Collections Introduction Optensity, Inc. offers a self-service analytic app composition platform, AppSymphony, which enables data

More information

How To Understand The Benefits Of Big Data

How To Understand The Benefits Of Big Data Findings from the research collaboration of IBM Institute for Business Value and Saïd Business School, University of Oxford Analytics: The real-world use of big data How innovative enterprises extract

More information

8970/15 FMA/AFG/cb 1 DG G 3 C

8970/15 FMA/AFG/cb 1 DG G 3 C Council of the European Union Brussels, 19 May 2015 (OR. en) 8970/15 NOTE RECH 141 TELECOM 119 COMPET 228 IND 80 From: Permanent Representatives Committee (Part 1) To: Council No. prev. doc.: 8583/15 RECH

More information