A standards-based open source processing chain for ocean modeling in the GEOSS Architecture Implementation Pilot Phase 8 (AIP-8)

Size: px
Start display at page:

Download "A standards-based open source processing chain for ocean modeling in the GEOSS Architecture Implementation Pilot Phase 8 (AIP-8)"

Transcription

1 NATO Science & Technology Organization Centre for Maritime Research and Experimentation (STO-CMRE) Viale San Bartolomeo, La Spezia, Italy A standards-based open source processing chain for ocean modeling in the GEOSS Architecture Implementation Pilot Phase 8 (AIP-8) Response Due Date: 17 March 2014 Programmatic POC Alessandro Berni Technical POC Elena Camossi

2 STO-CMRE Response to the GEOSS AIP-8 CFP Overview This response is submitted by STO-CMRE (http://www.cmre.nato.int), leveraging resources provided by the partner Terradue Srl (http://www.terradue.com). The purpose of this participation is to continue and extend the scope (adding in-situ data and OGC SOS service) of the work done in AIP-7. Our goal for AIP-8 is to experiment further with the key enablers in the domain of integrated environmental knowledge and decision support tools. The objective will be the exploitation of key ocean environmental variables from in-situ observations, leveraging environmental models and geo-processing, with specific attention to interoperability and connectivity with the comprehensive global network of environmental data collection and delivery. The proposed development is leveraging ocean forecasting data feeds, computeintensive models for data processing, and registration of ocean forecasting products in the GEOSS Common Infrastructure: Results exploitation will be performed in a concrete framework for policy and decision making, where scientific collaboration is having a key role in supporting informed decision; Application development will consist in the continuation/extension of earlier work conducted during AIP-7 on porting an ocean forecasting model into a dedicated Cloud environment, a Platform as a Service (PaaS) that provides parallel processing and standard OGC interfaces to the existing model. Application development will also provide a non-expert (streamlined) user dashboard to visualize and monitor the environment variables involved in the processing; The participation provides benefits for the GEO community in terms of new contributions of research data, in the form of increased availability of ocean in-situ data and added-value outputs. This represents an example of potential GEOSS Future Products, as well as a demonstration of the possible use by the GEO community of Cloud processing services, generating these products from remotely sensed data and near real-time in-situ data feeds. 2

3 Proposed Contributions STO-CMRE started to engage with the GEOSS community during AIP-7, during which some development activities were made using the Regional Ocean Modeling System (ROMS) in conjunction to one sea trial, as an enabler to subsequent model validation activities. Objectives of the continued engagement with GEOSS include linking with an international network and an advanced ocean data management capability, related to research activities in the Maritime domain. Through GEOSS AIP, STO-CMRE is seeking to further develop research collaborations with GEO stakeholders working in the Ocean monitoring and Ocean forecasting fields. Fig. 1: In-situ environmental surveys & collection of ocean data for model validation The new development in AIP-8 will consist in the integration of observational data from ocean gliders using the Sensor Observation Service (SOS) to register in-situ observations such as data from gliders, and then to serve them (after they have been validated) to an App (e.g. Regional Ocean Model System ROMS) running in a PaaS cloud system. 3

4 1.1.1 Key application support STO-CMRE and Terradue are contributing directly to the design and development of a key app listed in the CFP, with the following information on requirements and contributions: Name of the application: Standards-based open source processing chain for ocean modeling Contributing organizations and contact information: STO-CMRE (Alessandro Berni), TERRADUE (Hervé Caumont). SBA(s) supported by the application: the proposed App is supporting the GEO SBAs related to the improvement of weather information, forecasting and warning (Weather SBA) and related to the management and protection of coastal and marine resources (Ecosystems SBA). Names and contact information of end-user(s) who will provide the requirements: several end-user prospects will be addressed by CMRE in the frame of bilateral scientific collaborations, as well as in the domain of commercial activities dependent on ocean data. Overview of the App functionality (use case): the compute-intensive processing functionality is an extension of the Developer Cloud Sandbox service, initially contributed to GEOSS through the EC FP7 GEOWOW project, with a specific application to ocean forecasting models. Application Development Framework to be used: Terradue s Developer Cloud Sandboxes PaaS (Platform as a Service). Deployment: demonstrate a standards-based open source processing chain for ocean modeling, with browser-based access for users, leveraging GEOSS data, and instantiated for Ocean observations and commercial fisheries applications. List of data sources, standards based interface protocols (OGC): o The data sources that will be exploited on the Cloud infrastructure consist in forecast elements that are in turn derived from the following assimilated information: In-Situ data: in situ temperature and salinity profiles acquired with ARGO floats, CTD, XBT, and Glider data EO data: sea level anomaly, sea surface temperature (SST) from atmospheric and ocean models output, and bathymetries o The data products generated and contributed to GEOSS consist in experimental model output products, encoded in OGC Network Common Data Format (NetCDF): Sea surface temperature, Sea level anomaly Salinity, Currents Supporting Technologies This proposal falls in the Model Web thread, exploiting the similarities between the Information to Knowledge theme with CMRE s data to decision vision of establishing information supply chains than span from the collection of observational data, to their curation and processing, to derive useful products to support decision-making processes. This, in turn, requires interoperability with the 4

5 global Earth Observation networks, incorporating feedback between decision-making and observational data requirements. STO-CMRE Data Services (Spatial Data Infrastructure) Spatial Data Infrastructure Standard interfaces: o OGC WPS o OGC WMS o OGC CSW Metadata applied in near real time (ISO 19115, /19139, with NATO Geospatial Metadata Profile); data fed into a OGCcompliant catalogue; o OGC SOS Use of the istsos distribution https://geoservice.ist.supsi.ch/projects/istsos/ CKAN user portal: o Periodic harvesting and indexing of catalogue data; THREDDS server (to serve data to the model running on the cloud sandbox) Cloud Infrastructure and Cloud Services (Terradue) Terradue s contribution will leverage the company s experience as an operator of a private Cloud infrastructure, supporting bursting to commercial Clouds via APIs. Terradue s infrastructure is outsourced, rented with a setup fee and exploited through a subscription cost model. Cloud bursting to commercial providers is supporting both pay-as-you-go or subscription costs, offering a fine grain Infrastructure-as-a-Service in Terradue s Cloud Services portfolio. The result is a Cloud infrastructure composed of a set of Virtual Processing and Archiving centers. In the context of AIP-8, the cloud bursting capability will leverage the Amazon Web Services public Cloud to perform compute intensive tasks. Terradue s Cloud platform is operating on a virtualized data center, enabled with the OpenNebula cloud controller, notably featuring Orchestration and Auto-Scaling of Cloud Multi-Tier Applications. The platform s Virtual Hosts are made available with the following default configuration: - 4-Core CPU, 8 GB RAM - 2 TB local disk - OS: CentOS 6 - Other software pre-installed: command line utilities for data staging Building on top of Terradue s Cloud platform, partners can host geospatial data processing, available from and to OGC providers worldwide. To this end, the "Developer Cloud Sandboxes" service builds on technology designed for use cases having data-intensive requirements. Partner organizations can implement processing chains with full control of code, parameters & data flows, in a collaborative way 5

6 within a shared Platform delivered as a Service, and seamlessly exploiting Cloud APIs to stage data and deploy code on ad-hoc computing clusters. Science Application Development Layer Platform Tools Data Repositories Query Lead Scientist Data Access Results Publication & Visualization Scientists Direct Acyclic Graph Compilers MatLab Project Tool Suite Versioning R IDL Ticketing Oozie Workflow Scheduler for Hadoop Distributed Data Storage HDFS Distributed Data Analytics Hadoop Map Reduce Streaming Virtualization Layer Fig. 2: The Developer Cloud Sandboxes service on Terradue s Platform Using the power of 'Hadoop Streaming' for legacy applications integration, processing algorithms are enabled to access distributed data holdings and to scale over computing clusters. The service leverages the power of OpenNebula technology for managing virtualized computing nodes, and of standard Cloud APIs for resources provisioning and appliances deployment. OpenNebula (ONe): OpenNebula.org is an open-source project developing the industry standard solution for building and managing virtualized enterprise data centers and enterprise private clouds. Terradue is a contributor to the OpenNebula open source baseline and it delivers native capabilities for Orchestration and Auto- Scaling of Cloud Multi-Tier Applications. Terradue is a Channel Partner (Value Added Reseller) of the OpenNebula C12G Partner Program and was a speaker at the OpenNebula Global Conference in Berlin, September 2013, offering an overview of Terradue s Cloud Infrastructure and DevOps-enabled Services (http://www.slideshare.net/terradue/opennebulaconf2013). Apache Hadoop: Terradue is operating a Cloud processing service leveraging the Apache Hadoop framework (Cloudera distribution). The WPS-Hadoop Cloud service runs on an OpenNebula-driven private Cloud, with: - One appliance with Terradue s WPS-Hadoop I/F. 6

7 - Several Hadoop pseudo-clusters, configured from the Cloudera's Distribution Including Apache Hadoop (CDH3) baseline Architecture and Interoperability Arrangement Development STO-CMRE and Terradue jointly plans to contribute and support the refinement of the GEOSS AIP Architecture and Interoperability Arrangements, in the following ways: Type of contribution: o Cloud services interoperability (GCI evolutions for Cloud processing workflow management) o Cloud Services architecture (GCI evolutions for cloud interoperability) Data sources supported: o Essential Ocean Variables (netcdf) Standards supported to provide access to data and products: o OGC CSW o OGC CSW 3.0 OpenSearch Geo & Time Extensions o OGC WCS 1.0 o OGC WPS 1.0 o OGC SOS 2.0 7

8 Description of STO-CMRE NATO s Science & Technology Organization (STO) Centre for Maritime Research and Experimentation (CMRE) is an established, world-class scientific research and experimentation facility that organizes and conducts scientific research and technology development, centered on the maritime domain, delivering innovative and field tested Science & Technology (S&T) solutions. Located in La Spezia (Italy), the CMRE is built on more than 50 years of experience and has produced a cadre of leaders in ocean science, modeling and simulation, acoustics and other disciplines, as well as producing critical results and understanding for the benefit of member Nations. CMRE operates two research vessels that enable science and technology solutions to be explored and developed at sea. The largest of these vessels, the NATO Research Vessel Alliance, is an ice-capable global class vessel that is one of the world's quietest vessels, allowing for precision acoustic studies to be conducted at sea. The engagement with AIP-8 is performed within the scope of project Decisions in Uncertain Ocean Environments, funded by Allied Command Transformation as part of the Environmental Knowledge and Operational Effectiveness Programme. The technologies that will be tested and showcased represent some of the enablers of CMRE s data to decision concept, an information supply chain in which data from real world and the model web are combined to provide decision-makers with new and improved supporting tools. STO-CMRE key persons are: Alessandro Berni, Elena Camossi, Raul Vicen-Bueno and Giampaolo Cimino. 8

Task AR-09-01a Progress and Contributions

Task AR-09-01a Progress and Contributions Doug Nebert, POC U.S. Geological Survey ddnebert@usgs.gov March 2010 Task AR-09-01a Progress and Contributions Background and scope for AR-09-01a This Task defines the minimum, common technical capabilities

More information

The distribution of marine OpenData via distributed data networks and Web APIs. The example of ERDDAP, the message broker and data mediator from NOAA

The distribution of marine OpenData via distributed data networks and Web APIs. The example of ERDDAP, the message broker and data mediator from NOAA The distribution of marine OpenData via distributed data networks and Web APIs. The example of ERDDAP, the message broker and data mediator from NOAA Dr. Conor Delaney 9 April 2014 GeoMaritime, London

More information

Big Data, Cloud Computing, Spatial Databases Steven Hagan Vice President Server Technologies

Big Data, Cloud Computing, Spatial Databases Steven Hagan Vice President Server Technologies Big Data, Cloud Computing, Spatial Databases Steven Hagan Vice President Server Technologies Big Data: Global Digital Data Growth Growing leaps and bounds by 40+% Year over Year! 2009 =.8 Zetabytes =.08

More information

Using Message Brokering and Data Mediation to use Distributed Data Networks of Earth Science Data to Enhance Global Maritime Situational Awareness.

Using Message Brokering and Data Mediation to use Distributed Data Networks of Earth Science Data to Enhance Global Maritime Situational Awareness. Using Message Brokering and Data Mediation to use Distributed Data Networks of Earth Science Data to Enhance Global Maritime Situational Awareness. Conor Delaney, Alfredo Alessandrini & Harm Greidanus

More information

Linking Sensor Web Enablement and Web Processing Technology for Health-Environment Studies

Linking Sensor Web Enablement and Web Processing Technology for Health-Environment Studies Linking Sensor Web Enablement and Web Processing Technology for Health-Environment Studies Simon Jirka 1, Stefan Wiemann 2, Johannes Brauner 2, and Eike Hinderk Jürrens 1 1 52 North Initiative for Geospatial

More information

Sistemi Operativi e Reti. Cloud Computing

Sistemi Operativi e Reti. Cloud Computing 1 Sistemi Operativi e Reti Cloud Computing Facoltà di Scienze Matematiche Fisiche e Naturali Corso di Laurea Magistrale in Informatica Osvaldo Gervasi ogervasi@computer.org 2 Introduction Technologies

More information

GENESI-DEC: a federative e-infrastructure for Earth Science data discovery, access, and on-demand processing

GENESI-DEC: a federative e-infrastructure for Earth Science data discovery, access, and on-demand processing GENESI-DEC: a federative e-infrastructure for Earth Science data discovery, access, and on-demand processing Roberto Cossu 1, Fabrizio Pacini 2, Fabrice Brito 3,Luigi Fusco 1, Eliana Li Santi 1, and Andrea

More information

Insights from 40 years of data services at NCAR s Research Data Archive. Douglas Schuster and Steven Worley

Insights from 40 years of data services at NCAR s Research Data Archive. Douglas Schuster and Steven Worley Insights from 40 years of data services at NCAR s Research Archive Douglas Schuster (schuster@ucar.edu) and Steven Worley (worley@ucar.edu) Highlights What is the? Evolution of Services User Registration

More information

The Arctic Observing Network and its Data Management Challenges Florence Fetterer (NSIDC/CIRES/CU), James A. Moore (NCAR/EOL), and the CADIS team

The Arctic Observing Network and its Data Management Challenges Florence Fetterer (NSIDC/CIRES/CU), James A. Moore (NCAR/EOL), and the CADIS team The Arctic Observing Network and its Data Management Challenges Florence Fetterer (NSIDC/CIRES/CU), James A. Moore (NCAR/EOL), and the CADIS team Photo courtesy Andrew Mahoney NSF Vision What is AON? a

More information

Big Data - Infrastructure Considerations

Big Data - Infrastructure Considerations April 2014, HAPPIEST MINDS TECHNOLOGIES Big Data - Infrastructure Considerations Author Anand Veeramani / Deepak Shivamurthy SHARING. MINDFUL. INTEGRITY. LEARNING. EXCELLENCE. SOCIAL RESPONSIBILITY. Copyright

More information

George Mason University (GMU)

George Mason University (GMU) George Mason University (GMU) Center for Spatial Information Science and Systems Organization (CSISS) 4400 University Drive, MSN 6E1 George Mason University Fairfax, VA 22030, USA Telephone: +1 703 993

More information

A Future Scenario of interconnected EO Platforms How will EO data be used in 2025?

A Future Scenario of interconnected EO Platforms How will EO data be used in 2025? A Future Scenario of interconnected EO Platforms How will EO data be used in 2025? ESA UNCLASSIFIED For Official Use European EO data asset Heritage missions Heritage Core GS (data preservation, curation

More information

Enabling the SmartGrid through Cloud Computing

Enabling the SmartGrid through Cloud Computing Enabling the SmartGrid through Cloud Computing April 2012 Creating Value, Delivering Results 2012 eglobaltech Incorporated. Tech, Inc. All rights reserved. 1 Overall Objective To deliver electricity from

More information

Big Data and Cloud Computing for GHRSST

Big Data and Cloud Computing for GHRSST Big Data and Cloud Computing for GHRSST Jean-Francois Piollé (jfpiolle@ifremer.fr) Frédéric Paul, Olivier Archer CERSAT / Institut Français de Recherche pour l Exploitation de la Mer Facing data deluge

More information

GeoNetwork, The Open Source Solution for the interoperable management of geospatial metadata

GeoNetwork, The Open Source Solution for the interoperable management of geospatial metadata GeoNetwork, The Open Source Solution for the interoperable management of geospatial metadata Ing. Emanuele Tajariol, GeoSolutions Ing. Simone Giannecchini, GeoSolutions GeoSolutions GeoSolutions GeoNetwork

More information

FIWARE Lab Solution for Managing Resources & Services in a Cloud Federation

FIWARE Lab Solution for Managing Resources & Services in a Cloud Federation FIWARE Lab Solution for Managing Resources & Services in a Cloud Federation Yahya Al-Hazmi Technische Universität Berlin yahya.al-hazmi@tu-berlin.de XIFI Webinar GoToWebinar February 23, 2015, 11-12 AM

More information

Open Cloud System. (Integration of Eucalyptus, Hadoop and AppScale into deployment of University Private Cloud)

Open Cloud System. (Integration of Eucalyptus, Hadoop and AppScale into deployment of University Private Cloud) Open Cloud System (Integration of Eucalyptus, Hadoop and into deployment of University Private Cloud) Thinn Thu Naing University of Computer Studies, Yangon 25 th October 2011 Open Cloud System University

More information

NASA's Strategy and Activities in Server Side Analytics

NASA's Strategy and Activities in Server Side Analytics NASA's Strategy and Activities in Server Side Analytics Tsengdar Lee, Ph.D. High-end Computing Program Manager NASA Headquarters Presented at the ESGF/UVCDAT Conference Lawrence Livermore National Laboratory

More information

Collaborative Open Market to Place Objects at your Service

Collaborative Open Market to Place Objects at your Service Collaborative Open Market to Place Objects at your Service D6.2.1 Developer SDK First Version D6.2.2 Developer IDE First Version D6.3.1 Cross-platform GUI for end-user Fist Version Project Acronym Project

More information

Outcomes of the CDS Technical Infrastructure Workshop

Outcomes of the CDS Technical Infrastructure Workshop Outcomes of the CDS Technical Infrastructure Workshop Baudouin Raoult Baudouin.raoult@ecmwf.int Funded by the European Union Implemented by Evaluation & QC function C3S architecture from European commission

More information

Coastal Research Proposal Abstracts

Coastal Research Proposal Abstracts Coastal Research Proposal Abstracts An Automated Data Analysis, Processing, and Management System Proposal to NASA with SAIC Total Budget $15K Submitted 10/31/2008 The goal of this study is to design,

More information

Adobe Deploys Hadoop as a Service on VMware vsphere

Adobe Deploys Hadoop as a Service on VMware vsphere Adobe Deploys Hadoop as a Service A TECHNICAL CASE STUDY APRIL 2015 Table of Contents A Technical Case Study.... 3 Background... 3 Why Virtualize Hadoop on vsphere?.... 3 The Adobe Marketing Cloud and

More information

13 th EC GI & GIS Workshop WIN: A new OGC compliant SOA. for risk management. GMV, 2007 Property of GMV All rights reserved

13 th EC GI & GIS Workshop WIN: A new OGC compliant SOA. for risk management. GMV, 2007 Property of GMV All rights reserved 13 th EC GI & GIS Workshop WIN: A new OGC compliant SOA for risk management GMV, 2007 Property of GMV All rights reserved Content 1. Introduction 2. Objectives 3. Architecture and Model 4. Technical aspects

More information

The ORIENTGATE data platform

The ORIENTGATE data platform Seminar on Proposed and Revised set of indicators June 4-5, 2014 - Belgrade (Serbia) The ORIENTGATE data platform WP2, Action 2.4 Alessandra Nuzzo, Sandro Fiore, Giovanni Aloisio Scientific Computing and

More information

Global Earth Observation Integrated Data Environment (GEO-IDE) Presentation to the Data Archiving and Access Requirements Working Group (DAARWG)

Global Earth Observation Integrated Data Environment (GEO-IDE) Presentation to the Data Archiving and Access Requirements Working Group (DAARWG) Global Earth Observation Integrated Data Environment (GEO-IDE) Presentation to the Data Archiving and Access Requirements Working Group (DAARWG) Ken McDonald Data Management Integration Architect National

More information

CA Technologies Big Data Infrastructure Management Unified Management and Visibility of Big Data

CA Technologies Big Data Infrastructure Management Unified Management and Visibility of Big Data Research Report CA Technologies Big Data Infrastructure Management Executive Summary CA Technologies recently exhibited new technology innovations, marking its entry into the Big Data marketplace with

More information

GENESIS Employing Web Processing Services and Sensor Web Technology for Environmental Management

GENESIS Employing Web Processing Services and Sensor Web Technology for Environmental Management GENESIS Employing Web Processing Services and Sensor Web Technology for Environmental Management Steven Smolders (1), Christian Alegre (2), Simone Gianfranceschi (3), Marc Gilles (4), Bernd Resch (5),

More information

A Study of Data Management Technology for Handling Big Data

A Study of Data Management Technology for Handling Big Data Available Online at www.ijcsmc.com International Journal of Computer Science and Mobile Computing A Monthly Journal of Computer Science and Information Technology IJCSMC, Vol. 3, Issue. 9, September 2014,

More information

Bringing Big Data Modelling into the Hands of Domain Experts

Bringing Big Data Modelling into the Hands of Domain Experts Bringing Big Data Modelling into the Hands of Domain Experts David Willingham Senior Application Engineer MathWorks david.willingham@mathworks.com.au 2015 The MathWorks, Inc. 1 Data is the sword of the

More information

D3.4.1: Data Fusion Tools

D3.4.1: Data Fusion Tools COMPETITIVINESS AND INNOVATION FRAMEWORK PROGRAMME CIP-ICT-PSP-2013-7 Pilot Type B WP3 Service platform integration and deployment in cloud infrastructure D3.4.1: Data Fusion Tools Deliverable Lead: NETCAD

More information

Vision. South Pacific GIS/RS Conference 2015 11/17/2015. Applying Geography Everywhere. Applying Geography Everywhere

Vision. South Pacific GIS/RS Conference 2015 11/17/2015. Applying Geography Everywhere. Applying Geography Everywhere South Pacific GIS/RS Conference 2015 Applying Geography Everywhere Jithen (J) Singh Eagle Technology Group Ltd (Esri New Zealand) Suva, Fiji Vision Applying Geography Everywhere 1 GIS Is Evolving A Whole

More information

GOSIC NEXRAD NIDIS NOMADS

GOSIC NEXRAD NIDIS NOMADS NOAA National Climatic Data Center GOSIC NEXRAD NIDIS NOMADS Christina Lief NOAA/NESDIS/NCDC GOSIC Program Manager NOAA/NESDIS/NCDC Asheville, NC 28801 GEOSS AIP Phase 2 Workshop September 25-26, 2008

More information

A Service for Data-Intensive Computations on Virtual Clusters

A Service for Data-Intensive Computations on Virtual Clusters A Service for Data-Intensive Computations on Virtual Clusters Executing Preservation Strategies at Scale Rainer Schmidt, Christian Sadilek, and Ross King rainer.schmidt@arcs.ac.at Planets Project Permanent

More information

Integrated Risk Management System Components in the GEO Architecture Implementation Pilot Phase 2 (AIP-2)

Integrated Risk Management System Components in the GEO Architecture Implementation Pilot Phase 2 (AIP-2) Meraka Institute ICT for Earth Observation PO Box 395 Pretoria 0001, Gauteng, South Africa Telephone: +27 12 841 3028 Facsimile: +27 12 841 4720 University of KwaZulu- Natal School of Computer Science

More information

Optimizing Service Levels in Public Cloud Deployments

Optimizing Service Levels in Public Cloud Deployments WHITE PAPER OCTOBER 2014 Optimizing Service Levels in Public Cloud Deployments Keys to Effective Service Management 2 WHITE PAPER: OPTIMIZING SERVICE LEVELS IN PUBLIC CLOUD DEPLOYMENTS ca.com Table of

More information

Product Navigator User Guide

Product Navigator User Guide Product Navigator User Guide Table of Contents Contents About the Product Navigator... 1 Browser support and settings... 2 Searching in detail... 3 Simple Search... 3 Extended Search... 4 Browse By Theme...

More information

Understanding. Purpose. Creating Relationships

Understanding. Purpose. Creating Relationships Welcome Understanding Purpose Creating Relationships You A Global Community of GIS Professionals Changing the World Together Applying Geography Everywhere Your WORK Showcasing Your Web Apps Vision Applying

More information

The Future of Data Management

The Future of Data Management The Future of Data Management with Hadoop and the Enterprise Data Hub Amr Awadallah (@awadallah) Cofounder and CTO Cloudera Snapshot Founded 2008, by former employees of Employees Today ~ 800 World Class

More information

SAP HANA Cloud Platform Overview Customer

SAP HANA Cloud Platform Overview Customer SAP HANA Cloud Platform Overview Customer BUILD New Cloud Apps EXTEND On-Premise Apps EXTEND Cloud Apps ON-PREMISE SOLUTION CLOUD SOLUTION Data Data RUN Application Management and Runtime 2014 SAP AG or

More information

Mr. Apichon Witayangkurn apichon@iis.u-tokyo.ac.jp Department of Civil Engineering The University of Tokyo

Mr. Apichon Witayangkurn apichon@iis.u-tokyo.ac.jp Department of Civil Engineering The University of Tokyo Sensor Network Messaging Service Hive/Hadoop Mr. Apichon Witayangkurn apichon@iis.u-tokyo.ac.jp Department of Civil Engineering The University of Tokyo Contents 1 Introduction 2 What & Why Sensor Network

More information

Environment Canada Data Management Program. Paul Paciorek Corporate Services Branch May 7, 2014

Environment Canada Data Management Program. Paul Paciorek Corporate Services Branch May 7, 2014 Environment Canada Data Management Program Paul Paciorek Corporate Services Branch May 7, 2014 EC Data Management Program (ECDMP) consists of 5 foundational, incremental projects which will implement

More information

EUMETSAT EO Portal. End User Image Access using OGC WMS/WCS services. EUM/OPS/VWG/10/0095 Issue <1> <14/01/2010> Slide: 1

EUMETSAT EO Portal. End User Image Access using OGC WMS/WCS services. EUM/OPS/VWG/10/0095 Issue <1> <14/01/2010> Slide: 1 EUMETSAT EO Portal End User Image Access using OGC WMS/WCS services Slide: 1 Overview Introduction: status of data access and visualization EUMETSAT datasets Architecture Web Map Service implementation

More information

Applying Apache Hadoop to NASA s Big Climate Data!

Applying Apache Hadoop to NASA s Big Climate Data! National Aeronautics and Space Administration Applying Apache Hadoop to NASA s Big Climate Data! Use Cases and Lessons Learned! Glenn Tamkin (NASA/CSC)!! Team: John Schnase (NASA/PI), Dan Duffy (NASA/CO),!

More information

Cloud Computing: Making the right choices

Cloud Computing: Making the right choices Cloud Computing: Making the right choices Kalpak Shah Clogeny Technologies Pvt Ltd 1 About Me Kalpak Shah Founder & CEO, Clogeny Technologies Passionate about economics and technology evolving through

More information

Sextant. Spatial Data Infrastructure for Marine Environment. C. Satra Le Bris, E. Quimbert, M. Treguer

Sextant. Spatial Data Infrastructure for Marine Environment. C. Satra Le Bris, E. Quimbert, M. Treguer Sextant On-Line information system for marine geographical information E. Quimbert, M. Bellouis, F. Lecuy, M. Treguer Centre de Bretagne BP 70, Plouzané 29280 France E-mail: sextant@ifremer.fr Sextant

More information

Building Clouds with OpenNebula 3.4

Building Clouds with OpenNebula 3.4 OSDC 2012 24th April, Nürnberg Building Clouds with OpenNebula 3.4 Constantino Vázquez Blanco dsa-research.org Distributed Systems Architecture Research Group Universidad Complutense de Madrid Building

More information

CDI/THREDDS Interoperability: the SeaDataNet developments. P. Mazzetti 1,2, S. Nativi 1,2, 1. CNR-IMAA; 2. PIN-UNIFI

CDI/THREDDS Interoperability: the SeaDataNet developments. P. Mazzetti 1,2, S. Nativi 1,2, 1. CNR-IMAA; 2. PIN-UNIFI CDI/THREDDS Interoperability: the SeaDataNet developments P. Mazzetti 1,2, S. Nativi 1,2, 1. CNR-IMAA; 2. PIN-UNIFI Outline Interoperability Issues in SeaDataNet A broker solution for CDI/THREDDS interoperability

More information

Using standards for ocean data

Using standards for ocean data Pan-European infrastructure for marine and ocean data management Using standards for ocean data By Dick M.A. Schaap SeaDataNet Technical Coordinator Vienna, April 09 Supported by the European Commission

More information

11-12 June 2015, Bari-Italy. Stefano Nativi CNR-IIA

11-12 June 2015, Bari-Italy. Stefano Nativi CNR-IIA 11-12 June 2015, Bari-Italy Stefano Nativi CNR-IIA Coordinating an Observation Network of Networks EnCompassing satellite and IN-situ to fill the Gaps in European Observations GEOSS Information System

More information

CloudSigma. Customisable cloud infrastructure and data federation in the GEOSS Architecture Implementation Pilot Phase 8 (AIP-8)

CloudSigma. Customisable cloud infrastructure and data federation in the GEOSS Architecture Implementation Pilot Phase 8 (AIP-8) CloudSigma Tzarigradsko Shosse 47A Office center Poligrafia CloudSigma, Floor 3, office 308 Sofia 1124 Bulgaria Customisable cloud infrastructure and data federation in the GEOSS Architecture Implementation

More information

Cisco Data Preparation

Cisco Data Preparation Data Sheet Cisco Data Preparation Unleash your business analysts to develop the insights that drive better business outcomes, sooner, from all your data. As self-service business intelligence (BI) and

More information

Cloud-based Infrastructures. Serving INSPIRE needs

Cloud-based Infrastructures. Serving INSPIRE needs Cloud-based Infrastructures Serving INSPIRE needs INSPIRE Conference 2014 Workshop Sessions Benoit BAURENS, AKKA Technologies (F) Claudio LUCCHESE, CNR (I) June 16th, 2014 This content by the InGeoCloudS

More information

MyOcean Copernicus Marine Service Architecture and data access Experience

MyOcean Copernicus Marine Service Architecture and data access Experience MyOcean Copernicus Marine Service Architecture and data access Experience Sophie Besnard CLS, Toulouse, France February 2015 MyOcean Story MyOcean Challenge & Success MyOcean Service MyOcean System MyOcean

More information

Big Data Use Case. How Rackspace is using Private Cloud for Big Data. Bryan Thompson. May 8th, 2013

Big Data Use Case. How Rackspace is using Private Cloud for Big Data. Bryan Thompson. May 8th, 2013 Big Data Use Case How Rackspace is using Private Cloud for Big Data Bryan Thompson May 8th, 2013 Our Big Data Problem Consolidate all monitoring data for reporting and analytical purposes. Every device

More information

Modern Data Architecture for Predictive Analytics

Modern Data Architecture for Predictive Analytics Modern Data Architecture for Predictive Analytics David Smith VP Marketing and Community - Revolution Analytics John Kreisa VP Strategic Marketing- Hortonworks Hortonworks Inc. 2013 Page 1 Your Presenters

More information

Orbiter Series Service Oriented Architecture Applications

Orbiter Series Service Oriented Architecture Applications Workshop on Science Agency Uses of Clouds and Grids Orbiter Series Service Oriented Architecture Applications Orbiter Project Overview Mark L. Green mlgreen@txcorp.com Tech-X Corporation, Buffalo Office

More information

HADOOP AT NOKIA JOSH DEVINS, NOKIA HADOOP MEETUP, JANUARY 2011 BERLIN

HADOOP AT NOKIA JOSH DEVINS, NOKIA HADOOP MEETUP, JANUARY 2011 BERLIN HADOOP AT NOKIA JOSH DEVINS, NOKIA HADOOP MEETUP, JANUARY 2011 BERLIN Two parts: * technical setup * applications before starting Question: Hadoop experience levels from none to some to lots, and what

More information

AIP Development Process. GEOSS Architecture Implementation Pilot (AIP)

AIP Development Process. GEOSS Architecture Implementation Pilot (AIP) GEOSS Architecture Implementation Pilot (AIP) Version: 28 th January 205 Preface This document defines the development process employed on the GEOSS Architecture Implementation Pilot (AIP) an element of

More information

Microsoft Research Microsoft Azure for Research Training

Microsoft Research Microsoft Azure for Research Training Copyright 2014 Microsoft Corporation. All rights reserved. Except where otherwise noted, these materials are licensed under the terms of the Apache License, Version 2.0. You may use it according to the

More information

IaaS Federation. Contrail project. IaaS Federation! Objectives and Challenges! & SLA management in Federations 5/23/11

IaaS Federation. Contrail project. IaaS Federation! Objectives and Challenges! & SLA management in Federations 5/23/11 Cloud Computing (IV) s and SPD Course 19-20/05/2011 Massimo Coppola IaaS! Objectives and Challenges! & management in s Adapted from two presentations! by Massimo Coppola (CNR) and Lorenzo Blasi (HP) Italy)!

More information

Geoprocessing in Hybrid Clouds

Geoprocessing in Hybrid Clouds Geoprocessing in Hybrid Clouds Theodor Foerster, Bastian Baranski, Bastian Schäffer & Kristof Lange Institute for Geoinformatics, University of Münster, Germany {theodor.foerster; bastian.baranski;schaeffer;

More information

Cluster, Grid, Cloud Concepts

Cluster, Grid, Cloud Concepts Cluster, Grid, Cloud Concepts Kalaiselvan.K Contents Section 1: Cluster Section 2: Grid Section 3: Cloud Cluster An Overview Need for a Cluster Cluster categorizations A computer cluster is a group of

More information

A Performance Analysis of Distributed Indexing using Terrier

A Performance Analysis of Distributed Indexing using Terrier A Performance Analysis of Distributed Indexing using Terrier Amaury Couste Jakub Kozłowski William Martin Indexing Indexing Used by search

More information

Data Center Specific Thermal and Energy Saving Techniques

Data Center Specific Thermal and Energy Saving Techniques Data Center Specific Thermal and Energy Saving Techniques Tausif Muzaffar and Xiao Qin Department of Computer Science and Software Engineering Auburn University 1 Big Data 2 Data Centers In 2013, there

More information

DISMAR: Data Integration System for Marine Pollution and Water Quality

DISMAR: Data Integration System for Marine Pollution and Water Quality DISMAR: Data Integration System for Marine Pollution and Water Quality T. Hamre a,, S. Sandven a, É. Ó Tuama b a Nansen Environmental and Remote Sensing Center, Thormøhlensgate 47, N-5006 Bergen, Norway

More information

Getting Started Hacking on OpenNebula

Getting Started Hacking on OpenNebula LinuxTag 2013 Berlin, Germany, May 22nd Getting Started Hacking on OpenNebula Carlos Martín Project Engineer Acknowledgments The research leading to these results has received funding from Comunidad de

More information

OpenNebula Open Souce Solution for DC Virtualization. C12G Labs. Online Webinar

OpenNebula Open Souce Solution for DC Virtualization. C12G Labs. Online Webinar OpenNebula Open Souce Solution for DC Virtualization C12G Labs Online Webinar What is OpenNebula? Multi-tenancy, Elasticity and Automatic Provision on Virtualized Environments I m using virtualization/cloud,

More information

Model-driven Performance Estimation, Deployment, and Resource Management for Cloud-hosted Services

Model-driven Performance Estimation, Deployment, and Resource Management for Cloud-hosted Services DSM Workshop 2013 Model-driven Performance Estimation, Deployment, and Resource Management for Cloud-hosted Services Faruk Caglar, Kyoungho An, Shashank Shekhar, and Aniruddha Gokhale Vanderbilt University,

More information

Norwegian Satellite Earth Observation Database for Marine and Polar Research http://normap.nersc.no USE CASES

Norwegian Satellite Earth Observation Database for Marine and Polar Research http://normap.nersc.no USE CASES Norwegian Satellite Earth Observation Database for Marine and Polar Research http://normap.nersc.no USE CASES The NORMAP Project team has prepared this document to present functionality of the NORMAP portal.

More information

Agile Analytics on Extreme-Size Earth Science Data

Agile Analytics on Extreme-Size Earth Science Data Agile Analytics on Extreme-Size Earth Science Data COPERNICUS Big Data Workshop Brussels / BE, 2014-mar-14 Peter Baumann Jacobs University rasdaman GmbH p.baumann@jacobs-university.de What are the Big

More information

Denis Caromel, CEO Ac.veEon. Orchestrate and Accelerate Applica.ons. Open Source Cloud Solu.ons Hybrid Cloud: Private with Burst Capacity

Denis Caromel, CEO Ac.veEon. Orchestrate and Accelerate Applica.ons. Open Source Cloud Solu.ons Hybrid Cloud: Private with Burst Capacity Cloud computing et Virtualisation : applications au domaine de la Finance Denis Caromel, CEO Ac.veEon Orchestrate and Accelerate Applica.ons Open Source Cloud Solu.ons Hybrid Cloud: Private with Burst

More information

SCOOP Data Management: A Standards-based Distributed System for Coastal Data and Modeling

SCOOP Data Management: A Standards-based Distributed System for Coastal Data and Modeling SCOOP Data Management: A Standards-based Distributed System for Coastal Data and Modeling Helen Conover, Bruce Beaumont, Marilyn Drewry, Sara Graves, Ken Keiser, Manil Maskey, Matt Smith The University

More information

Hadoop in the Hybrid Cloud

Hadoop in the Hybrid Cloud Presented by Hortonworks and Microsoft Introduction An increasing number of enterprises are either currently using or are planning to use cloud deployment models to expand their IT infrastructure. Big

More information

ArcGIS. Server. A Complete and Integrated Server GIS

ArcGIS. Server. A Complete and Integrated Server GIS ArcGIS Server A Complete and Integrated Server GIS ArcGIS Server A Complete and Integrated Server GIS ArcGIS Server enables you to distribute maps, models, and tools to others within your organization

More information

Cloudera Manager Training: Hands-On Exercises

Cloudera Manager Training: Hands-On Exercises 201408 Cloudera Manager Training: Hands-On Exercises General Notes... 2 In- Class Preparation: Accessing Your Cluster... 3 Self- Study Preparation: Creating Your Cluster... 4 Hands- On Exercise: Working

More information

RED HAT AND HORTONWORKS: OPEN MODERN DATA ARCHITECTURE FOR THE ENTERPRISE

RED HAT AND HORTONWORKS: OPEN MODERN DATA ARCHITECTURE FOR THE ENTERPRISE WHITEPAPER RED HAT AND HORTONWORKS: OPEN MODERN DATA ARCHITECTURE FOR THE ENTERPRISE A Hortonworks and Red Hat whitepaper INTRODUCTION WHAT IS HADOOP? Apache Hadoop is an opensource technology born out

More information

WHAT S NEW IN SAS 9.4

WHAT S NEW IN SAS 9.4 WHAT S NEW IN SAS 9.4 PLATFORM, HPA & SAS GRID COMPUTING MICHAEL GODDARD CHIEF ARCHITECT SAS INSTITUTE, NEW ZEALAND SAS 9.4 WHAT S NEW IN THE PLATFORM Platform update SAS Grid Computing update Hadoop support

More information

Ganzheitliches Datenmanagement

Ganzheitliches Datenmanagement Ganzheitliches Datenmanagement für Hadoop Michael Kohs, Senior Sales Consultant @mikchaos The Problem with Big Data Projects in 2016 Relational, Mainframe Documents and Emails Data Modeler Data Scientist

More information

Cloud Models and Platforms

Cloud Models and Platforms Cloud Models and Platforms Dr. Sanjay P. Ahuja, Ph.D. 2010-14 FIS Distinguished Professor of Computer Science School of Computing, UNF A Working Definition of Cloud Computing Cloud computing is a model

More information

GEOSPATIAL SERVICE PLATFORM FOR EDUCATION AND RESEARCH

GEOSPATIAL SERVICE PLATFORM FOR EDUCATION AND RESEARCH GEOSPATIAL SERVICE PLATFORM FOR EDUCATION AND RESEARCH Jianya Gong, Huayi Wu, Wanshou Jiang, Wei Guo, Xi Zhai, Peng Yue State Key Laboratory of Information Engineering in Surveying, Mapping and Remote

More information

A New Cloud-based Deployment of Image Analysis Functionality

A New Cloud-based Deployment of Image Analysis Functionality 243 A New Cloud-based Deployment of Image Analysis Functionality Thomas BAHR 1 and Bill OKUBO 2 1 Exelis Visual Information Solutions GmbH, Gilching/Germany thomas.bahr@exelisvis.com 2 Exelis Visual Information

More information

Cloud Computing @ JPL Science Data Systems

Cloud Computing @ JPL Science Data Systems Cloud Computing @ JPL Science Data Systems Emily Law, GSAW 2011 Outline Science Data Systems (SDS) Space & Earth SDSs SDS Common Architecture Components Key Components using Cloud Computing Use Case 1:

More information

Oracle Big Data SQL Technical Update

Oracle Big Data SQL Technical Update Oracle Big Data SQL Technical Update Jean-Pierre Dijcks Oracle Redwood City, CA, USA Keywords: Big Data, Hadoop, NoSQL Databases, Relational Databases, SQL, Security, Performance Introduction This technical

More information

Protecting Big Data Data Protection Solutions for the Business Data Lake

Protecting Big Data Data Protection Solutions for the Business Data Lake White Paper Protecting Big Data Data Protection Solutions for the Business Data Lake Abstract Big Data use cases are maturing and customers are using Big Data to improve top and bottom line revenues. With

More information

Microsoft Research Windows Azure for Research Training

Microsoft Research Windows Azure for Research Training Copyright 2013 Microsoft Corporation. All rights reserved. Except where otherwise noted, these materials are licensed under the terms of the Apache License, Version 2.0. You may use it according to the

More information

Cloudera in the Public Cloud

Cloudera in the Public Cloud Cloudera in the Public Cloud Deployment Options for the Enterprise Data Hub Version: Q414-102 Table of Contents Executive Summary 3 The Case for Public Cloud 5 Public Cloud vs On-Premise 6 Public Cloud

More information

Deploying Hadoop with Manager

Deploying Hadoop with Manager Deploying Hadoop with Manager SUSE Big Data Made Easier Peter Linnell / Sales Engineer plinnell@suse.com Alejandro Bonilla / Sales Engineer abonilla@suse.com 2 Hadoop Core Components 3 Typical Hadoop Distribution

More information

Resource Scalability for Efficient Parallel Processing in Cloud

Resource Scalability for Efficient Parallel Processing in Cloud Resource Scalability for Efficient Parallel Processing in Cloud ABSTRACT Govinda.K #1, Abirami.M #2, Divya Mercy Silva.J #3 #1 SCSE, VIT University #2 SITE, VIT University #3 SITE, VIT University In the

More information

Databricks. A Primer

Databricks. A Primer Databricks A Primer Who is Databricks? Databricks vision is to empower anyone to easily build and deploy advanced analytics solutions. The company was founded by the team who created Apache Spark, a powerful

More information

Cloud computing for research

Cloud computing for research & for research Cécile Cavet cecile.cavet at apc.univ-paris7.fr Centre François Arago (FACe), Laboratoire APC, Université Paris Diderot January 11, 2015 Plan & 1 2 & 3 4 What is cloud computing? &??? Provides

More information

Data processing goes big

Data processing goes big Test report: Integration Big Data Edition Data processing goes big Dr. Götz Güttich Integration is a powerful set of tools to access, transform, move and synchronize data. With more than 450 connectors,

More information

OpenNebula Open Souce Solution for DC Virtualization

OpenNebula Open Souce Solution for DC Virtualization OSDC 2012 25 th April, Nürnberg OpenNebula Open Souce Solution for DC Virtualization Constantino Vázquez Blanco OpenNebula.org What is OpenNebula? Multi-tenancy, Elasticity and Automatic Provision on Virtualized

More information

Keystone Image Management System

Keystone Image Management System Image management solutions for satellite and airborne sensors Overview The Keystone Image Management System offers solutions that archive, catalogue, process and deliver digital images from a vast number

More information

Web-based spatio-temporal visualization and analysis of the Siberian Earth System Science Cluster (SIB-ESS-C)

Web-based spatio-temporal visualization and analysis of the Siberian Earth System Science Cluster (SIB-ESS-C) Web-based spatio-temporal visualization and analysis of the Siberian Earth System Science Cluster (SIB-ESS-C) Roman Gerlach Supervisor: Prof. C. Schmullius (Dept. of Geography, Friedrich-Schiller-University

More information

GeoNetwork, The Open Source Solution for the interoperable management of geospatial metadata

GeoNetwork, The Open Source Solution for the interoperable management of geospatial metadata GeoNetwork, The Open Source Solution for the interoperable management of geospatial metadata Ing. Simone Giannecchini, GeoSolutions Ing. Emanuele Tajariol, GeoSolutions Outline GeoNetwork Introduction

More information

Agile Infrastructure Update Monitoring

Agile Infrastructure Update Monitoring Agile Infrastructure Update Monitoring Pedro Andrade IT/GT 6 th July 2012 IT Technical Forum CERN IT Department CH-1211 Genève 23 Switzerland www.cern.ch/it Overview Introduction Motivation, Challenge,

More information

Ensuring High Service Levels for Public Cloud Deployments Keys to Effective Service Management

Ensuring High Service Levels for Public Cloud Deployments Keys to Effective Service Management Ensuring High Service Levels for Public Cloud Deployments Keys to Effective Service Management Table of Contents Executive Summary... 3 Introduction: Cloud Deployment Models... 3 Private Clouds...3 Public

More information

BIG DATA-AS-A-SERVICE

BIG DATA-AS-A-SERVICE White Paper BIG DATA-AS-A-SERVICE What Big Data is about What service providers can do with Big Data What EMC can do to help EMC Solutions Group Abstract This white paper looks at what service providers

More information

Big Data Management and Security

Big Data Management and Security Big Data Management and Security Audit Concerns and Business Risks Tami Frankenfield Sr. Director, Analytics and Enterprise Data Mercury Insurance What is Big Data? Velocity + Volume + Variety = Value

More information