Big Data at ECMWF Providing access to multi-petabyte datasets Past, present and future

Save this PDF as:
 WORD  PNG  TXT  JPG

Size: px
Start display at page:

Download "Big Data at ECMWF Providing access to multi-petabyte datasets Past, present and future"

Transcription

1 Big Data at ECMWF Providing access to multi-petabyte datasets Past, present and future Baudouin Raoult Principal Software Strategist ECMWF Slide 1

2 ECMWF An independent intergovernmental organisation established in 1975 with 20 Member States 14 Co-operating States Slide 2

3 Slide 3

4 Slide 4

5 Major assimilated datasets Surface stations Polar, infrared Radiosonde balloons Polar, microwave Aircraft Geostationary, IR Slide 5

6 The forecast process Slide 6

7 ERA-20C completed: Climate monitoring of the 20th Century Using >5% of ECMWF s computing power Assimilating billions of observations Producing 2,400 global forecasts per day Generating 1 PB of reanalysis data in 200 days Currently serving reanalysis products to 20,000 users Slide 7

8 Online catalogue, quick-looks and data European Air Quality Global atmospheric composition Surface fluxes: greenhouse gases, fires, emissions Radiation and ozone layer Slide 8

9 ECMWF products ECMWF currently receives 300 million observation from 130 sources daily. ECMWF operational models produce 13 millions fields daily, for a total of around 8 TB. 77 million products disseminated ever day, for a total of 6 TB. Slide 9

10 What is Big Data? Wikipedia: Big Data is the term for a collection of data sets so large and complex that it becomes difficult to process using on-hand database management tools or traditional data processing applications. The challenges include capture, curation, storage, search, sharing, transfer, analysis and visualization. Gartner: Big Data is high volume, high velocity, and/or high variety information assets that require new forms of processing to enable enhanced decision making, insight discovery and process optimization. (The 3 V of Big Data). I would like to add two S : Scalability and Sustainability. Slide 10

11 V is for Volume (or coping with exponential growth) Deletion of 1 PB Slide 11

12 V is for Velocity (or coping with exponential growth, part 2) ECMWF s archive grows exponentially: At ECMWF, r is around 0.5, which is a 50% increase per year The daily amount of data added to the archive grows exponentially at the same rate! In 1995, the size of the archive was increasing at a rate of 14 TB/year. In 2014, the size of the archive increases at a rate higher than 60 TB/day (with peaks at 100 TB). Slide 12

13 V is for Variety (or coping with complexity) 10T 1T X-MP/4 Y-MP/8 C90/12 C90/16 T106L19 T213L31 T106L16 VPP VPP T319L50 T319L31 VPP5000 IBM-P4 IBM-P5 IBM-P5+ IBM-P6 T319L60 T511L60 00Z EPS T799L91 Weekly Monthly T1279L91 Ensemble data assimilation 00Z 10 day FC Overlap, CalVal Vareps/Monthy 4d-Var Model errors 100G 4D-var increments SCDA 4D-Var DCDA Wave EFIs EPS PT levels DCDA Wave 10G Weekend EPS Extra fields, new gaussian grid End sensitivity Wave proba. SCDA Waves SCDA Forecast Wave 4V PT and PV levels SCDA Forecast EFIs Multi-Analysis 4D-Var Wave EPS SCDA Analysis Errors if FG, surface 1G 00Z Run 00Z Run Sensitivity Other centers OI Tubes Errors in AN and FG NCEP EPS Errors in FG Clusters EPS Waves FG Probabilities Ensemble means & stdev 100M FC Model levels SSTs Waves Chernobyl TOGA FC 3DVar 4DVar 12 Hour 4DVar DCDA Vareps/Monthy EDA FC Pressure levels 50 Members EPS EPS 15 days 10M Slide 13

14 ECMWF s Meteorological Archival and Retrieval System 28 years in existence A managed archive MARS is not a file system Users are not aware of the location of the data Retrievals are expressed in meteorological terms An archive, not a database Metadata online Data offline (automated tape library) Slide 14

15 ECMWF s Meteorological Archival and Retrieval System directly addressable objects Unique hypercube-based indexing Data is kept forever: For many studies, a dataset becomes useful once enough data has been accumulated Deleting old data in an exponentially growing archive is meaningless 200 million objects/65 TB added daily 7000 registered users 650 active users, 100 TB retrieved per day, in 1.5 million requests Slide 15

16 Scalability and sustainability Distributed service-oriented architecture, to scale out Queues and priorities to ensure quality of service and scaling with the demand Indirection is the key to scalability: Allow services to be modified/redeployed Allow data to be moved to different media/storage without any impact on users It has allowed us to migrate several times during the past decades: MVS to AIX, AIX to Linux, single server to clusters CFS to TSM to HPSS PL/I and Fortran to C/C++ Slide 16

17 Continuous evolution Change is driven by new needs and expectations Users are not domain experts anymore users expect information at Google speed ECMWF is continuously evolving its data delivery methods and services to cater for the new user requirements Interoperability is key Follow standards and governance to enable interoperability OGC, INSPIRE, ISO 19xxx series, NetCDF-CF, WMO Information System, GEOSS Provide high-level services on the data Data portals for data discovery Web services, REST APIs for data retrieval and manipulation Close-to-the-data processes Slide 17

18 Data exploring tools Slide 18

19 Interoperability Slide 19

20 The next steps The era of pushing the data to the user is coming to an end The volumes involved are too large We live in a post-pc era Users want to access data from smart devices anywhere, anytime, any device and share their results. We need to bring the user processing to the data: Cloud Computing is now mature enough to implement operational services We need to build a platform as a service (PaaS), on which to software as a service (SaaS) solutions for environmental data and products Slide 20

21 Thank you Slide 21 21

High Performance Computing for Numerical Weather Prediction

High Performance Computing for Numerical Weather Prediction High Performance Computing for Numerical Weather Prediction Isabella Weger European Centre for Medium-Range Weather Forecasts Slide 1 ECMWF ECMWF A European organisation with headquarters in the UK Established

More information

Outcomes of the CDS Technical Infrastructure Workshop

Outcomes of the CDS Technical Infrastructure Workshop Outcomes of the CDS Technical Infrastructure Workshop Baudouin Raoult Baudouin.raoult@ecmwf.int Funded by the European Union Implemented by Evaluation & QC function C3S architecture from European commission

More information

EUMETSAT DATA CENTRES AND ARCHIVE AND LONG-TERM DATA PRESERVATION

EUMETSAT DATA CENTRES AND ARCHIVE AND LONG-TERM DATA PRESERVATION Prepared by EUMETSAT Agenda Item: WGIV/5 EUMETSAT DATA CENTRES AND ARCHIVE AND LONG-TERM DATA PRESERVATION The importance of historic data and products derived from Meteorological Satellites, e.g. for

More information

Copernicus Atmosphere Monitoring Service (CAMS) Copernicus Climate Change Service (C3S)

Copernicus Atmosphere Monitoring Service (CAMS) Copernicus Climate Change Service (C3S) Vincent-Henri Peuch ECMWF, Head of Copernicus Atmosphere Monitoring Service Copernicus Atmosphere Monitoring Service (CAMS) Copernicus Climate Change Service (C3S) European Centre for Medium-Range Weather

More information

Baudouin Raoult, Iryna Rozum, Dick Dee

Baudouin Raoult, Iryna Rozum, Dick Dee ECMWF contribution to the EU funded CHARME Project: A Significant Event Viewer tool Matthew Manoussakis Baudouin Raoult, Iryna Rozum, Dick Dee 5th Workshop on the use of GIS/OGC standards in meteorology

More information

Distributed Computing. Mark Govett Global Systems Division

Distributed Computing. Mark Govett Global Systems Division Distributed Computing Mark Govett Global Systems Division Modeling Activities Prediction & Research Weather forecasts, climate prediction, earth system science Observing Systems Denial experiments Observing

More information

Web applications update

Web applications update Web applications update Current status, recent developments and future plans Cihan Sahin On behalf of Web development team cihan.sahin@ecmwf.int ECMWF June 7, 2016 Web applications ECMWF has a suite of

More information

A Binary Tree SMART CTO Webinar. Analyzing and Rationalizing Big Data in Messaging

A Binary Tree SMART CTO Webinar. Analyzing and Rationalizing Big Data in Messaging A Binary Tree SMART CTO Webinar Analyzing and Rationalizing Big Data in Messaging Who is Binary Tree? A Developer of Enterprise Email Migration Software Provides highest level of fidelity, performance

More information

DATA ACCESS AT EUMETSAT

DATA ACCESS AT EUMETSAT 1 EUM/OPS/VWG/15/793789 v1a DATA ACCESS AT EUMETSAT Copernicus Climate Data Store Workshop ECMWF 3-6 March 2015 Harald Rothfuss Overview of Presentation 1. Introduction to EUMETSAT 2. EUMETSAT Data Access

More information

Profile. Business solutions with a difference

Profile. Business solutions with a difference Profile Business solutions with a difference Overview ITeM Group was founded in 1999 and has a successful history of delivering IT solutions in Australia, New Zealand, Indonesia, China and Canada. We specialise

More information

Enabling embedded maps

Enabling embedded maps Enabling embedded maps Case: National geoportal of Finland Antti Rainio National Land Survey of Finland Trends: Web Services, Mapping Mashup, Open Source, Open Data Decentralised data s Service interfaces

More information

World Data Center for Remote Sensing of the Atmosphere, WDC-RSAT

World Data Center for Remote Sensing of the Atmosphere, WDC-RSAT World Data Center for Remote Sensing of the Atmosphere, WDC-RSAT Michael Bittner http://wdc.dlr.de Folie 1 WDC-RSAT Mission Provide a portal for free and simplified access to atmosphere related satellite

More information

Innovative Architectures For Cloud Data Centers

Innovative Architectures For Cloud Data Centers Innovative Architectures For Cloud Data Centers Cloud computing has ushered in a new era for data centers. Huawei offers efficient, flexible, and innovative cloud data centers and delivers extensive project

More information

MARS. Introduction and basic concepts

MARS. Introduction and basic concepts MARS Introduction and basic concepts Computer User Training Course 2014 Carsten Maaß User Support Slide 1 COM INTRO 2014: MARS Introduction and basic concepts 1 Contents Introduction Meteorological content

More information

Big Data in the context of Preservation and Value Adding

Big Data in the context of Preservation and Value Adding Big Data in the context of Preservation and Value Adding R. Leone, R. Cosac, I. Maggio, D. Iozzino ESRIN 06/11/2013 ESA UNCLASSIFIED Big Data Background ESA/ESRIN organized a 'Big Data from Space' event

More information

Copernicus Atmosphere Monitoring Service

Copernicus Atmosphere Monitoring Service Martin Suttie Martin.Suttie@ecmwf.int Copernicus Atmosphere Monitoring Service Atmosphere Monitoring Service MISSION Supporting the European strategy "Living well within the boundaries of Turning our planet"

More information

FACILITATION OF SATELLITE DATA EXCHANGE UNDER WMO WIS

FACILITATION OF SATELLITE DATA EXCHANGE UNDER WMO WIS CGMS40 EUMWP15 Prepared by EUMETSAT Agenda Item: WGIV/6 Discussed in WGIV FACILITATION OF SATELLITE DATA EXCHANGE UNDER WMO WIS The WMO Information System (WIS) has reached a stage where users can discover

More information

BUSINESS MANAGEMENT SUPPORT

BUSINESS MANAGEMENT SUPPORT BUSINESS MANAGEMENT SUPPORT Business disadvantages using cloud computing? Author: Maikel Mardjan info@bm-support.org 2010 BM-Support.org Foundation. All rights reserved. EXECUTIVE SUMMARY Cloud computing

More information

Tools and Services for the Long Term Preservation and Access of Digital Archives

Tools and Services for the Long Term Preservation and Access of Digital Archives Tools and Services for the Long Term Preservation and Access of Digital Archives Joseph JaJa, Mike Smorul, and Sangchul Song Institute for Advanced Computer Studies Department of Electrical and Computer

More information

Digital Forensics. Lab 10: Cloud Computing & the Future of Digital Forensics

Digital Forensics. Lab 10: Cloud Computing & the Future of Digital Forensics Digital Forensics Lab 10: Cloud Computing & the Future of Digital Forensics Today's Topics Cloud Computing Overview Applications of Cloud Computing Impact of CC to Digital Forensics Future of Digital Forensics

More information

National Snow and Ice Data Center A brief overview and data management projects

National Snow and Ice Data Center A brief overview and data management projects Cooperative Institute for Research in Environmental Sciences University of Colorado National Snow and Ice Data Center A brief overview and data management projects Walt Meier, Mark Parsons, and Mark Serreze

More information

The Copernicus Atmosphere Monitoring Service (CAMS)

The Copernicus Atmosphere Monitoring Service (CAMS) The Copernicus Atmosphere Monitoring Service (CAMS) Products, services and opportunities Vincent Henri Peuch Head of CAMS Vincent Henri.Peuch@ecmwf.int Funded by the European Union Implemented by Atmospheric

More information

Building Platform as a Service for Scientific Applications

Building Platform as a Service for Scientific Applications Building Platform as a Service for Scientific Applications Moustafa AbdelBaky moustafa@cac.rutgers.edu Rutgers Discovery Informa=cs Ins=tute (RDI 2 ) The NSF Cloud and Autonomic Compu=ng Center Department

More information

Guiding SOA Evolution through Governance From SOA 101 to Virtualization to Cloud Computing

Guiding SOA Evolution through Governance From SOA 101 to Virtualization to Cloud Computing Guiding SOA Evolution through Governance From SOA 101 to Virtualization to Cloud Computing 3-day seminar The evolution of how companies employ SOA can be broken down into three phases: the initial phase

More information

Why HSMs still have a place at ECMWF

Why HSMs still have a place at ECMWF Why HSMs still have a place at ECMWF Tape is dead! Or is it? Francis Dequenne francis.dequenne@ecmwf.int ECMWF June 2, 2015 European Centre for Medium-range Weather Forecast June 2, 2015 What is ECMWF?

More information

2014 Latin America Cloud Computing Market

2014 Latin America Cloud Computing Market 2014 Latin America Cloud Computing Market Changing the Way Companies Run their Businesses January 2015 Contents Section Slide Number Executive Summary 3 Introduction 5 Cloud Computing Business Impact 21

More information

Reference Architectures for Digital Libraries

Reference Architectures for Digital Libraries Reference Architectures for Digital Libraries Keith Rajecki Education Solutions Architect Sun Microsystems, Inc. 1 Agenda Challenges Digital Library Solution Architectures > Open Storage/Open Archive >

More information

UK-EOF Data Solutions Workshop

UK-EOF Data Solutions Workshop UK-EOF Data Solutions Workshop Breakout Session C: National Infrastructure David Lister & Liz Fox 1 Environment Research Funders Forum Contents: What do we mean by National Infrastructure? Why are we looking

More information

A Cloud Computing Based Sales Forecasting System for Small and Medium Scale Textile Industries

A Cloud Computing Based Sales Forecasting System for Small and Medium Scale Textile Industries A Cloud Computing Based Sales Forecasting System for Small and Medium Scale Textile Industries Apurva Pandey, Raj Kumar Somani Abstract As the textile market becomes progressively more competitive and

More information

INTRODUCTION TO CLOUD COMPUTING CEN483 PARALLEL AND DISTRIBUTED SYSTEMS

INTRODUCTION TO CLOUD COMPUTING CEN483 PARALLEL AND DISTRIBUTED SYSTEMS INTRODUCTION TO CLOUD COMPUTING CEN483 PARALLEL AND DISTRIBUTED SYSTEMS CLOUD COMPUTING Cloud computing is a model for enabling convenient, ondemand network access to a shared pool of configurable computing

More information

Cloud Computing. The impact for IT departments and the IT professional. by Maurice van der Woude

Cloud Computing. The impact for IT departments and the IT professional. by Maurice van der Woude Cloud Computing The impact for IT departments and the IT professional by Maurice van der Woude Cloud Computing The impact for IT departments and the IT professional Preface 3 Organizational changes 4 Moving

More information

EMC Data Protection Advisor 6.0

EMC Data Protection Advisor 6.0 White Paper EMC Data Protection Advisor 6.0 Abstract EMC Data Protection Advisor provides a comprehensive set of features to reduce the complexity of managing data protection environments, improve compliance

More information

Infrastructure as a Service

Infrastructure as a Service Infrastructure as a Service Next Generation Data Center & Cloud Paul Chen paul.chen@cskytech.com 2013 Csky Information All rights reserved. Changing More Users More Devices More Data >1 Billion More Netizen

More information

INTRODUCTION TO EUMETSAT DATA CENTRE

INTRODUCTION TO EUMETSAT DATA CENTRE Joint COSPAR and WMO Capacity Building Workshop on Satellite Remote Sensing, Water Cycle and Climate Change 29 July,2014 Tver, Russia MONITORING WEATHER AND CLIMATE FROM SPACE INTRODUCTION TO EUMETSAT

More information

WORLD METEOROLOGICAL ORGANIZATION. Introduction to. GRIB Edition1 and GRIB Edition 2

WORLD METEOROLOGICAL ORGANIZATION. Introduction to. GRIB Edition1 and GRIB Edition 2 WORLD METEOROLOGICAL ORGANIZATION Introduction to GRIB Edition1 and GRIB Edition 2 JUNE 2003 GRIB EDITION 1 and GRIB EDITION 2 HISTORY 1. The GRIB code has great advantages in comparison with the traditional

More information

Topics. Images courtesy of Majd F. Sakr or from Wikipedia unless otherwise noted.

Topics. Images courtesy of Majd F. Sakr or from Wikipedia unless otherwise noted. Cloud Computing Topics 1. What is the Cloud? 2. What is Cloud Computing? 3. Cloud Service Architectures 4. History of Cloud Computing 5. Advantages of Cloud Computing 6. Disadvantages of Cloud Computing

More information

Environment Canada Data Management Program. Paul Paciorek Corporate Services Branch May 7, 2014

Environment Canada Data Management Program. Paul Paciorek Corporate Services Branch May 7, 2014 Environment Canada Data Management Program Paul Paciorek Corporate Services Branch May 7, 2014 EC Data Management Program (ECDMP) consists of 5 foundational, incremental projects which will implement

More information

International coordination for continuity and interoperability: a CGMS perspective

International coordination for continuity and interoperability: a CGMS perspective International coordination for continuity and interoperability: a CGMS perspective Peng Zhang, CGMS WG-III Co-Chair NSMC/CMA In Cooperation with Suzanne Hilding, CGMS WG-III Co-Chair OPPA/NESDIS/NOAA 1

More information

Statistical data editing near the source using cloud computing concepts

Statistical data editing near the source using cloud computing concepts Distr. GENERAL WP.20 6 May 2011 ENGLISH ONLY UNITED NATIONS ECONOMIC COMMISSION FOR EUROPE (UNECE) CONFERENCE OF EUROPEAN STATISTICIANS EUROPEAN COMMISSION STATISTICAL OFFICE OF THE EUROPEAN UNION (EUROSTAT)

More information

Global Terrestrial Network HYDROLOGY (GTN-H)

Global Terrestrial Network HYDROLOGY (GTN-H) Terrestrial Network HYDROLOGY (GTN-H) 1 GEO GTN - H GCOS IGWCO 2 Main Objectives Make available data from existing global hydrological observation networks and enhance their value through integration Generation

More information

Cloud Computing Cluster Introduction to Cloud Computing. Rick Martin, Co-chair, Cloud Computing Cluster August 26, 2013

Cloud Computing Cluster Introduction to Cloud Computing. Rick Martin, Co-chair, Cloud Computing Cluster August 26, 2013 From Science to Solutions Cloud Computing Cluster Introduction to Cloud Computing Rick Martin, Co-chair, Cloud Computing Cluster August 26, 2013 Senior IT Strategist SAIC What is Cloud Computing? Cloud

More information

The STAR Algorithm Integration Team (AIT) Research to Operations Process

The STAR Algorithm Integration Team (AIT) Research to Operations Process The STAR Algorithm Integration Team (AIT) Research to Operations Process Presented by Tom King 1 The Problem Scientists write code that needs to go to operations, but:» Many scientists often prefer interpreted

More information

Current posture analysis is nothing without providing bilateral feedback aiming towards improving performance through appropriate remedial actions:

Current posture analysis is nothing without providing bilateral feedback aiming towards improving performance through appropriate remedial actions: NATIONAL UPPER-AIR NETWORK PERFORMANCE MONITORING, TASKS AND EXPERIENCE A. Kats, A. Naumov, A. Ivanov Central Aerological Observatory, Roshydromet 3 Pervomaiskaya Street, Dolgoprudny, 141700, Russian Federation

More information

White Paper on CLOUD COMPUTING

White Paper on CLOUD COMPUTING White Paper on CLOUD COMPUTING INDEX 1. Introduction 2. Features of Cloud Computing 3. Benefits of Cloud computing 4. Service models of Cloud Computing 5. Deployment models of Cloud Computing 6. Examples

More information

IBM Infrastructure for Long Term Digital Archiving

IBM Infrastructure for Long Term Digital Archiving IBM Infrastructure for Long Term Digital Archiving A new generation of archival storage Rudolf Hruška Information Infrastructure Leader IBM Systems & Technology Group rudolf_hruska@cz.ibm.com 2010 IBM

More information

Data Integration and long-term planning of the Observing Systems as a cross-cutting process in a NMS

Data Integration and long-term planning of the Observing Systems as a cross-cutting process in a NMS Data Integration and long-term planning of the Observing Systems as a cross-cutting process in a NMS ECAC Zurich, Setpember 15 2020 Ch. Häberli Deputy Head Climate Division/Head Meteorological Data Coordination

More information

Workload Characterization and Analysis of Storage and Bandwidth Needs of LEAD Workspace

Workload Characterization and Analysis of Storage and Bandwidth Needs of LEAD Workspace Workload Characterization and Analysis of Storage and Bandwidth Needs of LEAD Workspace Beth Plale Indiana University plale@cs.indiana.edu LEAD TR 001, V3.0 V3.0 dated January 24, 2007 V2.0 dated August

More information

ICSU/WMO World Data Center for Remote Sensing of the Atmosphere (WDC RSAT)

ICSU/WMO World Data Center for Remote Sensing of the Atmosphere (WDC RSAT) ICSU/WMO World Data Center for Remote Sensing of the Atmosphere (WDC RSAT) Beate Hildenbrand (et al.) German Aerospace Center (DLR) GAW 2009, Geneva, 05 07 May 2009 http://wdc.dlr.de WDC RSAT overview

More information

Big Data and the Earth Observation and Climate Modelling Communities: JASMIN and CEMS

Big Data and the Earth Observation and Climate Modelling Communities: JASMIN and CEMS Big Data and the Earth Observation and Climate Modelling Communities: JASMIN and CEMS Workshop on the Future of Big Data Management 27-28 June 2013 Philip Kershaw Centre for Environmental Data Archival

More information

Statistical analysis of geospatial information relevance of SDI s and international standards. Linda Peters, Esri

Statistical analysis of geospatial information relevance of SDI s and international standards. Linda Peters, Esri Statistical analysis of geospatial information relevance of SDI s and international standards Linda Peters, Esri The geospatial and statistical communities are major contributors of information used as

More information

Cloud Computing. Key Considerations for Adoption. Abstract. Ramkumar Dargha

Cloud Computing. Key Considerations for Adoption. Abstract. Ramkumar Dargha Cloud Computing Key Considerations for Adoption Ramkumar Dargha Abstract Cloud Computing technology and services have been witnessing quite a lot of attention for the past couple of years now. We believe

More information

Climate SuperComputing at NASA Goddard Space Flight Center

Climate SuperComputing at NASA Goddard Space Flight Center Climate SuperComputing at NASA Goddard Space Flight Center Fifteenth ECMWF Workshop on the Use of High Performance Computing in Meteorology Dr. W. Phillip Webster 3 October 2012 NASA Center for Climate

More information

NetApp Big Content Solutions: Agile Infrastructure for Big Data

NetApp Big Content Solutions: Agile Infrastructure for Big Data White Paper NetApp Big Content Solutions: Agile Infrastructure for Big Data Ingo Fuchs, NetApp April 2012 WP-7161 Executive Summary Enterprises are entering a new era of scale, in which the amount of data

More information

SDN/Virtualization and Cloud Computing

SDN/Virtualization and Cloud Computing SDN/Virtualization and Cloud Computing Agenda Software Define Network (SDN) Virtualization Cloud Computing Software Defined Network (SDN) What is SDN? Traditional Network and Limitations Traditional Computer

More information

August 2009. Transforming your Information Infrastructure with IBM s Storage Cloud Solution

August 2009. Transforming your Information Infrastructure with IBM s Storage Cloud Solution August 2009 Transforming your Information Infrastructure with IBM s Storage Cloud Solution Page 2 Table of Contents Executive summary... 3 Introduction... 4 A Story or three for inspiration... 6 Oops,

More information

Siemens SAP Cloud - Infrastructure as a Service

Siemens SAP Cloud - Infrastructure as a Service Siemens SAP Cloud - Infrastructure as a Service Siemens Value Proposition April 2010 Agenda 1 Challenges with traditional infrastructure setup 2 Siemens SAP Cloud solution 3 Benefits of Siemens SAP Cloud

More information

Datamation. Find the Right Cloud Computing Solution. Executive Brief. In This Paper

Datamation. Find the Right Cloud Computing Solution. Executive Brief. In This Paper Find the Right Cloud Computing Solution In This Paper There are three main cloud computing deployment models: private, public, and hybrid The true value of the cloud is achieved when the services it delivers

More information

Quality assurance plan for the Ensemble air quality re-analysis

Quality assurance plan for the Ensemble air quality re-analysis Quality assurance plan for the Ensemble air quality re-analysis re analysis chain Date: 07/2014 Authors: Laurence Reference : D112.3 ROUÏL (INERIS), Date 07/2014 Status Final Version Authors Reference

More information

CLOUD COMPUTING. Keywords: Cloud Computing, Data Centers, Utility Computing, Virtualization, IAAS, PAAS, SAAS.

CLOUD COMPUTING. Keywords: Cloud Computing, Data Centers, Utility Computing, Virtualization, IAAS, PAAS, SAAS. CLOUD COMPUTING Mr. Dhananjay Kakade CSIT, CHINCHWAD, Mr Giridhar Gundre CSIT College Chinchwad Abstract: Cloud computing is a technology that uses the internet and central remote servers to maintain data

More information

Use case River discharge modeling and validation

Use case River discharge modeling and validation Use case River discharge modeling and validation GEOSS interoperability for Weather, Ocean and Water 3 rd AfWCCIWorkshop 04-05 Feb 2013 THEME[ENV.2011.4.1.3-1]: Inter-operable integration of shared Earth

More information

Development of Earth Science Observational Data Infrastructure of Taiwan. Fang-Pang Lin National Center for High-Performance Computing, Taiwan

Development of Earth Science Observational Data Infrastructure of Taiwan. Fang-Pang Lin National Center for High-Performance Computing, Taiwan Development of Earth Science Observational Data Infrastructure of Taiwan Fang-Pang Lin National Center for High-Performance Computing, Taiwan GLIF 13, Singapore, 4 Oct 2013 The Path from Infrastructure

More information

High Performance Computing Cloud Computing. Dr. Rami YARED

High Performance Computing Cloud Computing. Dr. Rami YARED High Performance Computing Cloud Computing Dr. Rami YARED Outline High Performance Computing Parallel Computing Cloud Computing Definitions Advantages and drawbacks Cloud Computing vs Grid Computing Outline

More information

Grid Computing vs Cloud

Grid Computing vs Cloud Chapter 3 Grid Computing vs Cloud Computing 3.1 Grid Computing Grid computing [8, 23, 25] is based on the philosophy of sharing information and power, which gives us access to another type of heterogeneous

More information

WOS Cloud. ddn.com. Personal Storage for the Enterprise. DDN Solution Brief

WOS Cloud. ddn.com. Personal Storage for the Enterprise. DDN Solution Brief DDN Solution Brief Personal Storage for the Enterprise WOS Cloud Secure, Shared Drop-in File Access for Enterprise Users, Anytime and Anywhere 2011 DataDirect Networks. All Rights Reserved DDN WOS Cloud

More information

The NERC DataGrid (NDG)

The NERC DataGrid (NDG) The NERC DataGrid (NDG) Roy Lowry on behalf of the NDG, BADC and BODC. Ray Cramer, Marta Gutierrez, Kerstin Kleese Van Dam, Venkatasiva Kondapalli, Susan Latham, Bryan Lawrence, Kevin O Neill, Ag Stephens,

More information

Key Speculations & Problems faced by Cloud service user s in Today s time. Wipro Recommendation: GRC Framework for Cloud Computing

Key Speculations & Problems faced by Cloud service user s in Today s time. Wipro Recommendation: GRC Framework for Cloud Computing Contents Introduction Why GRC Assessment Benefits of Cloud computing and Problem Statement Key Speculations & Problems faced by Cloud service user s in Today s time Threats, Vulnerabilities and related

More information

Migrating NASA Archives to Disk: Challenges and Opportunities. NASA Langley Research Center Chris Harris June 2, 2015

Migrating NASA Archives to Disk: Challenges and Opportunities. NASA Langley Research Center Chris Harris June 2, 2015 Migrating NASA Archives to Disk: Challenges and Opportunities NASA Langley Research Center Chris Harris June 2, 2015 MSST 2015 Topics ASDC Who we are? What we do? Evolution of storage technologies Why

More information

Geospatial Imaging Cloud Storage Capturing the World at Scale with WOS TM. ddn.com. DDN Whitepaper. 2011 DataDirect Networks. All Rights Reserved.

Geospatial Imaging Cloud Storage Capturing the World at Scale with WOS TM. ddn.com. DDN Whitepaper. 2011 DataDirect Networks. All Rights Reserved. DDN Whitepaper Geospatial Imaging Cloud Storage Capturing the World at Scale with WOS TM Table of Contents Growth and Complexity Challenges for Geospatial Imaging 3 New Solutions to Drive Insight, Simplicity

More information

THE BRITISH LIBRARY. Unlocking The Value. The British Library s Collection Metadata Strategy 2015-2018. Page 1 of 8

THE BRITISH LIBRARY. Unlocking The Value. The British Library s Collection Metadata Strategy 2015-2018. Page 1 of 8 THE BRITISH LIBRARY Unlocking The Value The British Library s Collection Metadata Strategy 2015-2018 Page 1 of 8 Summary Our vision is that by 2020 the Library s collection metadata assets will be comprehensive,

More information

Blog: http://blogs.microsoft.co.il/blogs/applisec/

Blog: http://blogs.microsoft.co.il/blogs/applisec/ Blog: http://blogs.microsoft.co.il/blogs/applisec/ Copyright SELA software & Education Labs Ltd. 14-18 Baruch Hirsch St.Bnei Brak 51202 Israel www.sela.co.il The idea behind the cloud Basic Concepts Type

More information

Research Paper Available online at: www.ijarcsse.com A COMPARATIVE STUDY OF CLOUD COMPUTING SERVICE PROVIDERS

Research Paper Available online at: www.ijarcsse.com A COMPARATIVE STUDY OF CLOUD COMPUTING SERVICE PROVIDERS Volume 2, Issue 2, February 2012 ISSN: 2277 128X International Journal of Advanced Research in Computer Science and Software Engineering Research Paper Available online at: A COMPARATIVE STUDY OF CLOUD

More information

Information Growth: Challenge or Opportunity for the Storage User?

Information Growth: Challenge or Opportunity for the Storage User? Information Growth: Challenge or Opportunity for the Storage User? Roberto Basilio Hitachi Data Systems Roberto.Basilio @hds.com @RB_HDS Insert Custom Session QR if Desired Information growth a challenge

More information

The CLASS Cloud Access Pilot

The CLASS Cloud Access Pilot The CLASS Cloud Access Pilot 16 May 2012 Kenneth S. Casey, NODC OnBehalf of the CLASS Operations Working Group (COWG) First, some context. The NOAA National Data Centers National Oceanographic Data Center

More information

The Cloud: Searching for Meaning

The Cloud: Searching for Meaning APRIL 2012 The Cloud: Searching for Meaning Is Your Data Cloud-Ready? CGI GROUP INC. All rights reserved _experience the commitment TM Agenda Finding meaningful data in the Cloud Example 1: Prime Time

More information

118358 SUPERENSEMBLE FORECASTS WITH A SUITE OF MESOSCALE MODELS OVER THE CONTINENTAL UNITED STATES

118358 SUPERENSEMBLE FORECASTS WITH A SUITE OF MESOSCALE MODELS OVER THE CONTINENTAL UNITED STATES 118358 SUPERENSEMBLE FORECASTS WITH A SUITE OF MESOSCALE MODELS OVER THE CONTINENTAL UNITED STATES Donald F. Van Dyke III * Florida State University, Tallahassee, Florida T. N. Krishnamurti Florida State

More information

Join the Lean Wave. Asanka Abeysinghe Director, Solutions Architecture. WSO2, Inc. Friday, July 22, 11

Join the Lean Wave. Asanka Abeysinghe Director, Solutions Architecture. WSO2, Inc. Friday, July 22, 11 Join the Lean Wave Asanka Abeysinghe Director, Solutions Architecture. WSO2, Inc. 1 Asanka Abeysinghe 10 + years industry experience working on projects ranging from desktop, web applications through to

More information

Insights from 40 years of data services at NCAR s Research Data Archive. Douglas Schuster and Steven Worley

Insights from 40 years of data services at NCAR s Research Data Archive. Douglas Schuster and Steven Worley Insights from 40 years of data services at NCAR s Research Archive Douglas Schuster (schuster@ucar.edu) and Steven Worley (worley@ucar.edu) Highlights What is the? Evolution of Services User Registration

More information

Global Earth Observation Integrated Data Environment (GEO-IDE) Presentation to the Data Archiving and Access Requirements Working Group (DAARWG)

Global Earth Observation Integrated Data Environment (GEO-IDE) Presentation to the Data Archiving and Access Requirements Working Group (DAARWG) Global Earth Observation Integrated Data Environment (GEO-IDE) Presentation to the Data Archiving and Access Requirements Working Group (DAARWG) Ken McDonald Data Management Integration Architect National

More information

Cloud-based Infrastructures. Serving INSPIRE needs

Cloud-based Infrastructures. Serving INSPIRE needs Cloud-based Infrastructures Serving INSPIRE needs INSPIRE Conference 2014 Workshop Sessions Benoit BAURENS, AKKA Technologies (F) Claudio LUCCHESE, CNR (I) June 16th, 2014 This content by the InGeoCloudS

More information

TECHNOLOGY TRANSFER PRESENTS MAX DOLGICER THE NEW INTEGRATION MANIFESTO APPLICATIONS, DATA, CLOUD, MOBILE, AND THE INTERNET OF THINGS

TECHNOLOGY TRANSFER PRESENTS MAX DOLGICER THE NEW INTEGRATION MANIFESTO APPLICATIONS, DATA, CLOUD, MOBILE, AND THE INTERNET OF THINGS TECHNOLOGY TRANSFER PRESENTS MAX DOLGICER THE NEW INTEGRATION MANIFESTO APPLICATIONS, DATA, CLOUD, MOBILE, AND THE INTERNET OF THINGS DECEMBER 10-11, 2015 RESIDENZA DI RIPETTA - VIA DI RIPETTA, 231 ROME

More information

Confidence in the Cloud Five Ways to Capitalize with Symantec

Confidence in the Cloud Five Ways to Capitalize with Symantec Five Ways to Capitalize with Symantec Solution Brief: Confidence in the Cloud Confidence in the Cloud Contents Overview...............................................................................................

More information

Infrastructure as a Service (IaaS) Cloud Computing for Enterprises

<Insert Picture Here> Infrastructure as a Service (IaaS) Cloud Computing for Enterprises Infrastructure as a Service (IaaS) Cloud Computing for Enterprises Speaker Title The following is intended to outline our general product direction. It is intended for information

More information

Build and Manage Private and Hybrid Cloud. Urban Järund, Sr Regional Services Manager Nordics, Red Hat

Build and Manage Private and Hybrid Cloud. Urban Järund, Sr Regional Services Manager Nordics, Red Hat Build and Manage Private and Hybrid Cloud Urban Järund, Sr Regional Services Manager Nordics, Red Hat CLOUD DEPLOYMENT MODELS HYBRID CLOUD Interoperable combination of private and public cloud. PRIVATE

More information

Evolution of Chinese Research Data Policy

Evolution of Chinese Research Data Policy Bilateral US-China CODATA Workshop 2014 Evolution of Chinese Research Data Policy Jianhui Li(lijh@cnic.cn) Computer Network Information Center, CAS CODATA-China 25 Aug 2014 Outline Scientific Data Sharing

More information

UPnP: The Discovery & Service Layer For The Internet of Things April 2015

UPnP: The Discovery & Service Layer For The Internet of Things April 2015 UPnP: The Discovery & Service Layer For The Internet of Things April 2015 The First Chapter: The Connected Home In late 1999, the founding members of what would become the UPnP Forum started to put together

More information

Final Report - HydrometDB Belize s Climatic Database Management System. Executive Summary

Final Report - HydrometDB Belize s Climatic Database Management System. Executive Summary Executive Summary Belize s HydrometDB is a Climatic Database Management System (CDMS) that allows easy integration of multiple sources of automatic and manual stations, data quality control procedures,

More information

NIST Big Data PWG & RDA Big Data Infrastructure WG: Implementation Strategy: Best Practice Guideline for Big Data Application Development

NIST Big Data PWG & RDA Big Data Infrastructure WG: Implementation Strategy: Best Practice Guideline for Big Data Application Development NIST Big Data PWG & RDA Big Data Infrastructure WG: Implementation Strategy: Best Practice Guideline for Big Data Application Development Wo Chang wchang@nist.gov National Institute of Standards and Technology

More information

HUNGARIAN METEOROLOGICAL SERVICE, BUDAPEST

HUNGARIAN METEOROLOGICAL SERVICE, BUDAPEST WWW TECHNICAL PROGRESS REPORT ON THE GLOBAL DATA- PROCESSING AND FORECASTING SYSTEM (GDPFS), AND THE ANNUAL NUMERICAL WEATHER PREDICTION (NWP) PROGRESS REPORT FOR THE YEAR 2005 HUNGARIAN METEOROLOGICAL

More information

ARCHIVE FIRST: AN INTELLIGENT DATA ARCHIVAL STRATEGY, PART 1 OF 3

ARCHIVE FIRST: AN INTELLIGENT DATA ARCHIVAL STRATEGY, PART 1 OF 3 2013 Hitachi Data Systems Corporation. All Rights Reserved. ARCHIVE FIRST: AN INTELLIGENT DATA ARCHIVAL STRATEGY, PART 1 OF 3 PETER SJOBERG, SENIOR DIRECTOR, AMERICAS FILE, CONTENT AND CLOUD SOLUTIONS

More information

Meaning Based Computing: Managing the Avalanche of Unstructured Data

Meaning Based Computing: Managing the Avalanche of Unstructured Data Meaning Based Computing: Managing the Avalanche of Unstructured Data Bill Tolson, Director of Information Governance POWER PROTECT PROMOTE Moving away from structured data Accelerated unstructured information

More information

New Globus Features for Campuses

New Globus Features for Campuses New Globus Features for Campuses Steve Tuecke The University of Chicago Instrument 1 Researcher initiates transfer request; or requested automatically by script, science gateway Globus SaaS: Research data

More information

Hadoop in the Hybrid Cloud

Hadoop in the Hybrid Cloud Presented by Hortonworks and Microsoft Introduction An increasing number of enterprises are either currently using or are planning to use cloud deployment models to expand their IT infrastructure. Big

More information

Google Apps Overview

Google Apps Overview Google Apps Overview Agenda Messaging & collaboration challenges Why existing solutions are costly & limited Google s innovative approach The time is now to switch to the cloud Google Apps Premier Edition

More information

Microwave observations in the presence of cloud and precipitation

Microwave observations in the presence of cloud and precipitation Microwave observations in the presence of cloud and precipitation Alan Geer Thanks to: Bill Bell, Peter Bauer, Fabrizio Baordo, Niels Bormann Slide 1 ECMWF/EUMETSAT satellite course 2015: Microwave 2 Slide

More information

Successful Data Management Strategies for the Modern Data Center & Beyond

Successful Data Management Strategies for the Modern Data Center & Beyond Successful Data Management Strategies for the Modern Data Center & Beyond May 3 rd 2016 2016 COMMVAULT SYSTEMS, INC. ALL RIGHTS RESERVED. Business Dynamics and Challenges Budget Constraints Lines of Business

More information

Master of Science Advanced Software Services. Courses description

Master of Science Advanced Software Services. Courses description Master of Science Advanced Software Services Courses description Architecture of Service Oriented Information Systems The course explains the concept of the different architectural views (e.g. function

More information

CS 695 Topics in Virtualization and Cloud Computing and Storage Systems. Introduction

CS 695 Topics in Virtualization and Cloud Computing and Storage Systems. Introduction CS 695 Topics in Virtualization and Cloud Computing and Storage Systems Introduction Hot or not? source: Gartner Hype Cycle for Emerging Technologies, 2014 2 Source: http://geekandpoke.typepad.com/ 3 Cloud

More information

Astrium GEO UK Multi-Mission PDGS Facilities and Services

Astrium GEO UK Multi-Mission PDGS Facilities and Services Astrium GEO UK Multi-Mission PDGS Facilities and Services Introduction UK-MM-PAC ERS-1/2 UK-MM-PAC ENVISAT Data Hosting Activities Future Activities Infoterra Archives LTDP Preparation Collaborative Activities

More information

Metadata for Data Discovery: The NERC Data Catalogue Service. Steve Donegan

Metadata for Data Discovery: The NERC Data Catalogue Service. Steve Donegan Metadata for Data Discovery: The NERC Data Catalogue Service Steve Donegan Introduction NERC, Science and Data Centres NERC Discovery Metadata The Data Catalogue Service NERC Data Services Case study:

More information

Big Data Services at DKRZ

Big Data Services at DKRZ Big Data Services at DKRZ Michael Lautenschlager and Colleagues from DKRZ and Scientific Computing Research Group MPI-M Seminar Hamburg, March 31st, 2015 Big Data in Climate Research Big data is an all-encompassing

More information