Databases & Web Applications Lab Big Data Project A

Size: px
Start display at page:

Download "320473 Databases & Web Applications Lab 320454 Big Data Project A"

Transcription

1 Databases & Web Applications Lab Big Data Project A Instructor: Peter Baumann tel: office: room 88, Research Databases & Web Applications (P. Baumann)

2 Big Science Data [OGC Ocean Science Interoperability Experiment; image source: Mbari] 2

3 OGC Coverage Types Coverage = digital representation of space/time varying phenomenon n-d MultiSolid Coverage MultiSurface Coverage MultiCurve CoverageMultiPoint Coverage «FeatureType» Abstract Coverage Referenceable GridCoverage Grid Coverage Rectified GridCoverage 3

4 Facing the Coverage Deluge sensor feeds [OGC SWE] coverage server 4 4

5 Taming the Coverage Deluge sensor feeds [OGC SWE] coverage server 5 5

6 Let s Take a Closer Look... t Divergent access patterns for ingest and retrieval Server must mediate between access patterns 6

7 Our Research Large-Scale Scientific Information Services (L-SIS) Research Group flexible, scalable services on massive multi-dimensional scientific data Particular focus: n-d arrays Massive = multi-tb multi-pb per object Results: rasdaman array DBMS (www.rasdaman.org), demo at Geoservice standards: OGC WCS suite, ISO 9075 SQL Part 15: MDA (under work) 7

8 rasdaman: Scalable Array Analytics raster data manager : Array Database = SQL + n-d arrays select img.green[x0:x1,y0:y1] > 130 from LandsatArchive as img tile streaming architecture: scaling from laptop to cloud rasdaman Web visitors 8

9 Use Case: Satellite ImageTime Series [Diedrich et al 2001] 9

10 EarthServer Big Earth Data Analytics Up to 130 TB databases for all Earth sciences + planetary science EU FP7-INFRA, 3 years, 5.85 meur Platform: rasdaman; strictly open standards Cryospheric Science landcover mapping Airborne Science high-altitude drones Atmospheric Science climate variables Geology geological models Oceanography marine model runs + in-situ data Planetary Science Mars geology 10

11 Database Visualization select encode( struct { red: (char) s.image.b7[x0:x1,x0:x1], green: (char) s.image.b5[x0:x1,x0:x1], blue: (char) s.image.b0[x0:x1,x0:x1], alpha: (char) scale( d.elev, 20 ) }, "image/png" ) from SatImage as s, DEM as d [JacobsU, Fraunhofer 2012; [data courtesy BGS, ESA] [JacobsU, Fraunhofer; data courtesy BGS, ESA] 11

12 Parallel / Distributed Query Processing ad-hoc federation mixed hardware Dataset D select max((a.nir - A.red) / (A.nir + A.red)) - max((b.nir - B.red) / (B.nir + B.red)) - max((c.nir - C.red) / (C.nir + C.red)) - max((d.nir - D.red) / (D.nir + D.red)) from A, B, C, D Dataset C Dataset A Dataset B 12

13 Secured Archive Integration First-ever direct, ad-hoc mix from protected NASA & ESA services in OGC WCS/WCPS Web client (EarthServer + CobWeb) 13

14 Demo 14

15 Next: On-Board Query Intelligence [OPS-SAT: ESA CubeSat] Democratize direct data access [imagery courtesy ESA, NASA] 15

16 Summary Project work embedded in international projects & collaborations Present Publish 16

17 Big Picture Databases and Web Applications Fall lecture, undergrad + grad Advanced course in spring: Information Architectures Databases and Web Applications Lab Lab, grad Big Data Project A Project, grad New meeting slot: Tue 09:45, Research 1, room 88 17

18 Project Task Pick a topic Perform task planful: Spec document 20% -- Sep 26 Oct 03 Prototype 1: breakthrough implementation 20% -- Oct 17 Prototype 2: ready for benchmark 20% -- Oct 31 Benchmark results 20% -- Nov 14 Publication 10% -- Nov 28 Prototype 3: ready for handover 10% -- Dec 05 18

19 Resources rasdaman website demo Our publications Instructor: the rasdaman team 19

20 Main Evaluation Criteria complete wrt. requirements Solid engineering bug-free, project & code documentation, coding quality,... user-friendliness and appealing look&feel complexity (in absolute terms and in comparison to other teams' work) Good writeup Specification, documentation, paper (no particular order) 20

21 Project/Lab Topics -> course list -> list of topics 21

Big Data Analytics WG: Use Case Array Databases

Big Data Analytics WG: Use Case Array Databases Big Data Analytics WG: Use Case Array Databases RDA 4th Plenary 2014-sep-22, Amsterdam, The Netherlands Peter Baumann Jacobs University rasdaman GmbH baumann@rasdaman.com Structural Variety in Big Data

More information

Adding Big Earth Data Analytics to GEOSS

Adding Big Earth Data Analytics to GEOSS Research funded through EU FP7 283610 EarthServer European Scalable Earth Science Service Environment Adding Big Earth Data Analytics to GEOSS GEO IX Plenary Foz do Iguacu, 2012-nov-20 Peter Baumann, Stefano

More information

Agile Analytics on Extreme-Size Earth Science Data

Agile Analytics on Extreme-Size Earth Science Data Agile Analytics on Extreme-Size Earth Science Data COPERNICUS Big Data Workshop Brussels / BE, 2014-mar-14 Peter Baumann Jacobs University rasdaman GmbH p.baumann@jacobs-university.de What are the Big

More information

Agile Retrieval of Big Data with. EarthServer. ECMWF Visualization Week, Reading, 2015-sep-29

Agile Retrieval of Big Data with. EarthServer. ECMWF Visualization Week, Reading, 2015-sep-29 Agile Retrieval of Big Data with EarthServer ECMWF Visualization Week, Reading, 2015-sep-29 Peter Baumann Jacobs University rasdaman GmbH baumann@rasdaman.com [co-funded by EU through EarthServer, PublicaMundi]

More information

WCS as a Download Service for Big (and Small) Data

WCS as a Download Service for Big (and Small) Data WCS as a Download Service for Big (and Small) Data INSPIRE 2013 Florence, Italy, 2013-jun-25 Peter Baumann 1, Stephan Meissl 2, Alan Beccati 1 1 Jacobs University rasdaman GmbH, Bremen, Germany 2 EOX GmbH,

More information

OGC s Big Data Initiative

OGC s Big Data Initiative OGC s Big Data Initiative Open-Source Park, INTERGEO 2013 Peter Baumann Jacobs University rasdaman GmbH 1 Big Data Research @ Jacobs U Large-Scale Scientific Information Systems research group www.jacobs-university.de/lsis

More information

Handling Heterogeneous EO Datasets via the Web Coverage Processing Service

Handling Heterogeneous EO Datasets via the Web Coverage Processing Service Handling Heterogeneous EO Datasets via the Web Coverage Processing Service Piero Campalani a*, Simone Mantovani b, Alan Beccati a, a a a Jacobs University Bremen (DE) / b MEEO Srl (IT) * {p.campalani@jacobs-university.de

More information

On the Efficient Evaluation of Array Joins

On the Efficient Evaluation of Array Joins [co-funded by EU through H2020 EarthServer -2] On the Efficient Evaluation of Array Joins Big Data in the Geosciences Workshop IEEE Big Data, Santa Clara, US, 2015-oct-29 Peter Baumann, Vlad Merticariu

More information

EarthServer @ RDA. RDA: Co-chairing - Big Data IG - Geospatial IG. OGC co-chairing: - editor, Big Geo Data stds - BigData.DWG

EarthServer @ RDA. RDA: Co-chairing - Big Data IG - Geospatial IG. OGC co-chairing: - editor, Big Geo Data stds - BigData.DWG RDA: Co-chairing - Big Data IG - Geospatial IG OGC co-chairing: - editor, Big Geo Data stds - BigData.DWG EarthServer @ RDA RDA-DE/DINI Workshop, Karlsruhe, 2015-may-29 Peter Baumann Jacobs University

More information

A Big Picture for Big Data

A Big Picture for Big Data Supported by EU FP7 SCIDIP-ES, EU FP7 EarthServer A Big Picture for Big Data FOSS4G-Europe, Bremen, 2014-07-15 Peter Baumann Jacobs University rasdaman GmbH p.baumann@jacobs-university.de Our Stds Involvement

More information

Big Data Volume & velocity data management with ERDAS APOLLO. Alain Kabamba Hexagon Geospatial

Big Data Volume & velocity data management with ERDAS APOLLO. Alain Kabamba Hexagon Geospatial Big Data Volume & velocity data management with ERDAS APOLLO Alain Kabamba Hexagon Geospatial Intergraph is Part of the Hexagon Family Hexagon is dedicated to delivering actionable information through

More information

HPC technology and future architecture

HPC technology and future architecture HPC technology and future architecture Visual Analysis for Extremely Large-Scale Scientific Computing KGT2 Internal Meeting INRIA France Benoit Lange benoit.lange@inria.fr Toàn Nguyên toan.nguyen@inria.fr

More information

Elephant Meeting on Big Data Services

Elephant Meeting on Big Data Services 1 di 5 15/04/2016 14:23 Home About News Services Technology Training Contact Tweets by EarthServer_EU Elephant Meeting on Big Data Services Elephant Meeting on Big Data Services EarthServer.eu 2 di 5 15/04/2016

More information

Data Management Innovations for Massive Point Cloud, DEM, and 3D Vector Databases

<Insert Picture Here> Data Management Innovations for Massive Point Cloud, DEM, and 3D Vector Databases Data Management Innovations for Massive Point Cloud, DEM, and 3D Vector Databases Xavier Lopez, Director, Product Management 3D Data Management Technology Drivers: Challenges & Benefits

More information

The distribution of marine OpenData via distributed data networks and Web APIs. The example of ERDDAP, the message broker and data mediator from NOAA

The distribution of marine OpenData via distributed data networks and Web APIs. The example of ERDDAP, the message broker and data mediator from NOAA The distribution of marine OpenData via distributed data networks and Web APIs. The example of ERDDAP, the message broker and data mediator from NOAA Dr. Conor Delaney 9 April 2014 GeoMaritime, London

More information

NetCDF and HDF Data in ArcGIS

NetCDF and HDF Data in ArcGIS 2013 Esri International User Conference July 8 12, 2013 San Diego, California Technical Workshop NetCDF and HDF Data in ArcGIS Nawajish Noman Kevin Butler Esri UC2013. Technical Workshop. Outline NetCDF

More information

RDA PROPOSAL FOR Array Database Working Group (AD-WG) Peter Baumann, Jacobs University

RDA PROPOSAL FOR Array Database Working Group (AD-WG) Peter Baumann, Jacobs University RDA PROPOSAL FOR Array Database Working Group (AD-WG) Peter Baumann, Jacobs University Co-Chairs Two or three co-chairs (tbd) will be elected from the member body, by simple majority within AD-WG and considering

More information

VITO Centre of Image Processing

VITO Centre of Image Processing 07/11/2013 VITO Centre of Image Processing Towards an Improved Scientific Exploitation of EO Data Sources in Support of Vegetation Monitoring Erwin Goor, VITO nv Outline» Introduction» The role & and objectives

More information

Coastal Research Proposal Abstracts

Coastal Research Proposal Abstracts Coastal Research Proposal Abstracts An Automated Data Analysis, Processing, and Management System Proposal to NASA with SAIC Total Budget $15K Submitted 10/31/2008 The goal of this study is to design,

More information

GIS Initiative: Developing an atmospheric data model for GIS. Olga Wilhelmi (ESIG), Jennifer Boehnert (RAP/ESIG) and Terri Betancourt (RAP)

GIS Initiative: Developing an atmospheric data model for GIS. Olga Wilhelmi (ESIG), Jennifer Boehnert (RAP/ESIG) and Terri Betancourt (RAP) GIS Initiative: Developing an atmospheric data model for GIS Olga Wilhelmi (ESIG), Jennifer Boehnert (RAP/ESIG) and Terri Betancourt (RAP) Unidata seminar August 30, 2004 Presentation Outline Overview

More information

PART 1. Representations of atmospheric phenomena

PART 1. Representations of atmospheric phenomena PART 1 Representations of atmospheric phenomena Atmospheric data meet all of the criteria for big data : they are large (high volume), generated or captured frequently (high velocity), and represent a

More information

Development of Earth Science Observational Data Infrastructure of Taiwan. Fang-Pang Lin National Center for High-Performance Computing, Taiwan

Development of Earth Science Observational Data Infrastructure of Taiwan. Fang-Pang Lin National Center for High-Performance Computing, Taiwan Development of Earth Science Observational Data Infrastructure of Taiwan Fang-Pang Lin National Center for High-Performance Computing, Taiwan GLIF 13, Singapore, 4 Oct 2013 The Path from Infrastructure

More information

Using Message Brokering and Data Mediation to use Distributed Data Networks of Earth Science Data to Enhance Global Maritime Situational Awareness.

Using Message Brokering and Data Mediation to use Distributed Data Networks of Earth Science Data to Enhance Global Maritime Situational Awareness. Using Message Brokering and Data Mediation to use Distributed Data Networks of Earth Science Data to Enhance Global Maritime Situational Awareness. Conor Delaney, Alfredo Alessandrini & Harm Greidanus

More information

Use of OGC Sensor Web Enablement Standards in the Meteorology Domain. in partnership with

Use of OGC Sensor Web Enablement Standards in the Meteorology Domain. in partnership with Use of OGC Sensor Web Enablement Standards in the Meteorology Domain in partnership with Outline Introduction to OGC Sensor Web Enablement Standards Web services Metadata encodings SWE as front end of

More information

Databases for 3D Data Management: From Point Cloud to City Model

Databases for 3D Data Management: From Point Cloud to City Model Databases for 3D Data Management: From Point Cloud to City Model Xavier Lopez, Ph.D. Senior Director, Spatial and Graph Technologies Oracle Program Agenda Approach: Spatially-enable the Enterprise Oracle

More information

Ad-Hoc Task Force on Big Data NAC Science Committee

Ad-Hoc Task Force on Big Data NAC Science Committee Ad-Hoc Task Force on Big Data NAC Science Committee Dr. Erin Smith Executive Secretary, Big Data Task Force Strategic Integration and Management Division Science Mission Directorate Task Force Membership

More information

EED Task Order. Contract: NNG10HP02C Contractor: Raytheon Task Type:

EED Task Order. Contract: NNG10HP02C Contractor: Raytheon Task Type: EED Task Order Title: Studies CMR Phase 0 No-Cost Extension Task Number: 9 Rev 15 Originator: Marinelli Effective Date: Dec 11, 2013 ESDIS POC: Marinelli Task Estimate Cost and Maximum Available Fee Estimate

More information

Training for Big Data

Training for Big Data Training for Big Data Learnings from the CATS Workshop Raghu Ramakrishnan Technical Fellow, Microsoft Head, Big Data Engineering Head, Cloud Information Services Lab Store any kind of data What is Big

More information

CSE 544 Principles of Database Management Systems. Magdalena Balazinska (magda) Spring 2006 Lecture 1 - Class Introduction

CSE 544 Principles of Database Management Systems. Magdalena Balazinska (magda) Spring 2006 Lecture 1 - Class Introduction CSE 544 Principles of Database Management Systems Magdalena Balazinska (magda) Spring 2006 Lecture 1 - Class Introduction Outline Introductions Class overview What is the point of a database? Course Staff

More information

National Snow and Ice Data Center A brief overview and data management projects

National Snow and Ice Data Center A brief overview and data management projects Cooperative Institute for Research in Environmental Sciences University of Colorado National Snow and Ice Data Center A brief overview and data management projects Walt Meier, Mark Parsons, and Mark Serreze

More information

Big Data and Analytics: Getting Started with ArcGIS. Mike Park Erik Hoel

Big Data and Analytics: Getting Started with ArcGIS. Mike Park Erik Hoel Big Data and Analytics: Getting Started with ArcGIS Mike Park Erik Hoel Agenda Overview of big data Distributed computation User experience Data management Big data What is it? Big Data is a loosely defined

More information

Arrays in database systems, the next frontier?

Arrays in database systems, the next frontier? Arrays in database systems, the next frontier? What is an array? An array is a systematic arrangement of objects, usually in rows and columns Get(A, X, Y) => Value Set(A, X, Y)

More information

Cloud Computing and Advanced Relationship Analytics

Cloud Computing and Advanced Relationship Analytics Cloud Computing and Advanced Relationship Analytics Using Objectivity/DB to Discover the Relationships in your Data By Brian Clark Vice President, Product Management Objectivity, Inc. 408 992 7136 brian.clark@objectivity.com

More information

Big Data in the context of Preservation and Value Adding

Big Data in the context of Preservation and Value Adding Big Data in the context of Preservation and Value Adding R. Leone, R. Cosac, I. Maggio, D. Iozzino ESRIN 06/11/2013 ESA UNCLASSIFIED Big Data Background ESA/ESRIN organized a 'Big Data from Space' event

More information

Task AR-09-01a Progress and Contributions

Task AR-09-01a Progress and Contributions Doug Nebert, POC U.S. Geological Survey ddnebert@usgs.gov March 2010 Task AR-09-01a Progress and Contributions Background and scope for AR-09-01a This Task defines the minimum, common technical capabilities

More information

BIG DATA IN THE CLOUD : CHALLENGES AND OPPORTUNITIES MARY- JANE SULE & PROF. MAOZHEN LI BRUNEL UNIVERSITY, LONDON

BIG DATA IN THE CLOUD : CHALLENGES AND OPPORTUNITIES MARY- JANE SULE & PROF. MAOZHEN LI BRUNEL UNIVERSITY, LONDON BIG DATA IN THE CLOUD : CHALLENGES AND OPPORTUNITIES MARY- JANE SULE & PROF. MAOZHEN LI BRUNEL UNIVERSITY, LONDON Overview * Introduction * Multiple faces of Big Data * Challenges of Big Data * Cloud Computing

More information

GIS Databases With focused on ArcSDE

GIS Databases With focused on ArcSDE Linköpings universitet / IDA / Div. for human-centered systems GIS Databases With focused on ArcSDE Imad Abugessaisa g-imaab@ida.liu.se 20071004 1 GIS and SDBMS Geographical data is spatial data whose

More information

Helix Nebula, the Science Cloud: Potential for Earth Science Franco-British Workshop on Big Data in Science 6-7 November 2012

Helix Nebula, the Science Cloud: Potential for Earth Science Franco-British Workshop on Big Data in Science 6-7 November 2012 Helix Nebula, the Science Cloud: Potential for Earth Science 6-7 November 2012 Strategic Goal Helix Nebula, the Science Cloud is a partnership that has been created to support the massive IT requirements

More information

MEETING TOMORROW S EARTH OBSERVATION CHALLENGES _ WHITE PAPER

MEETING TOMORROW S EARTH OBSERVATION CHALLENGES _ WHITE PAPER MEETING TOMORROW S EARTH OBSERVATION CHALLENGES _ WHITE PAPER CONTENTS I. INTRODUCTION 3 II. CHALLENGES FOR TOMORROW S EARTH OBSERVATION APPLICATIONS 4 III. CONNECT, VISUALIZE, ANALYZE, ACT: LUCIAD AS

More information

INTEROPERABLE IMAGE DATA ACCESS THROUGH ARCGIS SERVER

INTEROPERABLE IMAGE DATA ACCESS THROUGH ARCGIS SERVER INTEROPERABLE IMAGE DATA ACCESS THROUGH ARCGIS SERVER Qian Liu Environmental Systems Research Institute 380 New York Street Redlands, CA92373, U.S.A - qliu@esri.com KEY WORDS: OGC, Standard, Interoperability,

More information

A New Cloud-based Deployment of Image Analysis Functionality

A New Cloud-based Deployment of Image Analysis Functionality 243 A New Cloud-based Deployment of Image Analysis Functionality Thomas BAHR 1 and Bill OKUBO 2 1 Exelis Visual Information Solutions GmbH, Gilching/Germany thomas.bahr@exelisvis.com 2 Exelis Visual Information

More information

Mr. Apichon Witayangkurn apichon@iis.u-tokyo.ac.jp Department of Civil Engineering The University of Tokyo

Mr. Apichon Witayangkurn apichon@iis.u-tokyo.ac.jp Department of Civil Engineering The University of Tokyo Sensor Network Messaging Service Hive/Hadoop Mr. Apichon Witayangkurn apichon@iis.u-tokyo.ac.jp Department of Civil Engineering The University of Tokyo Contents 1 Introduction 2 What & Why Sensor Network

More information

The USGS Landsat Big Data Challenge

The USGS Landsat Big Data Challenge The USGS Landsat Big Data Challenge Brian Sauer Engineering and Development USGS EROS bsauer@usgs.gov U.S. Department of the Interior U.S. Geological Survey USGS EROS and Landsat 2 Data Utility and Exploitation

More information

Scientific Research data archiving in Resources and Environment fields of 973 Program

Scientific Research data archiving in Resources and Environment fields of 973 Program Scientific Research data archiving in Resources and Environment fields of 973 Program Fu Xiaofeng, Dr. Dept. of Basic Research, Ministry of Science & Technology, P. R. China Outline I. Background II. Data

More information

Statistical analysis of geospatial information relevance of SDI s and international standards. Linda Peters, Esri

Statistical analysis of geospatial information relevance of SDI s and international standards. Linda Peters, Esri Statistical analysis of geospatial information relevance of SDI s and international standards Linda Peters, Esri The geospatial and statistical communities are major contributors of information used as

More information

Where is... How do I get to...

Where is... How do I get to... Big Data, Fast Data, Spatial Data Making Sense of Location Data in a Smart City Hans Viehmann Product Manager EMEA ORACLE Corporation August 19, 2015 Copyright 2014, Oracle and/or its affiliates. All rights

More information

The Arctic Observing Network and its Data Management Challenges Florence Fetterer (NSIDC/CIRES/CU), James A. Moore (NCAR/EOL), and the CADIS team

The Arctic Observing Network and its Data Management Challenges Florence Fetterer (NSIDC/CIRES/CU), James A. Moore (NCAR/EOL), and the CADIS team The Arctic Observing Network and its Data Management Challenges Florence Fetterer (NSIDC/CIRES/CU), James A. Moore (NCAR/EOL), and the CADIS team Photo courtesy Andrew Mahoney NSF Vision What is AON? a

More information

Big Data in OpenTopography

Big Data in OpenTopography Big Data in OpenTopography Vishu Nandigam San Diego Supercomputer Center NSF Big Data in Educa

More information

Sextant. Spatial Data Infrastructure for Marine Environment. C. Satra Le Bris, E. Quimbert, M. Treguer

Sextant. Spatial Data Infrastructure for Marine Environment. C. Satra Le Bris, E. Quimbert, M. Treguer Sextant On-Line information system for marine geographical information E. Quimbert, M. Bellouis, F. Lecuy, M. Treguer Centre de Bretagne BP 70, Plouzané 29280 France E-mail: sextant@ifremer.fr Sextant

More information

NASA Earth System Science: Structure and data centers

NASA Earth System Science: Structure and data centers SUPPLEMENT MATERIALS NASA Earth System Science: Structure and data centers NASA http://nasa.gov/ NASA Mission Directorates Aeronautics Research Exploration Systems Science http://nasascience.nasa.gov/

More information

GLOBAL DATA SPATIALLY INTERRELATE SYSTEM FOR SCIENTIFIC BIG DATA SPATIAL-SEAMLESS SHARING

GLOBAL DATA SPATIALLY INTERRELATE SYSTEM FOR SCIENTIFIC BIG DATA SPATIAL-SEAMLESS SHARING GLOBAL DATA SPATIALLY INTERRELATE SYSTEM FOR SCIENTIFIC BIG DATA SPATIAL-SEAMLESS SHARING Jieqing Yu a, Lixin WU b, a, c*, Yizhou Yang c, Xie Lei d, Wang He d a School of Environment Science and Spatial

More information

Norwegian Satellite Earth Observation Database for Marine and Polar Research http://normap.nersc.no USE CASES

Norwegian Satellite Earth Observation Database for Marine and Polar Research http://normap.nersc.no USE CASES Norwegian Satellite Earth Observation Database for Marine and Polar Research http://normap.nersc.no USE CASES The NORMAP Project team has prepared this document to present functionality of the NORMAP portal.

More information

Welcome to the first Workshop on Big data Open Source Systems (BOSS)

Welcome to the first Workshop on Big data Open Source Systems (BOSS) Welcome to the first Workshop on Big data Open Source Systems (BOSS) September 4th, 2015 Co-located with VLDB 2015 Tilmann Rabl Hands on Big Data 8 parallel tutorials 8 systems Open source Publicly available

More information

Use of ISO standards by NERC (a snapshot!)

Use of ISO standards by NERC (a snapshot!) Use of ISO standards by NERC (a snapshot!) Dr Andrew Woolf andrew.woolf@stfc.ac.uk STFC Rutherford Appleton Laboratory Outline NERC overview The NERC SDI Metadata Data Services Standards activities UK/EU

More information

CSE 599c Scientific Data Management. Magdalena Balazinska and Bill Howe Spring 2010 Lecture 3 Science in the Cloud

CSE 599c Scientific Data Management. Magdalena Balazinska and Bill Howe Spring 2010 Lecture 3 Science in the Cloud CSE 599c Scientific Data Management Magdalena Balazinska and Bill Howe Spring 2010 Lecture 3 Science in the Cloud References Existing Clouds Amazon Web services, Google App Engine, & Windows Azure And

More information

ExArch: Climate analytics on distributed exascale data archives Martin Juckes, V. Balaji, B.N. Lawrence, M. Lautenschlager, S. Denvil, G. Aloisio, P.

ExArch: Climate analytics on distributed exascale data archives Martin Juckes, V. Balaji, B.N. Lawrence, M. Lautenschlager, S. Denvil, G. Aloisio, P. ExArch: Climate analytics on distributed exascale data archives Martin Juckes, V. Balaji, B.N. Lawrence, M. Lautenschlager, S. Denvil, G. Aloisio, P. Kushner, D. Waliser, S. Pascoe, A. Stephens, P. Kershaw,

More information

Deploying ArcGIS for Server Using Managed Services

Deploying ArcGIS for Server Using Managed Services Deploying ArcGIS for Server Using Managed Services Andrew Sakowicz Erin Ross Sridhar Karra Agenda Introduction Program Overview - Overview - Methodology - Tools Customer Deployments - Architecture and

More information

Angelos Tzotsos IMIS Athena, Scientific & Technical Manager OSGeo Charter Member Copernicus Big Data Workshop, March 2014

Angelos Tzotsos IMIS Athena, Scientific & Technical Manager OSGeo Charter Member Copernicus Big Data Workshop, March 2014 Angelos Tzotsos IMIS Athena, Scientific & Technical Manager OSGeo Charter Member Copernicus Big Data Workshop, March 2014 Genesis of the project Consortium established on active research, commercial, and

More information

Developing Fleet and Asset Tracking Solutions with Web Maps

Developing Fleet and Asset Tracking Solutions with Web Maps Developing Fleet and Asset Tracking Solutions with Web Maps Introduction Many organizations have mobile field staff that perform business processes away from the office which include sales, service, maintenance,

More information

Insights from 40 years of data services at NCAR s Research Data Archive. Douglas Schuster and Steven Worley

Insights from 40 years of data services at NCAR s Research Data Archive. Douglas Schuster and Steven Worley Insights from 40 years of data services at NCAR s Research Archive Douglas Schuster (schuster@ucar.edu) and Steven Worley (worley@ucar.edu) Highlights What is the? Evolution of Services User Registration

More information

Scalable Distributed Service Integrity Attestation for Software-as-a-Service Clouds

Scalable Distributed Service Integrity Attestation for Software-as-a-Service Clouds Scalable Distributed Service Integrity Attestation for Software-as-a-Service Clouds ABSTRACT: Software-as-a-service (SaaS) cloud systems enable application service providers to deliver their applications

More information

A Professional Big Data Master s Program to train Computational Specialists

A Professional Big Data Master s Program to train Computational Specialists A Professional Big Data Master s Program to train Computational Specialists Anoop Sarkar, Fred Popowich, Alexandra Fedorova! School of Computing Science! Education for Employable Graduates: Critical Questions

More information

A Web services solution for Work Management Operations. Venu Kanaparthy Dr. Charles O Hara, Ph. D. Abstract

A Web services solution for Work Management Operations. Venu Kanaparthy Dr. Charles O Hara, Ph. D. Abstract A Web services solution for Work Management Operations Venu Kanaparthy Dr. Charles O Hara, Ph. D Abstract The GeoResources Institute at Mississippi State University is leveraging Spatial Technologies and

More information

NASA Earth Science Research in Data and Computational Science Technologies Report of the ESTO/AIST Big Data Study Roadmap Team September 2015

NASA Earth Science Research in Data and Computational Science Technologies Report of the ESTO/AIST Big Data Study Roadmap Team September 2015 NASA Earth Science Research in Data and Computational Science Technologies Report of the ESTO/AIST Big Data Study Roadmap Team September 2015 I. Background Over the next decade, the dramatic growth of

More information

Big Data R&D Initiative

Big Data R&D Initiative Big Data R&D Initiative Howard Wactlar CISE Directorate National Science Foundation NIST Big Data Meeting June, 2012 Image Credit: Exploratorium. The Landscape: Smart Sensing, Reasoning and Decision Environment

More information

Analyse, Collaborate and Publish Statistics for Measuring Progress in our Society using Storytelling. The most ancient of social rituals

Analyse, Collaborate and Publish Statistics for Measuring Progress in our Society using Storytelling. The most ancient of social rituals Analyse, Collaborate and Publish Statistics for Measuring Progress in our Society using Storytelling Storytelling by Professor Mikael Jern The most ancient of social rituals Agenda Massive statistics data..interest

More information

Cloud-based Geospatial Data services and analysis

Cloud-based Geospatial Data services and analysis Cloud-based Geospatial Data services and analysis Xuezhi Wang Scientific Data Center Computer Network Information Center Chinese Academy of Sciences 2014-08-25 Outlines 1 Introduction of Geospatial Data

More information

Cleveland State University

Cleveland State University Cleveland State University CIS 612 Modern Database Programming & Big Data Processing (3-0-3) Fall 2014 Section 50 Class Nbr. 2670. Tues, Thur 4:00 5:15 PM Prerequisites: CIS 505 and CIS 530. CIS 611 Preferred.

More information

A standards-based open source processing chain for ocean modeling in the GEOSS Architecture Implementation Pilot Phase 8 (AIP-8)

A standards-based open source processing chain for ocean modeling in the GEOSS Architecture Implementation Pilot Phase 8 (AIP-8) NATO Science & Technology Organization Centre for Maritime Research and Experimentation (STO-CMRE) Viale San Bartolomeo, 400 19126 La Spezia, Italy A standards-based open source processing chain for ocean

More information

Microsoft SQL Server 2012: What to Expect

Microsoft SQL Server 2012: What to Expect ASPE RESOURCE SERIES Microsoft SQL Server 2012: What to Expect Prepared for ASPE by Global Knowledge's Brian D. Egler MCITP-DBA, MCT, Real Skills. Real Results. Real IT. in partnership with Microsoft SQL

More information

CSE 544 Principles of Database Management Systems. Magdalena Balazinska (magda) Winter 2009 Lecture 1 - Class Introduction

CSE 544 Principles of Database Management Systems. Magdalena Balazinska (magda) Winter 2009 Lecture 1 - Class Introduction CSE 544 Principles of Database Management Systems Magdalena Balazinska (magda) Winter 2009 Lecture 1 - Class Introduction Outline Introductions Class overview What is the point of a db management system

More information

MyOcean Copernicus Marine Service Architecture and data access Experience

MyOcean Copernicus Marine Service Architecture and data access Experience MyOcean Copernicus Marine Service Architecture and data access Experience Sophie Besnard CLS, Toulouse, France February 2015 MyOcean Story MyOcean Challenge & Success MyOcean Service MyOcean System MyOcean

More information

ARIS 9ARIS 9.6 map and Future Directions Die nächste Generation des Geschäftsprozessmanagements

ARIS 9ARIS 9.6 map and Future Directions Die nächste Generation des Geschäftsprozessmanagements ARIS 9ARIS 9.6 map and Future Directions Die nächste Generation des Geschäftsprozessmanagements Dr. Katrina Simon ARIS Product Management 2014 Software AG. All rights reserved. ARIS @ Software AG 2M END

More information

ISQS 3358 BUSINESS INTELLIGENCE FALL 2014

ISQS 3358 BUSINESS INTELLIGENCE FALL 2014 ISQS 3358 BUSINESS INTELLIGENCE FALL 2014 Instructor: Dr. Miguel. I. Aguirre-Urreta, Ph.D. Office: BA E322 Phone: 806.834.0765 Email: miguel.aguirre-urreta@ttu.edu Office Hours Tuesdays and Thursdays from

More information

Facilitate ES Data Interoperability using the SCIDIP-ES Data Virtualization Toolkit

Facilitate ES Data Interoperability using the SCIDIP-ES Data Virtualization Toolkit Facilitate ES Data Interoperability using the SCIDIP-ES Data Virtualization Toolkit Jinsongdi Yu, Peter Baumann Shirley Crompton PV2013 05 Nov 2013 SCIDIP-ES Project An EC FP7 funded project under the

More information

Establishing and operating an Ocean Data Interoperability Platform ODIP. EU US Australia cooperation

Establishing and operating an Ocean Data Interoperability Platform ODIP. EU US Australia cooperation Establishing and operating an Ocean Data Interoperability Platform ODIP EU US Australia cooperation By Helen Glaves NERC- BGS ODIP Coordinator & Dick M.A. Schaap MARIS ODIP Technical Coordinator Supported

More information

Oklahoma s Open Source Spatial Data Clearinghouse: OKMaps

Oklahoma s Open Source Spatial Data Clearinghouse: OKMaps Oklahoma s Open Source Spatial Data Clearinghouse: OKMaps Presented by: Mike Sharp State Geographic Information Coordinator Oklahoma Office of Geographic Information MAGIC 2014 Symposium April 28-May1,

More information

NASA s Big Data Challenges in Climate Science

NASA s Big Data Challenges in Climate Science NASA s Big Data Challenges in Climate Science Tsengdar Lee, Ph.D. High-end Computing Program Manager NASA Headquarters Presented at IEEE Big Data 2014 Workshop October 29, 2014 1 2 7-km GEOS-5 Nature Run

More information

ANY THREAT, ANYWHERE, ANYTIME Scalable.Infrastructure.to.Enable.the.Warfi.ghter

ANY THREAT, ANYWHERE, ANYTIME Scalable.Infrastructure.to.Enable.the.Warfi.ghter WHITEPAPER ANY THREAT, ANYWHERE, ANYTIME Scalable.Infrastructure.to.Enable.the.Warfi.ghter THE BIG DATA CHALLENGE AND OPPORTUNITY The.proliferation,.management.and.analysis.of.intelligence.data.is.a.fast.growing.concern.

More information

One-Size-Fits-All: A DBMS Idea Whose Time has Come and Gone. Michael Stonebraker December, 2008

One-Size-Fits-All: A DBMS Idea Whose Time has Come and Gone. Michael Stonebraker December, 2008 One-Size-Fits-All: A DBMS Idea Whose Time has Come and Gone Michael Stonebraker December, 2008 DBMS Vendors (The Elephants) Sell One Size Fits All (OSFA) It s too hard for them to maintain multiple code

More information

Cloud Computing for Research Roger Barga Cloud Computing Futures, Microsoft Research

Cloud Computing for Research Roger Barga Cloud Computing Futures, Microsoft Research Cloud Computing for Research Roger Barga Cloud Computing Futures, Microsoft Research Trends: Data on an Exponential Scale Scientific data doubles every year Combination of inexpensive sensors + exponentially

More information

Levels of Archival Stewardship at the NOAA National Oceanographic Data Center: A Conceptual Model 1

Levels of Archival Stewardship at the NOAA National Oceanographic Data Center: A Conceptual Model 1 Levels of Archival Stewardship at the NOAA National Oceanographic Data Center: A Conceptual Model 1 Levels of Archival Stewardship at the NOAA National Oceanographic Data Center: A Conceptual Model Dr.

More information

Understanding. Purpose. Creating Relationships

Understanding. Purpose. Creating Relationships Welcome Understanding Purpose Creating Relationships You A Global Community of GIS Professionals Changing the World Together Applying Geography Everywhere Your WORK Showcasing Your Web Apps Vision Applying

More information

Distributed Computing. Mark Govett Global Systems Division

Distributed Computing. Mark Govett Global Systems Division Distributed Computing Mark Govett Global Systems Division Modeling Activities Prediction & Research Weather forecasts, climate prediction, earth system science Observing Systems Denial experiments Observing

More information

Cloud Computing @ JPL Science Data Systems

Cloud Computing @ JPL Science Data Systems Cloud Computing @ JPL Science Data Systems Emily Law, GSAW 2011 Outline Science Data Systems (SDS) Space & Earth SDSs SDS Common Architecture Components Key Components using Cloud Computing Use Case 1:

More information

Big Data Explained. An introduction to Big Data Science.

Big Data Explained. An introduction to Big Data Science. Big Data Explained An introduction to Big Data Science. 1 Presentation Agenda What is Big Data Why learn Big Data Who is it for How to start learning Big Data When to learn it Objective and Benefits of

More information

Web and Mobile GIS Applications Development

Web and Mobile GIS Applications Development Web and Mobile GIS Applications Development Presented by : Aamir Ali Manager Section Head (GIS Software Customization) Pakistan Space and Upper Atmosphere Research Commission (SUPARCO) Geographical Information

More information

JDSU Partners with Infobright to Help the World s Largest Communications Service Providers Ensure the Highest Quality of Service

JDSU Partners with Infobright to Help the World s Largest Communications Service Providers Ensure the Highest Quality of Service JDSU Partners with Infobright to Help the World s Largest Communications Service Providers Ensure the Highest Quality of Service Overview JDSU (NASDAQ: JDSU; and TSX: JDU) innovates and markets diverse

More information

Status of the Met Ocean Domain WG

Status of the Met Ocean Domain WG Status of the Met Ocean Domain WG Chris Little, UK Met Office Third Workshop on Use of OGC / GIS standards in Meteorology Exeter, 2010-11-15/17 Contents Some OGC Background Some Met Ocean Domain WG History

More information

CSE 544 Principles of Database Management Systems. Magdalena Balazinska (magda) Fall 2007 Lecture 1 - Class Introduction

CSE 544 Principles of Database Management Systems. Magdalena Balazinska (magda) Fall 2007 Lecture 1 - Class Introduction CSE 544 Principles of Database Management Systems Magdalena Balazinska (magda) Fall 2007 Lecture 1 - Class Introduction Outline Introductions Class overview What is the point of a db management system

More information

GRID Stream Database Management for Scientific Applications Milena Ivanova (Koparanova) and Tore Risch. IT Department, Uppsala University, Sweden

GRID Stream Database Management for Scientific Applications Milena Ivanova (Koparanova) and Tore Risch. IT Department, Uppsala University, Sweden GRID Stream Database Management for Scientific Applications Milena Ivanova (Koparanova) and Tore Risch IT Department, Uppsala University, Sweden Outline Motivation Stream Data Management Computational

More information

EUMETSAT DATA CENTRES AND ARCHIVE AND LONG-TERM DATA PRESERVATION

EUMETSAT DATA CENTRES AND ARCHIVE AND LONG-TERM DATA PRESERVATION Prepared by EUMETSAT Agenda Item: WGIV/5 EUMETSAT DATA CENTRES AND ARCHIVE AND LONG-TERM DATA PRESERVATION The importance of historic data and products derived from Meteorological Satellites, e.g. for

More information

ATMOSPHERIC SCIENCE DATA CENTER STATUS. Jonathan Gleason ASDC CERES Science Team Meeting Oct 29, 2013

ATMOSPHERIC SCIENCE DATA CENTER STATUS. Jonathan Gleason ASDC CERES Science Team Meeting Oct 29, 2013 ATMOSPHERIC SCIENCE DATA CENTER STATUS Jonathan Gleason ASDC CERES Science Team Meeting Oct 29, 2013 CERES USER METRICS CERES and FLASHFlux Archive Volume By Data Date Through September 2013 Total Volume

More information

Metadata for Data Discovery: The NERC Data Catalogue Service. Steve Donegan

Metadata for Data Discovery: The NERC Data Catalogue Service. Steve Donegan Metadata for Data Discovery: The NERC Data Catalogue Service Steve Donegan Introduction NERC, Science and Data Centres NERC Discovery Metadata The Data Catalogue Service NERC Data Services Case study:

More information

CISC 432/CMPE 432/CISC 832 Advanced Database Systems

CISC 432/CMPE 432/CISC 832 Advanced Database Systems CISC 432/CMPE 432/CISC 832 Advanced Database Systems Course Info Instructor: Patrick Martin Goodwin Hall 630 613 533 6063 martin@cs.queensu.ca Office Hours: Wednesday 11:00 1:00 or by appointment Schedule:

More information

5 Keys to Unlocking the Big Data Analytics Puzzle. Anurag Tandon Director, Product Marketing March 26, 2014

5 Keys to Unlocking the Big Data Analytics Puzzle. Anurag Tandon Director, Product Marketing March 26, 2014 5 Keys to Unlocking the Big Data Analytics Puzzle Anurag Tandon Director, Product Marketing March 26, 2014 1 A Little About Us A global footprint. A proven innovator. A leader in enterprise analytics for

More information

ArcGIS. Server. A Complete and Integrated Server GIS

ArcGIS. Server. A Complete and Integrated Server GIS ArcGIS Server A Complete and Integrated Server GIS ArcGIS Server A Complete and Integrated Server GIS ArcGIS Server enables you to distribute maps, models, and tools to others within your organization

More information

Embedded inside the database. No need for Hadoop or customcode. True real-time analytics done per transaction and in aggregate. On-the-fly linking IP

Embedded inside the database. No need for Hadoop or customcode. True real-time analytics done per transaction and in aggregate. On-the-fly linking IP Operates more like a search engine than a database Scoring and ranking IP allows for fuzzy searching Best-result candidate sets returned Contextual analytics to correctly disambiguate entities Embedded

More information