GeoSciWave: Grid services for data streaming, mining and visualization
|
|
- Magdalene King
- 8 years ago
- Views:
Transcription
1 GeoSciWave: Grid services for data streaming, mining and visualization M. Zhizhin*, A. Andreev*, A. Platonov**, A. Soldatov**, V. Velikhov**, M. Boyarsky*** and R. Nazirov*** *Geophysical Center, ***Space Research Institute, Russian Acad. Sci. **Russian Research Center Kurchatov Institute
2 3-years Grid collaboration Space Research Institute, RAS Geophysical Center, RAS CS Department, Moscow State University Russian Foundation for Basic Research SKIF-Grid, Russia-Belorussia Union DEGREE SSA, EGEE II National Geophysical Data Center, NOAA Microsoft Research, Cambridge Microsoft Research, Redmond
3 Geo e-science Grid Services Parallel data processing pipeline: network + storage + mining + visualization TeraFlow Network with GLORIAD: 1 Gb/s LAN from Chicago to Moscow to stream data and video Data Center with NOAA and MSR: parallel Active Storage storage using MS SQL Server database cluster Data Rendering Farm with MSR and EGEEIII: parallel data mining cluster for climate and space weather modeling Tiled Display Video-Wall with EVL: 12 displays, 30 MPixel
4 Active Storage, Modeling, Data Mining and Visualization Services 7LPH VHULHV * ULGV 7UDMHFWRULHV
5 Global Lambda Integrated Facility Available Advanced Network Resources GLIF is a consortium of institutions, organizations, consortia and country National Research & Education Networks who voluntarily share optical networking resources and expertise to develop the Global LambdaGrid for the advancement of scientific collaboration and discovery. Visualization courtesy of Bob Patterson, NCSA; data compilation by Maxine Brown, UIC. Source: Joe Mambrotti
6 Source: John Silvester
7 GLORIAD: 10Gb Worldwide Ring Source: Natalia Bulashova
8 USA-Russia Lightpath for Fast Data Transfer of Terabyte-sized Scientific Datasets National Center for Data Mining (NCDM) at the University of Illinois at Chicago, Geophysical Center RAS and Space Research Institute RAS have successfully moved 1.4 TB of data in 4.5 hours over a 1 Gbps lightpath between Chicago and Moscow as part of the Teraflow Network initiative Using NCDM s open-source UDP-based Data Transfer protocol (UDT), we were able to transfer the MS SQL database with SDSS astronomy catalog. The 2.5 TB database dump was compressed to 1.4 TB, split into 60 files, transferred over a 1 Gbps lightpath and then decompressed in Moscow and loaded back to MS SQL Server The SkyServer portal and the SDSS database were developed by Jim Gray at MSR and Alex Szalay at JHU. Russian language mirror now resides at in Moscow Direct Lightpath link from IKI in Moscow to NGDC NOAA in Boulder has been successfully tested
9 TeraFlow Test Bed Topology Servers w/ 1 G NICs TFS 1*1GE TFS 10 G =TeraFlow Server Netherlight Amsterdam 3 G WAN OC192/STM64 10GE LAN 10G WAN Gloriad 1 G Moscow TFS L2 TFS TFS WAN Termination L2 TFS TFS Starlight, Chicago SL E1200 Nortel HDXc IRNC GLORIAD OME Nortel HDXc SL TF Server IP addresses Are , 15, and 16 All Paths Are L1/L2 3*1 G Within 10 WAN
10 UDT parallel UDP data UDT protocol is developed by the National Center for Data Mining, UIC UDT is an application level data transport protocol for emerging distributed data intensive applications over wide area highspeed networks UDT uses UDP to transfer bulk data and it has its own reliability control and congestion control mechanisms UDT is not only for private or QoSenabled links, but also for shared networks IKI RAS has developed UDT tunneling application for HTTP/FTP/ over UDT streaming
11 Russian Mirror:
12 GLORIAD Press Release, June 4, 2007
13 National Geophysical Data Center NOAA Space Physics Interactive Data Resource - SPIDR NOAA Satellite Data Products Cloud - CLASS Comprehensive Large Array Stewardship System Microsoft Research Cambridge Environmental Scenario Search Engine: statistics of extreme weather events Microsoft Research Redmond Climate Induced Vegetation Change Analysis Tool Vegetation types map GLC days average temperature of the beginning of vegetation period Time series as a trajectory in the two-dimensional phase space (P-pressure, T-temperature)
14 National Geophysical Data Center NOAA Space Physics Interactive Data Resource - SPIDR =
15 Defense Meteorological Satellite Program, DMSP orbit section Mosaics geolocated Stable night time lights Night time lights difference
16 Gas Flares Monitoring
17 Space Weather Re-Analysis Input: ground and satellite data from SPIDR Space weather numerical models Output: high-resolution representation of the near-earth space
18 Active Storage for Common Data Model Common Data Model is an array of values of a parameter at different times on regular grids, at specified locations (point or station data) or space-time trajectories W Index space explosion Database cluster design, 1999 ODW ORQ MySQL databases, $ % Active Storage for Common Data Model 2 TB of climate history data The data had been stored on a cluster of 58 separate databases, each containing 1 year data, stored in a separate binary field for each grid point. The databases were queried sequentially or in parallel threads at the application level Parallel database cluster Performance results: 4D array lat-lon-height-time air (4D array) 1,8 1,6 1,4 MS SQL2008 single server storage Time, s 1,2 1 0,8 0,6 0,4 0, SINGLE MULTI NCEP_G NCEP_FULL Query < Space Time >
19 Environmental Data Mining and Visualization Fuzzy Logic State Space OGSA-DAI Activities Web Interface for Fuzzy Queries Open Source ESSE team JVM ant, axis, xerces, Linux/Java EsseDataResource Tomcat OGSA-DAI WS-I OGSA -DAI Activities GetXmlData Fuzzy GzipCompression GzipDecompression MySQ Database JDBC MySQL Fuzzy Engine.NET Port Open Source Framework ESSE team (common) 2.0 ESSE team (C#) Microsoft EsseDataResource IIS OGSA-DAI.NET OGSA -DAI Activities GzipDecompression GzipCompression Fuzzy GetXmlData SQL Server Database JDBC ODBC Fuzzy Engine
20 GLORIAD Press Release, August 24, 2007
21 Tiled Display Video Wall: SAGE 3.0 ported to.net Tiled Display UI client UI client SAGE Receiver SAGE Receiver SAGE Receiver SAGE Receiver FreeSpace Manager SAIL SAIL SAIL App1 App2 App3 SAGE Messages Pixel Stream Synchronization Channel
22
23 Examples of Scientific Visualization: subject areas and SAGE applications Astrophysics Images from the Space Hubble Telescope: Magic Carpet Large-scale structure models of our Universe: mediaplayer Environmental studies and remote sensing DMSP night time lights database: Magic Carpet Reanalysis of the climate history: NASA World Wind Art museums Frescos and large icons: Magic Carpet HD and 4K-video streams: mediaplayer
24 Scientific Visualization Applicatons
25 Computational Photography for Digital Art Maps St. Ferapont Monastery in Ferapontovo, XVI century
Parallel storage, mining and visualization of environmental data archives
Parallel storage, mining and visualization of environmental data archives Mikhail Zhizhin, Dmitry Medvedev, Alexey Poyda, Dmitry Mishin and Sergei Berezin Space Research Institute and Geophysical Center
More informationData Processing Solutions - A Case Study
Sector & Sphere Exploring Data Parallelism and Locality in Wide Area Networks Yunhong Gu Univ. of Illinois at Chicago Robert Grossman Univ. of Illinois at Chicago and Open Data Group Overview Cloud Computing
More informationMigrating a (Large) Science Database to the Cloud
The Sloan Digital Sky Survey Migrating a (Large) Science Database to the Cloud Ani Thakar Alex Szalay Center for Astrophysical Sciences and Institute for Data Intensive Engineering and Science (IDIES)
More informationCollaborative Exploration and Visualization of Big Data
Collaborative Exploration and Visualization of Big Data Paul Wielinga SARA Visualization Group paul.wielinga@sara.nl on the occasion of the opening of the Collaboratorium Collaborative Exploration and
More informationAstrophysics with Terabyte Datasets. Alex Szalay, JHU and Jim Gray, Microsoft Research
Astrophysics with Terabyte Datasets Alex Szalay, JHU and Jim Gray, Microsoft Research Living in an Exponential World Astronomers have a few hundred TB now 1 pixel (byte) / sq arc second ~ 4TB Multi-spectral,
More informationOnVector 2009: Topology handling in GLIF Cees de Laat!
OnVector 2009: Topology handling in GLIF Cees de Laat! GLIF.is founding member! GLIF 2008! Visualization courtesy of Bob Patterson, NCSA Data collection by Maxine Brown. Optical Exchange as Black Box!
More informationLightpath Planning and Monitoring
Lightpath Planning and Monitoring Ronald van der Pol 1, Andree Toonk 2 1 SARA, Kruislaan 415, Amsterdam, 1098 SJ, The Netherlands Tel: +31205928000, Fax: +31206683167, Email: rvdp@sara.nl 2 SARA, Kruislaan
More informationPART 1. Representations of atmospheric phenomena
PART 1 Representations of atmospheric phenomena Atmospheric data meet all of the criteria for big data : they are large (high volume), generated or captured frequently (high velocity), and represent a
More informationVisualization Techniques for Big Data on Big Displays
Visualization Techniques for Big Data on Big Displays Andrew Johnson, Jason Leigh, Luc Renambot and a whole bunch of graduate students November 10, 2004 Electronic Visualization Laboratory, UIC Established
More informationLibraries and Large Data
Libraries and Large Data Super Computing 2012 Elisabeth Long University of Chicago Library What is the Library s Interest in Big Data? Large Data and Libraries We ve Always Collected Data Intellectual
More informationFile System Design and Implementation
WAN Transfer Acceleration Product Description Functionality Interfaces Specifications Index 1 Functionality... 3 2 Integration... 3 3 Interfaces... 4 3.1 Physical Interfaces...5 3.1.1 Ethernet Network...5
More informationNext Generation Clouds, The Chameleon Cloud Testbed, and Software Defined Networking (SDN)
Next Generation Clouds, The Chameleon Cloud Testbed, and Software Defined Networking (SDN) Joe Mambretti, Director, (j-mambretti@northwestern.edu) International Center for Advanced Internet Research (www.icair.org)
More informationCross-Matching Very Large Datasets
1 Cross-Matching Very Large Datasets María A. Nieto-Santisteban, Aniruddha R. Thakar, and Alexander S. Szalay Johns Hopkins University Abstract The primary mission of the National Virtual Observatory (NVO)
More informationUDR: UDT + RSYNC. Open Source Fast File Transfer. Allison Heath University of Chicago
UDR: UDT + RSYNC Open Source Fast File Transfer Allison Heath University of Chicago Motivation for High Performance Protocols High-speed networks (10Gb/s, 40Gb/s, 100Gb/s,...) Large, distributed datasets
More informationHigh-Performance Visualization of Geographic Data
High-Performance Visualization of Geographic Data Presented by Budhendra Bhaduri Alexandre Sorokine Geographic Information Science and Technology Computational Sciences and Engineering Managed by UT-Battelle
More informationThe role of open exchanges in research networking
The role of open exchanges in research networking Project : GigaPort3 Project Year : 2011 Project Manager : Gerben van Malenstein Author(s) : Stratix Consulting Completion Date : 30-06-2011 Version : 1.0
More informationHP OO 10.X - SiteScope Monitoring Templates
HP OO Community Guides HP OO 10.X - SiteScope Monitoring Templates As with any application continuous automated monitoring is key. Monitoring is important in order to quickly identify potential issues,
More informationSpace Physics Interactive Data Resource SPIDR. Web Services Guide REST API v1,v2
Space Physics Interactive Data Resource SPIDR Web Services Guide REST API v1,v2 Version 3.0 May 2010 Contents Contents...2 Introduction...3 Data Sources...3 Data Source Interface...3 Basic REST API Summary...4
More informationAn ArrayLibraryforMS SQL Server
An ArrayLibraryforMS SQL Server Scientific requirements and an implementation László Dobos 1,2 --dobos@complex.elte.hu Alex Szalay 2, José Blakeley 3, Tamás Budavári 2, István Csabai 1,2, Dragan Tomic
More informationMinimum Requirements for Cencon 4 with Microsoft R SQL 2008 R2 Express
Minimum Requirements for Cencon 4 with Microsoft R SQL 2008 R2 Express SQL 2008 R2 Express Restrictions 1 CPU 1 GB Ram Use Limit 10 GB Database Size High Availability Options None (No Database Mirroring,
More informationinformation planning communication art place making Visual Communication Methods
information planning communication art place making 1 introduction About This Document This document was created for the College of Urban Planning and Public Affairs, University of Illinois at Chicago
More informationDistributed Data Parallel Computing: The Sector Perspective on Big Data
Distributed Data Parallel Computing: The Sector Perspective on Big Data Robert Grossman July 25, 2010 Laboratory for Advanced Computing University of Illinois at Chicago Open Data Group Institute for Genomics
More informationGLOBAL FORUM London, October 24 & 25, 2012
GLOBAL FORUM London, October 24 & 25, 2012-1 - Global Observations of Gas Flares Improving Global Observations of Gas Flares With Data From the Suomi NPP Visible Infrared Imaging Radiometer Suite (VIIRS)
More informationGlobus Striped GridFTP Framework and Server. Raj Kettimuthu, ANL and U. Chicago
Globus Striped GridFTP Framework and Server Raj Kettimuthu, ANL and U. Chicago Outline Introduction Features Motivation Architecture Globus XIO Experimental Results 3 August 2005 The Ohio State University
More informationInnovative, High-Density, Massively Scalable Packet Capture and Cyber Analytics Cluster for Enterprise Customers
Innovative, High-Density, Massively Scalable Packet Capture and Cyber Analytics Cluster for Enterprise Customers The Enterprise Packet Capture Cluster Platform is a complete solution based on a unique
More informationASKAP Science Data Archive: Users and Requirements CSIRO ASTRONOMY AND SPACE SCIENCE (CASS)
ASKAP Science Data Archive: Users and Requirements CSIRO ASTRONOMY AND SPACE SCIENCE (CASS) Jessica Chapman, Data Workshop March 2013 ASKAP Science Data Archive Talk outline Data flow in brief Some radio
More informationSawmill Log Analyzer Best Practices!! Page 1 of 6. Sawmill Log Analyzer Best Practices
Sawmill Log Analyzer Best Practices!! Page 1 of 6 Sawmill Log Analyzer Best Practices! Sawmill Log Analyzer Best Practices!! Page 2 of 6 This document describes best practices for the Sawmill universal
More informationSoftware challenges in the implementation of large surveys: the case of J-PAS
Software challenges in the implementation of large surveys: the case of J-PAS 1/21 Paulo Penteado - IAG/USP pp.penteado@gmail.com http://www.ppenteado.net/ast/pp_lsst_201204.pdf (K. Taylor) (A. Fernández-Soto)
More informationSilviu Panica, Marian Neagul, Daniela Zaharie and Dana Petcu (Romania)
Silviu Panica, Marian Neagul, Daniela Zaharie and Dana Petcu (Romania) Outline Introduction EO challenges; EO and classical/cloud computing; EO Services The computing platform Cluster -> Grid -> Cloud
More informationEstimating Firn Emissivity, from 1994 to1998, at the Ski Hi Automatic Weather Station on the West Antarctic Ice Sheet Using Passive Microwave Data
Estimating Firn Emissivity, from 1994 to1998, at the Ski Hi Automatic Weather Station on the West Antarctic Ice Sheet Using Passive Microwave Data Mentor: Dr. Malcolm LeCompte Elizabeth City State University
More informationAmerica s Most Wanted a metric to detect persistently faulty machines in Hadoop
America s Most Wanted a metric to detect persistently faulty machines in Hadoop Dhruba Borthakur and Andrew Ryan dhruba,andrewr1@facebook.com Presented at IFIP Workshop on Failure Diagnosis, Chicago June
More informationThe Information Revolution for the Enterprise
Click Jon Butts to add IBM text Software Group Integration Manufacturing Industry jon.butts@uk.ibm.com The Information Revolution for the Enterprise 2013 IBM Corporation Disclaimer IBM s statements regarding
More informationSolution for private cloud computing
The CC1 system Solution for private cloud computing 1 Outline What is CC1? Features Technical details Use cases By scientist By HEP experiment System requirements and installation How to get it? 2 What
More informationThe World-Wide Telescope, an Archetype for Online Science
The World-Wide Telescope, an Archetype for Online Science Jim Gray, Microsoft Research Alex Szalay, Johns Hopkins University June 2002 Technical Report MSR-TR-2002-75 Microsoft Research Microsoft Corporation
More information1 International Center for Advanced Internet Research at Northwestern University, 750 N. Lake Shore Drive, Suite 600, Chicago, IL 60611, USA
OpenFlow Services for Science: An International Experimental Research Network Demonstrating Multi-Domain Automatic Network Topology Discovery, Direct Dynamic Path Provisioning Using Edge Signaling and
More informationOn the Varieties of Clouds for Data Intensive Computing
On the Varieties of Clouds for Data Intensive Computing Robert L. Grossman University of Illinois at Chicago and Open Data Group Yunhong Gu University of Illinois at Chicago Abstract By a cloud we mean
More informationAstronomical Data Analysis Software & Systems XVI
Astronomical Data Analysis Software & Systems XVI 15-18 October 2006 Tucson, Arizona, USA Events ADASS XVI Today Calendar Conference Schedule Meeting Program Recent News Birds of a Feather Banquet Conference
More informationSSIS Scaling and Performance
SSIS Scaling and Performance Erik Veerman Atlanta MDF member SQL Server MVP, Microsoft MCT Mentor, Solid Quality Learning Agenda Buffers Transformation Types, Execution Trees General Optimization Techniques
More informationMinimum Requirements for Cencon 4 with Microsoft R SQL 2008 R2 Standard
Minimum Requirements for Cencon 4 with Microsoft R SQL 2008 R2 Standard SQL 2008 R2 Standard Restrictions 4 CPU 64 GB Ram Use Limit 524 PB Database Size High Availability Options Limited Database Mirroring,
More informationMicrosoft Research Worldwide Presence
Microsoft Research Worldwide Presence MSR India MSR New England Redmond Redmond, Washington Sept, 1991 San Francisco, California Jun, 1995 Cambridge, United Kingdom July, 1997 Beijing, China Nov, 1998
More informationSARA Computing & Networking Services
SARA Computing & Networking Services Paul Wielinga Business unit manager High Performance Networking wielinga@sara.nl Agenda 14.00 uur Opening Paul Wielinga 14:00 14:15 Inleiding Paul Wielinga 14:15 15:00
More informationOperationalise Predictive Analytics
Operationalise Predictive Analytics Publish SPSS, Excel and R reports online Predict online using SPSS and R models Access models and reports via Android app Organise people and content into projects Monitor
More informationHigh Performance Data-Transfers in Grid Environment using GridFTP over InfiniBand
High Performance Data-Transfers in Grid Environment using GridFTP over InfiniBand Hari Subramoni *, Ping Lai *, Raj Kettimuthu **, Dhabaleswar. K. (DK) Panda * * Computer Science and Engineering Department
More informationData Management Plan Extended Baryon Oscillation Spectroscopic Survey
Data Management Plan Extended Baryon Oscillation Spectroscopic Survey Experiment description: eboss is the cosmological component of the fourth generation of the Sloan Digital Sky Survey (SDSS-IV) located
More informationThe Collaboratorium & Remote Visualization at SARA. Tijs de Kler SARA Visualization Group (tijs.dekler@sara.nl)
The Collaboratorium & Remote Visualization at SARA Tijs de Kler SARA Visualization Group (tijs.dekler@sara.nl) The Collaboratorium! Goals Support collaboration, presentations and visualization for the
More informationInteractive, photorealistic visualization
V ISU A L I Z A T I ON C ORNE R Editors: Cláudio T. Silva, csilva@cs.utah.edu Joel E. Tohline, tohline@lsu.edu AN EXPERIMENTAL DISTRIBUTED VISUALIZATION SYSTEM FOR PETASCALE COMPUTING By Jinghua Ge, Andrei
More informationNSF IRNC Program International Deployment and Experimental Efforts with SDN in 2013
NSF IRNC Program International Deployment and Experimental Efforts with SDN in 2013 The National Science Foundation (NSF) International Research Network Connections (IRNC) program is enabling the development
More informationIntroduction to Scientific Data and Workflow Management
Introduction to Scientific Data and Workflow Management Michael Gertz gertz@ucdavis.edu Bertram Ludäscher ludaesch@ucdavis.edu Department of Computer Science University of California at Davis UC DAVIS
More informationMANAGING AND MINING THE LSST DATA SETS
MANAGING AND MINING THE LSST DATA SETS Astronomy is undergoing an exciting revolution -- a revolution in the way we probe the universe and the way we answer fundamental questions. New technology enables
More informationThe Matsu Wheel: A Cloud-based Scanning Framework for Analyzing Large Volumes of Hyperspectral Data
The Matsu Wheel: A Cloud-based Scanning Framework for Analyzing Large Volumes of Hyperspectral Data Maria Patterson, PhD Open Science Data Cloud Center for Data Intensive Science (CDIS) University of Chicago
More informationBig Data Storage Challenges for the Industrial Internet of Things
Big Data Storage Challenges for the Industrial Internet of Things Shyam V Nath Diwakar Kasibhotla SDC September, 2014 Agenda Introduction to IoT and Industrial Internet Industrial & Sensor Data Big Data
More informationCluster, Grid, Cloud Concepts
Cluster, Grid, Cloud Concepts Kalaiselvan.K Contents Section 1: Cluster Section 2: Grid Section 3: Cloud Cluster An Overview Need for a Cluster Cluster categorizations A computer cluster is a group of
More informationAustralian Virtual Observatory
Australian Virtual Observatory International Astronomical Union GA 2003 Joint Discussion 08 17th-18th July 2003 Sydney David Barnes The University of Melbourne Our take on virtual observatories bring legacy
More informationSoftware services competence in research and development activities at PSNC. Cezary Mazurek PSNC, Poland
Software services competence in research and development activities at PSNC Cezary Mazurek PSNC, Poland Workshop on Actions for Better Participation of New Member States to FP7-ICT Timişoara, 18/19-03-2010
More informationDATA/SPEC SHEET VIRTUAL MATRIX DISPLAY CONTROLLER VERSION 8
DATA/SPEC SHEET VIRTUAL MATRIX DISPLAY CONTROLLER VERSION 8 V920 - PRODUCT DESCRIPTION Virtual Matrix Display Controller The Virtual Matrix Display Controller (VMDC) is a selfcontained, matrix control
More informationLSST Resources for Data Analysis
LSST Resources for the Community Lynne Jones University of Washington/LSST 1 Data Flow Nightly Operations : (at base facility) Each 15s exposure = 6.44 GB (raw) 2x15s = 1 visit 30 TB / night Generates
More informationRemote Sensitive Image Stations and Grid Services
International Journal of Grid and Distributed Computing 23 Remote Sensing Images Data Integration Based on the Agent Service Binge Cui, Chuanmin Wang, Qiang Wang College of Information Science and Engineering,
More informationSolution for private cloud computing
The CC1 system Solution for private cloud computing 1 Outline What is CC1? Features Technical details System requirements and installation How to get it? 2 What is CC1? The CC1 system is a complete solution
More informationBig Data on Microsoft Platform
Big Data on Microsoft Platform Prepared by GJ Srinivas Corporate TEG - Microsoft Page 1 Contents 1. What is Big Data?...3 2. Characteristics of Big Data...3 3. Enter Hadoop...3 4. Microsoft Big Data Solutions...4
More informationFrequently Asked Questions
Frequently Asked Questions 1. Q: What is the Network Data Tunnel? A: Network Data Tunnel (NDT) is a software-based solution that accelerates data transfer in point-to-point or point-to-multipoint network
More informationCTS2134 Introduction to Networking. Module 07: Wide Area Networks
CTS2134 Introduction to Networking Module 07: Wide Area Networks WAN cloud Central Office (CO) Local loop WAN components Demarcation point (demarc) Consumer Premises Equipment (CPE) Channel Service Unit/Data
More informationAssignment # 1 (Cloud Computing Security)
Assignment # 1 (Cloud Computing Security) Group Members: Abdullah Abid Zeeshan Qaiser M. Umar Hayat Table of Contents Windows Azure Introduction... 4 Windows Azure Services... 4 1. Compute... 4 a) Virtual
More informationWINDOWS AZURE DATA MANAGEMENT
David Chappell October 2012 WINDOWS AZURE DATA MANAGEMENT CHOOSING THE RIGHT TECHNOLOGY Sponsored by Microsoft Corporation Copyright 2012 Chappell & Associates Contents Windows Azure Data Management: A
More informationWINDOWS AZURE DATA MANAGEMENT AND BUSINESS ANALYTICS
WINDOWS AZURE DATA MANAGEMENT AND BUSINESS ANALYTICS Managing and analyzing data in the cloud is just as important as it is anywhere else. To let you do this, Windows Azure provides a range of technologies
More informationArrays in database systems, the next frontier?
Arrays in database systems, the next frontier? What is an array? An array is a systematic arrangement of objects, usually in rows and columns Get(A, X, Y) => Value Set(A, X, Y)
More informationSMB Direct for SQL Server and Private Cloud
SMB Direct for SQL Server and Private Cloud Increased Performance, Higher Scalability and Extreme Resiliency June, 2014 Mellanox Overview Ticker: MLNX Leading provider of high-throughput, low-latency server
More informationComparing SMB Direct 3.0 performance over RoCE, InfiniBand and Ethernet. September 2014
Comparing SMB Direct 3.0 performance over RoCE, InfiniBand and Ethernet Anand Rangaswamy September 2014 Storage Developer Conference Mellanox Overview Ticker: MLNX Leading provider of high-throughput,
More informationNASA s Big Data Challenges in Climate Science
NASA s Big Data Challenges in Climate Science Tsengdar Lee, Ph.D. High-end Computing Program Manager NASA Headquarters Presented at IEEE Big Data 2014 Workshop October 29, 2014 1 2 7-km GEOS-5 Nature Run
More informationBLACKBRIDGE SATELLITE IMAGERY THROUGH CLOUD COMPUTING
BLACKBRIDGE SATELLITE IMAGERY THROUGH CLOUD COMPUTING Jason Setzer Cloud Product Manager Slide 1 THE RAPID EYE CONSTELLATION 5 Identical Satellites in same obit Up to 5 million km² collected daily 1 billion
More informationThe Grid and the Network
The Grid and the Network The UK Network Infrastructure A summary of E-Science supported Network projects in the UK Protocols Middleware for network services 1 The Grid and the Network The UK Network Infrastructure
More informationParallel Replication for MySQL in 5 Minutes or Less
Parallel Replication for MySQL in 5 Minutes or Less Featuring Tungsten Replicator Robert Hodges, CEO, Continuent About Continuent / Continuent is the leading provider of data replication and clustering
More informationUsing High Availability Technologies Lesson 12
Using High Availability Technologies Lesson 12 Skills Matrix Technology Skill Objective Domain Objective # Using Virtualization Configure Windows Server Hyper-V and virtual machines 1.3 What Is High Availability?
More informationData Mining for Data Cloud and Compute Cloud
Data Mining for Data Cloud and Compute Cloud Prof. Uzma Ali 1, Prof. Punam Khandar 2 Assistant Professor, Dept. Of Computer Application, SRCOEM, Nagpur, India 1 Assistant Professor, Dept. Of Computer Application,
More information雲 端 運 算 願 景 與 實 現 馬 維 英 博 士 微 軟 亞 洲 研 究 院 常 務 副 院 長
雲 端 運 算 願 景 與 實 現 馬 維 英 博 士 微 軟 亞 洲 研 究 院 常 務 副 院 長 Important Aspects of the Cloud Software as a Service (SaaS) Platform as a Service (PaaS) Infrastructure as a Service (IaaS) Information and Knowledge
More informationE20-385. Data Domain Specialist Exam for Implementation Engineers. Version: Demo. Page <<1/7>>
E20-385 Data Domain Specialist Exam for Implementation Engineers Version: Demo Page 11. A customer is looking for a solution to backup their 20 TB of image and video data on a Microsoft Windows
More informationHow to Choose a Storage Solution Using BLADE Switches and Fibre Channel Technology
Using BLADE Switches for iscsi Case Study Balance the need for a robust/always-on IT infrastructure while keeping costs low Choosing the right storage technology was a key priority for IT when Blade Network
More informationExperiences with MPTCP in an intercontinental multipathed OpenFlow network
Experiences with MP in an intercontinental multipathed network Ronald van der Pol, Michael Bredel, Artur Barczyk SURFnet Radboudkwartier 273 3511 CK Utrecht, The Netherlands Email: Ronald.vanderPol@SURFnet.nl
More informationThe Arctic Observing Network and its Data Management Challenges Florence Fetterer (NSIDC/CIRES/CU), James A. Moore (NCAR/EOL), and the CADIS team
The Arctic Observing Network and its Data Management Challenges Florence Fetterer (NSIDC/CIRES/CU), James A. Moore (NCAR/EOL), and the CADIS team Photo courtesy Andrew Mahoney NSF Vision What is AON? a
More informationSince 1998, AT&T invested more than $35 billion to support customer needs in data, Internet protocol (IP), local and global services.
At The AT&T Overview AT&T Corp. is the industry leader in data, voice, and video communications, serving more than 4 million businesses and 50 million consumer customers worldwide. Backed by the research
More informationBig Data Mining Services and Knowledge Discovery Applications on Clouds
Big Data Mining Services and Knowledge Discovery Applications on Clouds Domenico Talia DIMES, Università della Calabria & DtoK Lab Italy talia@dimes.unical.it Data Availability or Data Deluge? Some decades
More informationOPCNet Broker TM for Industrial Network Security and Connectivity
OPCNet Broker TM for Industrial Network Security and Connectivity Tunneling Process Data Securely Through Firewalls A Solution To OPC - DCOM Connectivity from Integration Objects Compatible for DA, HDA
More informationBig Data a threat or a chance?
Big Data a threat or a chance? Helwig Hauser University of Bergen, Dept. of Informatics Big Data What is Big Data? well, lots of data, right? we come back to this in a moment. certainly, a buzz-word but
More informationTechnology Support Services. Bob Davis Associate Director User Support Services
Technology Support Services Bob Davis Associate Director User Support Services 1 Technology Support Services NUIT Technology Support Services (TSS) helps Northwestern faculty, staff, and students use computing
More informationBig Data and Analytics: Getting Started with ArcGIS. Mike Park Erik Hoel
Big Data and Analytics: Getting Started with ArcGIS Mike Park Erik Hoel Agenda Overview of big data Distributed computation User experience Data management Big data What is it? Big Data is a loosely defined
More informationNext-Generation Networking for Science
Next-Generation Networking for Science ASCAC Presentation March 23, 2011 Program Managers Richard Carlson Thomas Ndousse Presentation
More informationL-Series LAN Provisioning Best Practices for Local Area Network Deployment. Introduction. L-Series Network Provisioning
L-Series LAN Provisioning Best Practices for Local Area Network Deployment Introduction Scope NComputing s L-series access devices connect to a host computer through an Ethernet interface and IP protocol.
More informationWorkshop on Cloud Services for File Synchronisation and Sharing 17-18 NOV 2014
Workshop on Cloud Services for File Synchronisation and Sharing 17-18 NOV 2014 Peer-to-Peer powered Sync and Share by 18 NOV 2014 PowerFolder - Background & Contact Enterprise File Sync- and Share solutions
More informationThe Open Cloud Testbed: A Wide Area Testbed for Cloud Computing Utilizing High Performance Network Services
The Open Cloud Testbed: A Wide Area Testbed for Cloud Computing Utilizing High Performance Network Services Robert Grossman 1, 2, Yunhong Gu 1, Michal Sabala 1, Collin Bennett 2, Jonathan Seidman 2 and
More informationChapter 4 Cloud Computing Applications and Paradigms. Cloud Computing: Theory and Practice. 1
Chapter 4 Cloud Computing Applications and Paradigms Chapter 4 1 Contents Challenges for cloud computing. Existing cloud applications and new opportunities. Architectural styles for cloud applications.
More informationDesign and Modeling of Internet Protocols. Dmitri Loguinov March 1, 2005
Design and Modeling of Internet Protocols Dmitri Loguinov March 1, 2005 1 Agenda What is protocol scalability Why TCP does not scale Future high-speed applications AQM congestion control Other work at
More informationPrivate cloud computing advances
Building robust private cloud services infrastructures By Brian Gautreau and Gong Wang Private clouds optimize utilization and management of IT resources to heighten availability. Microsoft Private Cloud
More informationIn-Memory BigData. Summer 2012, Technology Overview
In-Memory BigData Summer 2012, Technology Overview Company Vision In-Memory Data Processing Leader: > 5 years in production > 100s of customers > Starts every 10 secs worldwide > Over 10,000,000 starts
More informationUNIDART Uniform Data Request Interface
UNIDART Uniform Data Request Interface Jürgen Seib Deutscher Wetterdienst e-mail: juergen.seib@dwd.de The main goal of UNIDART Development of a Web-based information system that allows a uniform and integrated
More informationDistributed Computing for CEPC. YAN Tian On Behalf of Distributed Computing Group, CC, IHEP for 4 th CEPC Collaboration Meeting, Sep.
Distributed Computing for CEPC YAN Tian On Behalf of Distributed Computing Group, CC, IHEP for 4 th CEPC Collaboration Meeting, Sep. 12-13, 2014 1 Outline Introduction Experience of BES-DIRAC Distributed
More informationBig Data on AWS. Services Overview. Bernie Nallamotu Principle Solutions Architect
on AWS Services Overview Bernie Nallamotu Principle Solutions Architect \ So what is it? When your data sets become so large that you have to start innovating around how to collect, store, organize, analyze
More informationAnalysis of Climatic and Environmental Changes Using CLEARS Web-GIS Information-Computational System: Siberia Case Study
Analysis of Climatic and Environmental Changes Using CLEARS Web-GIS Information-Computational System: Siberia Case Study A G Titov 1,2, E P Gordov 1,2, I G Okladnikov 1,2, T M Shulgina 1 1 Institute of
More informationThe Virtual Observatory: What is it and how can it help me? Enrique Solano LAEFF / INTA Spanish Virtual Observatory
The Virtual Observatory: What is it and how can it help me? Enrique Solano LAEFF / INTA Spanish Virtual Observatory Astronomy in the XXI century The Internet revolution (the dot com boom ) has transformed
More informationCloud Computing. Lecture 24 Cloud Platform Comparison 2014-2015
Cloud Computing Lecture 24 Cloud Platform Comparison 2014-2015 1 Up until now Introduction, Definition of Cloud Computing Pre-Cloud Large Scale Computing: Grid Computing Content Distribution Networks Cycle-Sharing
More informationThe Astronomical Data Warehouse
The Astronomical Data Warehouse Clive G. Page Department of Physics & Astronomy, University of Leicester, Leicester, LE1 7RH, UK Abstract. The idea of the astronomical data warehouse has arisen as the
More informationBroadband Marshall University 2013 Update
Broadband Marshall University 2013 Update Broadband Infrastructure Marshall University s Campus Network, MUnet, is a state-of-the-art 10 Gb Switched Ethernet based backbone network linking all buildings
More information