ASKAP Science Data Archive: Users and Requirements CSIRO ASTRONOMY AND SPACE SCIENCE (CASS)
|
|
- Ilene Summers
- 7 years ago
- Views:
Transcription
1 ASKAP Science Data Archive: Users and Requirements CSIRO ASTRONOMY AND SPACE SCIENCE (CASS) Jessica Chapman, Data Workshop March 2013
2 ASKAP Science Data Archive Talk outline Data flow in brief Some radio astronomy terms Archive overview ASKAP operations Science users and use cases Requirements summary
3 Murchison Radio Observatory Pawsey Centre Worldwide Central processor: Processes the raw visibilities and outputs science data products. Science Data Archive Facility: Data storage and access to science data products
4 ASKAP Central Processor Data from same set of visibilities can be passed through three pipelines for Transient, Continuum and Spectral Line imaging. In principle this allows up to three different experiments to be scheduled simultaneously.
5 latitude Digression: Some radio astronomy terms Visibilities Fourier transform Images Radio astronomy images are created using a Fourier Transform of the observed visibilities. A single image is a map of a region of sky. This may show the sky brightness or another measured parameter. An image cube is a set of images for one region covering a range of frequency (spectral) channels. longitude
6 OH spectral Line emission from an evolved star Continuum image cubes (small-n) have up to 300 frequency channels (300 x 1 MHz) Spectral line image cubes (large-n) up to 16,200 spectral channels Transient observations generate one image every 5 seconds.
7 ASKAP Data Processing levels
8 ASKAP Science Data Archive (ASDA) ASKAP responsible for storage and access to level 5 data products. Survey Science Project Teams responsible for level 6 data validation. Astronomers and their science teams responsible for generation of level 7 enhanced data products.
9 ASDA: Schematic Data Management Layer HSM Tape libraries 2 x 20 PB Disk storage > 4 PB Science Database Nodes Portal Nodes Database cache storage 24 TB Web server 1 Web Server 2 Web Server 3 Web Server 3 V O S E R V I C E S F I R E W A L L Storage ~ 50 TB International astronomy community
10 ASDA Data Flows Schematic
11 ASDA Components Hierarchical Storage Management System with: 2 x 20 PB Tape Libraries (one as backup of other) several PB disk storage Science Database Management System 4 science database nodes with ~ 50 TBytes of storage Web servers with ~ 24 TBytes for cached data storage Virtual Observatory interface to external users Workflow processes User support and training Access to High Performance Computing facilities
12 ASDA: Data Management Layer The HSM will control the transfer of data to and from storage media. An additional data management layer is needed for the science archive. This will include software for: Generation of science metadata and updates to the science database User registration and data access controls User services including web interfaces, search forms, provision of Virtual Observatory services. Workflow management of interactions between the archive, the Central Processor, HSM and users.
13 Science Data Access Access to files, tables and search results will be provided through Virtual Observatory services: Simple Image Access Protocol returns link to images/cubes identified for a given position. Can generate image cut-outs. Cone searches: Returns table results such as positions and fluxes for sources detected within an area around agiven position. Table Access Protocol: Allows for complex querying of tables. For example could return a list of detections for sources above a given flux density with negative spectral indices (slopes). Note: Radio Astronomy is less familiar with using VO services than optical astronomy. Full adoption of VO services will require a cultural shift for users.
14 ASDA Primary Data Products Product Calibrated continuum visibility data (stored as measurement sets ) Continuum (small-n) images and image cubes Spectral line (large-n) image cubes Postage stamp images (quick look images produced with fewer pixels) Continuum source detection tables Spectral line source detections Transient source detections Transient light curves Project Source Catalogues Data type File File File File Table Table Table Table Table
15 Examples: File-based Data Products Product Parameters used Data Size 12-hr calibrated continuum visibility data set Set of three continuum images (restored, residual and model) Set of four polarisation continuum image cubes Single spectral line image cube 36 beams 300 channels 4 polarisations 666 baselines 5s integration Imsize = 10,800 pix 1 pol 1 spectral channel 3 images per position 300 spectral channels 4 pols Imsize = 3,600 pix 16,200 channels 1 pol 2.2 TBytes 5.4 GBytes 556 GBytes 839 GBytes
16 ASKAP Operations Longest baseline is 6 km. 36 antennas gives 630 baselines
17 ASKAP Operations Operated as part of the Australia Telescope National Facility (ATNF) Current ATNF Facilities are: Parkes radio telescope, Australia Telescope Compact Array, Mopra radio telescope, Long Baseline Array National Facility time is allocated on basis of scientific merit and technical feasibility Data taken on ATNF facilities are owned by CSIRO All ASKAP observations taken remotely (usually from Marsfield SOC) Science data available to all astronomers, in most cases with NO proprietary period
18 ASKAP Archive Users Astronomy Group Number of individuals Member of Survey Science Project teams 350 Member of Guest Science Project team 400 General astronomy community 750+ CASS Astrophysics group 30 Supported by CASS Science Operations 30 ivec Operations ~15 Technical developers tbd
19 ASKAP Science Survey Projects (SSPs) Projects need at least 1500 hours of telescope time 10 SSPs were selected in Their science goals are drivers of ASKAP development. SSP teams will work with CASS staff on aspects of science and archive commissioning. Project teams are responsible for their data validation i.e. will assess whether the data taken are of science quality. For bad data, observations may be retaken.
20 ASKAP science: EMU (C) Evolutionary Map of the Universe A deep radio continuum (small-n) survey that will study the evolution of star formation and massive back holes in galaxies on cosmological timescales. The survey is expected to detect about 70 million sources. EMU observations and archiving Observations taken in 12-hour schedule blocks with one block per field. Visibility data stored for 4 polarisation products and 300 spectral channels. Images archived for 1 pol and 1 spectral channel. 11 images per field for full characterisation of spectrum and image quality. Source detection tables stored
21 ASKAP science: WALLABY (S) WALLABY a blind spectral line (large-n) survey for neutral hydrogen. Survey is expected to detect spectral line emission from over 500,000 galaxies. WALLABY data processing and archives Central Processor will process 97 PB of visibility data NOT archived! WALLABY images made with 1 pol and 16,200 spectral channels. 3 images per field archived (residual, restored and mode). Two sets of Postage stamp images also archived. Each field divided into 350 and 1000 sub-regions. These allow for quick looks of smaller regions.
22 ASKAP science: VAST (T) VAST a survey for variable sources and slow transients. Examples are flare stars, intermittent pulsars, X-ray binaries, variable extragalactic sources. VAST data processing and archives VAST observations will mostly be taken in a piggy-back mode, using same input data stream taken for other projects. Visibility data are averaged to ~30 channels. VAST data produce one image every five seconds. Images are searched for sources and detections written to tables. Images are NOT archived. Typically about source detections in 5 seconds. Source detections written as table rows. Images are NOT archived.
23 Survey Science Projects: File-based data sizes in ASDA SSP Type Nfields Time per field Visibility data size (PB) Image data size (PB) EMU C POSSUM C WALLABY S NA 2.1 DINGO S NA 1.3 FLASH S NA 0.04 GASKAP S NA 0.9 VAST T Small NA Notes: Data sizes for two other SSPs: CRAFT (fast transients) and COAST (pulsars) are not included here.
24 Guest Science Projects Will be allocated up to 25% of time. May be scheduled together with SSPs (using parallel data processing pipelines). Science cases not yet established. Proposals will be submitted as for other ATNF facilities on a 6-month cycle. For 750 hours per semester, can expect about 5 10 GSPs to be scheduled.
25 ASDA Example Use Cases Group General use GSP teams Notes: Expertise Level Low Medium Example Use cases (preliminary) Download small number of full-sized images Download larger numbers of postage stamp images Cone searches identify sources of interest Carry out more complex table searches Download light curve information for given source(s) Comparison of ASKAP data with data from other facilities As above plus: Some post-data processing may be requested Some data visualisation Many (but not all) requirements may be largely satisfied through web + VO services.
26 ASDA science use: Survey Science Projects Group SSP teams Expertise Level High Example Use cases (preliminary) As for the general use and: Production of final Source Catalogues High-volume data downloads for data validation Re-analysis of visibility data to generate different images with different parameters Data visualisation across large sky regions Post-analysis of image cubes: image stacking Rapid follow up observations of transient sources Most SSP teams are likely to require access to High Performance Computing and temporary data storage. HPC services are outside the scope of the CASS ASKAP project. They can be provided by ivec and other groups.
27 Preliminary ASDA requirements Essential Requirements Data ingest from CP as the data products become available. Data transfer to tape and/or disk. Indefinite data storage, with backups for all archives files, tables and associated metadata. Potential for data archive to be mirrored to other locations. Disaster recovery plans to minimise loss of data following a crisis such as fire or flood. Efficient data retrieval from tapes and/or disks. Fast access to some data products (Source catalogues, tables, some images..) Extensible and scalable: Design should allow for additional data products, increased data storage and new technologies. User registration through OPAL (as for other ATNF facilities). Validation tool for SSP teams (set metadata flag).
28 Essential Requirements (continued) User queries through web interfaces. Data retrieval across internet using VO services. Queue management for multiple user requests. Administration-level access and tools for Operations staff from CASS and ivec. Highly Desirable Requirements ASKAP provision for SSP teams to upload final source catalogues. User support and access to HPC facilities through ivec.
29 Thank you Jessica Chapman CSIRO Astronomy and Space Science t e Jessica.Chapman@csiro.au w atnf.csiro.au CSIRO ASTRONOMY AND SPACE SCIENCE
The Murchison Widefield Array Data Archive System. Chen Wu Int l Centre for Radio Astronomy Research The University of Western Australia
The Murchison Widefield Array Data Archive System Chen Wu Int l Centre for Radio Astronomy Research The University of Western Australia Agenda Dataflow Requirements Solutions & Lessons learnt Open solution
More informationAustralian Virtual Observatory
Australian Virtual Observatory International Astronomical Union GA 2003 Joint Discussion 08 17th-18th July 2003 Sydney David Barnes The University of Melbourne Our take on virtual observatories bring legacy
More informationUsing the Parkes Pulsar Data Archive
JART http://www.jart.ac.cn Using the Parkes Pulsar Data Archive J. Khoo 1, G. Hobbs 1, R. N. Manchester 1, D. Miller 2, J. Dempsey 2 1 CSIRO Astronomy and Space Science, Australia Telescope National Facility,
More informationScience Drivers for Big Data Joseph Lazio SKA Program Development Office & Jet Propulsion Laboratory, California Institute of Technology
Science Drivers for Big Data Joseph Lazio SKA Program Development Office & Jet Propulsion Laboratory, California Institute of Technology 2010 California Institute of Technology. Government sponsorship
More informationThis meeting of the AT Users Committee was held at the ATNF Headquarters from 26 th 27 th May 2010.
ATUC Report to the Director - May 2010 This meeting of the AT Users Committee was held at the ATNF Headquarters from 26 th 27 th May 2010. Attendance: Sarah Maddison (Chair), Stacy Mader (Secretary), Tara
More informationOrganization of VizieR's Catalogs Archival
Organization of VizieR's Catalogs Archival Organization of VizieR's Catalogs Archival Table of Contents Foreword...2 Environment applied to VizieR archives...3 The archive... 3 The producer...3 The user...3
More informationObserver Access to the Cherenkov Telescope Array
Observer Access to the Cherenkov Telescope Array IRAP, Toulouse, France E-mail: jknodlseder@irap.omp.eu V. Beckmann APC, Paris, France E-mail: beckmann@apc.in2p3.fr C. Boisson LUTh, Paris, France E-mail:
More informationDouglas Bock Assistant Director Operations ATUC, Oct 2011
ATNF Operations Douglas Bock Assistant Director Operations ATUC, Oct 2011 Today s program Operations focus! ATNF Operations Strategy Science Ops; Engineering Ops! Mopra observing statistics; impact; time
More informationETERNUS CS High End Unified Data Protection
ETERNUS CS High End Unified Data Protection Optimized Backup and Archiving with ETERNUS CS High End 0 Data Protection Issues addressed by ETERNUS CS HE 60% of data growth p.a. Rising back-up windows Too
More informationLSST Resources for Data Analysis
LSST Resources for the Community Lynne Jones University of Washington/LSST 1 Data Flow Nightly Operations : (at base facility) Each 15s exposure = 6.44 GB (raw) 2x15s = 1 visit 30 TB / night Generates
More informationMAST: The Mikulski Archive for Space Telescopes
MAST: The Mikulski Archive for Space Telescopes Richard L. White Space Telescope Science Institute 2015 April 1, NRC Space Science Week/CBPSS A model for open access The NASA astrophysics data archives
More informationTier 2 Nearline. As archives grow, Echo grows. Dynamically, cost-effectively and massively. What is nearline? Transfer to Tape
Tier 2 Nearline As archives grow, Echo grows. Dynamically, cost-effectively and massively. Large Scale Storage Built for Media GB Labs Echo nearline systems have the scale and performance to allow users
More informationALMA Overview. Leonardo Testi (European Southern Observatory)
ALMA Overview Leonardo Testi (European Southern Observatory) The ALMA Project Status Timeline towards Early and Full Science Operations Future Developments Atacama Large Millimeter Array At least 50x12m
More informationSoftware challenges in the implementation of large surveys: the case of J-PAS
Software challenges in the implementation of large surveys: the case of J-PAS 1/21 Paulo Penteado - IAG/USP pp.penteado@gmail.com http://www.ppenteado.net/ast/pp_lsst_201204.pdf (K. Taylor) (A. Fernández-Soto)
More informationArchival of raw and analysed radar data at EISCAT and worldwide
Archival of raw and analysed radar data at EISCAT and worldwide Carl-Fredrik Enell, EISCAT Scientific Association COOPEUS workshop and EGI-CC kickoff, 11 March 2015 C-F Enell, EISCAT Radar data archival
More informationThe Virtual Observatory: What is it and how can it help me? Enrique Solano LAEFF / INTA Spanish Virtual Observatory
The Virtual Observatory: What is it and how can it help me? Enrique Solano LAEFF / INTA Spanish Virtual Observatory Astronomy in the XXI century The Internet revolution (the dot com boom ) has transformed
More informationIBM Global Technology Services September 2007. NAS systems scale out to meet growing storage demand.
IBM Global Technology Services September 2007 NAS systems scale out to meet Page 2 Contents 2 Introduction 2 Understanding the traditional NAS role 3 Gaining NAS benefits 4 NAS shortcomings in enterprise
More informationThe PACS Software System. (A high level overview) Prepared by : E. Wieprecht, J.Schreiber, U.Klaas November,5 2007 Issue 1.
The PACS Software System (A high level overview) Prepared by : E. Wieprecht, J.Schreiber, U.Klaas November,5 2007 Issue 1.0 PICC-ME-DS-003 1. Introduction The PCSS, the PACS ICC Software System, is the
More informationSAN Conceptual and Design Basics
TECHNICAL NOTE VMware Infrastructure 3 SAN Conceptual and Design Basics VMware ESX Server can be used in conjunction with a SAN (storage area network), a specialized high speed network that connects computer
More information巨 量 資 料 分 層 儲 存 解 決 方 案
巨 量 資 料 分 層 儲 存 解 決 方 案 Lower Costs and Improve Efficiencies Cano Lei Senior Sales Consulting Manager Oracle Systems Oracle Confidential Internal/Restricted/Highly Restricted Agenda 1 2 3 Why Tiered Storage?
More informationA New Data Visualization and Analysis Tool
Title: A New Data Visualization and Analysis Tool Author: Kern Date: 22 February 2013 NRAO Doc. #: Version: 1.0 A New Data Visualization and Analysis Tool PREPARED BY ORGANIZATION DATE Jeff Kern NRAO 22
More informationLSST and the Cloud: Astro Collaboration in 2016 Tim Axelrod LSST Data Management Scientist
LSST and the Cloud: Astro Collaboration in 2016 Tim Axelrod LSST Data Management Scientist DERCAP Sydney, Australia, 2009 Overview of Presentation LSST - a large-scale Southern hemisphere optical survey
More informationLecture 5b: Data Mining. Peter Wheatley
Lecture 5b: Data Mining Peter Wheatley Data archives Most astronomical data now available via archives Raw data and high-level products usually available Data reduction software often specific to individual
More informationThe Very High Energy source catalogue at the ASI Science Data Center
The Very High Energy source catalogue at the ASI Science Data Center a,b, F. Lucarelli a,b, L. A. Antonelli a,b, P. Giommi b E-mail: alessandro.carosi@oa-roma.inaf.it a INAF, National Institute for Astrophysics,
More informationHPC and Big Data. EPCC The University of Edinburgh. Adrian Jackson Technical Architect a.jackson@epcc.ed.ac.uk
HPC and Big Data EPCC The University of Edinburgh Adrian Jackson Technical Architect a.jackson@epcc.ed.ac.uk EPCC Facilities Technology Transfer European Projects HPC Research Visitor Programmes Training
More informationALMA Science Operations
ALMA Science Operations Lars-Åke Nyman a, Paola Andreani b, John Hibbard c, Sachiko K. Okumura d a JAO, Alonso de Cordova 3107, Vitacura, Santiago, Chile b ESO, Karl-Schwarzschild-Strasse 2, D-85748 Garching
More informationCharting the Transient Radio Sky on Sub-Second Time-Scales with LOFAR
Charting the Transient Radio Sky on Sub-Second Time-Scales with LOFAR Jason Hessels (ASTRON / UvA) with the LOFAR Pulsar Working Group (Stappers, van Leeuwen, Kramer, Karastergiou, Alexov, Hassall, Noutsos,
More informationCanadian Astronomy Data Centre. Séverin Gaudet David Schade Canadian Astronomy Data Centre
Canadian Astronomy Data Centre Séverin Gaudet David Schade Canadian Astronomy Data Centre Data Activities in Astronomy Features of the astronomy data landscape Multi-wavelength datasets are increasingly
More informationEA-ARC ALMA ARCHIVE DATA USER GUIDEBOOK
EA-ARC ALMA ARCHIVE DATA USER GUIDEBOOK Prepared by James O. Chibueze NAOJ Chile Observatory Purpose of this handbook: This handbook is aimed at providing fundamental information on how and where to access
More informationHow Does the Cloud Fit into Active Archiving. Active Archive Alliance Panel
How Does the Cloud Fit into Active Archiving Active Archive Alliance Panel Active Archive Alliance Founded 2010 The Active Archive Alliance is a multi-vendor association aimed at evolving new technologies
More informationIonospheric Research with the LOFAR Telescope
Ionospheric Research with the LOFAR Telescope Leszek P. Błaszkiewicz Faculty of Mathematics and Computer Science, UWM Olsztyn LOFAR - The LOw Frequency ARray The LOFAR interferometer consist of a large
More informationA Backup Strategy for Informatics Craig Strachan Version 1.0 31/01/2011
A Backup Strategy for Informatics Craig Strachan Version 1.0 31/01/2011 This document identifies the various types of backup being carried out in the School of Informatics, briefly considers the characteristics
More informationThis work was done in collaboration with the Centre de Données astronomiques de Strasbourg (CDS) and in particular with F. X. Pineau.
Chapter 5 Database and website This work was done in collaboration with the Centre de Données astronomiques de Strasbourg (CDS) and in particular with F. X. Pineau. 5.1 Introduction The X-shooter Spectral
More informationXenData Product Brief: SX-550 Series Servers for LTO Archives
XenData Product Brief: SX-550 Series Servers for LTO Archives The SX-550 Series of Archive Servers creates highly scalable LTO Digital Video Archives that are optimized for broadcasters, video production
More informationAstrophysics with Terabyte Datasets. Alex Szalay, JHU and Jim Gray, Microsoft Research
Astrophysics with Terabyte Datasets Alex Szalay, JHU and Jim Gray, Microsoft Research Living in an Exponential World Astronomers have a few hundred TB now 1 pixel (byte) / sq arc second ~ 4TB Multi-spectral,
More informationData Mining Challenges and Opportunities in Astronomy
Data Mining Challenges and Opportunities in Astronomy S. G. Djorgovski (Caltech) With special thanks to R. Brunner, A. Szalay, A. Mahabal, et al. The Punchline: Astronomy has become an immensely datarich
More informationUNINETT Sigma2 AS: architecture and functionality of the future national data infrastructure
UNINETT Sigma2 AS: architecture and functionality of the future national data infrastructure Authors: A O Jaunsen, G S Dahiya, H A Eide, E Midttun Date: Dec 15, 2015 Summary Uninett Sigma2 provides High
More informationALMA Operations the role of the European ALMA Regional Centre Plans for cycle 1
ALMA Operations the role of the European ALMA Regional Centre Plans for cycle 1 Paola Andreani ALMA Operations Sites in Chile Antenna Operations Site (AOS) Operation Support Facility (OSF) 60 MB/s (peak)
More informationDisk-to-Disk-to-Offsite Backups for SMBs with Retrospect
Disk-to-Disk-to-Offsite Backups for SMBs with Retrospect Abstract Retrospect backup and recovery software provides a quick, reliable, easy-to-manage disk-to-disk-to-offsite backup solution for SMBs. Use
More informationA Preliminary Summary of The VLA Sky Survey
A Preliminary Summary of The VLA Sky Survey Eric J. Murphy and Stefi Baum (On behalf of the entire Science Survey Group) 1 Executive Summary After months of critical deliberation, the Survey Science Group
More informationTake Your Tax Practice to New Heights
Take Your Tax Practice to New Heights TM AgileTax.com AGILE TAX Agile Tax is the first professional tax software designed exclusively for the cloud. The intuitive, easy-to-use online software takes full
More information8.1 Radio Emission from Solar System objects
8.1 Radio Emission from Solar System objects 8.1.1 Moon and Terrestrial planets At visible wavelengths all the emission seen from these objects is due to light reflected from the sun. However at radio
More informationData Management and Retention for Standards Consortia
Data Management and Retention for Standards Consortia An Overview 15 May, 2006 Prepared by: Jeremy Towsey-French jfrench@kavi.com Kavi Corporation 1 of 6 Copyright 2006 All rights reserved Data Management
More informationRedefining Microsoft SQL Server Data Management. PAS Specification
Redefining Microsoft SQL Server Data Management APRIL Actifio 11, 2013 PAS Specification Table of Contents Introduction.... 3 Background.... 3 Virtualizing Microsoft SQL Server Data Management.... 4 Virtualizing
More informationXenData Product Brief: SX-520 Series Servers for Sony Optical Disc Archives
XenData Product Brief: SX-520 Series Servers for Sony Optical Disc Archives The SX-520 Series of Archive Servers creates highly scalable Optical Disc Digital Video Archives that are optimized for broadcasters,
More informationSTORAGE. 2015 Arka Service s.r.l.
STORAGE STORAGE MEDIA independently from the repository model used, data must be saved on a support (data storage media). Arka Service uses the most common methods used as market standard such as: MAGNETIC
More informationThe Solar Science Data Center and LOFAR
The Solar Science Data Center and LOFAR F. Breitling, G. Mann, C. Vocks 2009-01-27 1. Introduction Solar astronomy is a multi-disciplinary field where knowledge and data of various experiments has to be
More informationAmazon Cloud Storage Options
Amazon Cloud Storage Options Table of Contents 1. Overview of AWS Storage Options 02 2. Why you should use the AWS Storage 02 3. How to get Data into the AWS.03 4. Types of AWS Storage Options.03 5. Object
More informationNAS 254 Cloud Backup. Use Cloud Backup to backup your data to Amazon S3 A S U S T O R C O L L E G E
NAS 254 Cloud Backup Use Cloud Backup to backup your data to Amazon S3 A S U S T O R C O L L E G E COURSE OBJECTIVES Upon completion of this course you should be able to: 1. Have a basic understanding
More informationScientific Computing Meets Big Data Technology: An Astronomy Use Case
Scientific Computing Meets Big Data Technology: An Astronomy Use Case Zhao Zhang AMPLab and BIDS UC Berkeley zhaozhang@cs.berkeley.edu In collaboration with Kyle Barbary, Frank Nothaft, Evan Sparks, Oliver
More informationDiscover Smart Storage Server Solutions
Accelerating Your Success Discover Smart Server Solutions Performance, efficiency and value STORAGE Block and file explained Block explained IP-SAN, also known as Internet Small Computer System Interface
More informationProtecting Information in a Smarter Data Center with the Performance of Flash
89 Fifth Avenue, 7th Floor New York, NY 10003 www.theedison.com 212.367.7400 Protecting Information in a Smarter Data Center with the Performance of Flash IBM FlashSystem and IBM ProtecTIER Printed in
More informationHow To Use The Wynn Odi
WIYN ODI: Observing Process, Data Analysis and Archiving Pierre Martin Yale Survey Workshop, October 2009 ODI: Scientific Challenges ODI is designed to take advantage of the best seeing conditions at WIYN.
More informationBackup and Recovery 1
Backup and Recovery What is a Backup? Backup is an additional copy of data that can be used for restore and recovery purposes. The Backup copy is used when the primary copy is lost or corrupted. This Backup
More informationMANAGING AND MINING THE LSST DATA SETS
MANAGING AND MINING THE LSST DATA SETS Astronomy is undergoing an exciting revolution -- a revolution in the way we probe the universe and the way we answer fundamental questions. New technology enables
More informationThe safer, easier way to help you pass any IT exams. Exam : 000-115. Storage Sales V2. Title : Version : Demo 1 / 5
Exam : 000-115 Title : Storage Sales V2 Version : Demo 1 / 5 1.The IBM TS7680 ProtecTIER Deduplication Gateway for System z solution is designed to provide all of the following EXCEPT: A. ESCON attach
More informationEChO Ground Segment: Overview & Science Operations Assumptions
EChO Ground Segment: Overview & Science Operations Assumptions Matthias Ehle & the Science Ground Segment Working Group EChO Science Operations Study Manager ESA-ESAC, Madrid Science Operations Department/Division
More informationHitachi NAS Platform and Hitachi Content Platform with ESRI Image
W H I T E P A P E R Hitachi NAS Platform and Hitachi Content Platform with ESRI Image Aciduisismodo Extension to ArcGIS Dolore Server Eolore for Dionseq Geographic Uatummy Information Odolorem Systems
More informationCONFIGURATION GUIDELINES: EMC STORAGE FOR PHYSICAL SECURITY
White Paper CONFIGURATION GUIDELINES: EMC STORAGE FOR PHYSICAL SECURITY DVTel Latitude NVMS performance using EMC Isilon storage arrays Correct sizing for storage in a DVTel Latitude physical security
More informationArchiving the insurance data warehouse
REFLECTIONS Archiving the insurance data warehouse Surajit Basu, IBEXI Solutions Surajit Basu is an IT consultant with over 20 years of experience in design, development and implementation of enterprise-wide
More informationMicrosoft SQL Server 2008 R2 Enterprise Edition and Microsoft SharePoint Server 2010
Microsoft SQL Server 2008 R2 Enterprise Edition and Microsoft SharePoint Server 2010 Better Together Writer: Bill Baer, Technical Product Manager, SharePoint Product Group Technical Reviewers: Steve Peschka,
More informationScience and the Taiwan Airborne Telescope
Cosmic Variability Study in Taiwan Wen-Ping Chen Institute of Astronomy National Central University, Taiwan 2010 November 16@Jena/YETI Advantages in Taiwan: - Many high mountains - Western Pacific longitude
More informationData analysis of L2-L3 products
Data analysis of L2-L3 products Emmanuel Gangler UBP Clermont-Ferrand (France) Emmanuel Gangler BIDS 14 1/13 Data management is a pillar of the project : L3 Telescope Caméra Data Management Outreach L1
More informationLOFAR Software Development Roadmap
Netherlands Institute for Radio Astronomy LOFAR Software Development Roadmap LOFAR User Meeting 01/06/2015, Hanno Holties The team: Adriaan Renting, Arno Schoenmakers, Arthur Coolen, Jan David Mol, Jan
More informationHow To Understand And Understand The Science Of Astronomy
Introduction to the VO Christophe.Arviset@esa.int ESAVO ESA/ESAC Madrid, Spain The way Astronomy works Telescopes (ground- and space-based, covering the full electromagnetic spectrum) Observatories Instruments
More informationThe Astronomical Data Warehouse
The Astronomical Data Warehouse Clive G. Page Department of Physics & Astronomy, University of Leicester, Leicester, LE1 7RH, UK Abstract. The idea of the astronomical data warehouse has arisen as the
More informationEinstein Rings: Nature s Gravitational Lenses
National Aeronautics and Space Administration Einstein Rings: Nature s Gravitational Lenses Leonidas Moustakas and Adam Bolton Taken from: Hubble 2006 Science Year in Review The full contents of this book
More informationHow To Build A Supermicro Computer With A 32 Core Power Core (Powerpc) And A 32-Core (Powerpc) (Powerpowerpter) (I386) (Amd) (Microcore) (Supermicro) (
TECHNICAL GUIDELINES FOR APPLICANTS TO PRACE 7 th CALL (Tier-0) Contributing sites and the corresponding computer systems for this call are: GCS@Jülich, Germany IBM Blue Gene/Q GENCI@CEA, France Bull Bullx
More informationEnergy Efficient Storage - Multi- Tier Strategies For Retaining Data
Energy and Space Efficient Storage: Multi-tier Strategies for Protecting and Retaining Data NOTICE This White Paper may contain proprietary information protected by copyright. Information in this White
More informationIntroduction to Optical Archiving Library Solution for Long-term Data Retention
Introduction to Optical Archiving Library Solution for Long-term Data Retention CUC Solutions & Hitachi-LG Data Storage 0 TABLE OF CONTENTS 1. Introduction... 2 1.1 Background and Underlying Requirements
More informationData Data Everywhere, We are now in the Big Data era
Data Data Everywhere, We are now in the Big Data era Mike Liddell 13 th March 2014 UUVs @ OI 2014 Contents Menu Background Big Data What is Generating our Big Data Physical Management of Big Data Optimisation
More informationAugust, 2000 LIGO-G000313-00-D. http://www.ligo.caltech.edu/~smarka/sn/
LIGO-G000313-00-D Proposal to the LSC for Entry of LIGO into the Supernova Nova Early Warning System (SNEWS) and Prototype Development of Real-Time LIGO Supernova Alert 1 Barry Barish, Kenneth Ganezer(CSUDH),
More informationSharePoint & Azure: Digital Asset Management
SharePoint & Azure: Digital Asset Management Project Leadership Microsoft Solutions Provider Proven Results www.attunix.com Introduction Attunix Corporation: A Bellevue, WA based business & technology
More informationData Management Plan Extended Baryon Oscillation Spectroscopic Survey
Data Management Plan Extended Baryon Oscillation Spectroscopic Survey Experiment description: eboss is the cosmological component of the fourth generation of the Sloan Digital Sky Survey (SDSS-IV) located
More informationObjectif. Participant. Prérequis. Pédagogie. Oracle Database 11g - Administration Workshop II - LVC. 5 Jours [35 Heures]
Objectif Back up and recover a database Configure Oracle Database for optimal recovery Administer ASM disk groups Use an RMAN backup to duplicate a database Automating Tasks with the Scheduler Participant
More informationWide-Field Plate Database: Service for Astronomy
Wide-Field Plate Database: Service for Astronomy Milcho K. Tsvetkov To cite this version: Milcho K. Tsvetkov. Wide-Field Plate Database: Service for Astronomy. IMCCE. International Workshop NAROO-GAIA
More informationIT of SPIM Data Storage and Compression. EMBO Course - August 27th! Jeff Oegema, Peter Steinbach, Oscar Gonzalez
IT of SPIM Data Storage and Compression EMBO Course - August 27th Jeff Oegema, Peter Steinbach, Oscar Gonzalez 1 Talk Outline Introduction and the IT Team SPIM Data Flow Capture, Compression, and the Data
More informationOSKAR 2.4.0 Example Revision: 8
OSKAR Example Version history: Revision Date Modification 1 2012-04-24 Creation 2 2012-05-15 Updated figures and text for correct polarisation order. 3 2012-06-13 Updated figures and text to reflect changes
More informationBACKUP SECURITY GUIDELINE
Section: Information Security Revised: December 2004 Guideline: Description: Backup Security Guidelines: are recommended processes, models, or actions to assist with implementing procedures with respect
More informationArchiving Information Storage and Its Advantages
Data Archiving Using Enhanced MAID Aloke Guha CTO, 23nd IEEE13th NASA Mass Storage Conference May 18, 2006 Archive Data Needs Retention and Disposition Performance Corporate governance and regulatory guidelines
More informationConcepts and Architecture of Grid Computing. Advanced Topics Spring 2008 Prof. Robert van Engelen
Concepts and Architecture of Grid Computing Advanced Topics Spring 2008 Prof. Robert van Engelen Overview Grid users: who are they? Concept of the Grid Challenges for the Grid Evolution of Grid systems
More informationFermi LAT and GBM Data Access, File Structure and the Standard Tools. Fermi Solar Analysis Workshop GSFC, August 2012 Elizabeth Ferrara - FSSC 1
Fermi LAT and GBM Data Access, File Structure and the Standard Tools Fermi Solar Analysis Workshop GSFC, August 2012 Elizabeth Ferrara - FSSC 1 Fermi Science Support Center Home Current News, FAQ, Helpdesk
More informationEvolving Risks Of Data Storage. Neville G.H. Green Group Underwriting Manager HSB Engineering Insurance Ltd
Evolving Risks Of Data Storage Neville G.H. Green Group Underwriting Manager HSB Engineering Insurance Ltd Data Loss Data Damage Data Corruption Data Theft Data Compromise The Data Explosion 1996
More informationAn ArrayLibraryforMS SQL Server
An ArrayLibraryforMS SQL Server Scientific requirements and an implementation László Dobos 1,2 --dobos@complex.elte.hu Alex Szalay 2, José Blakeley 3, Tamás Budavári 2, István Csabai 1,2, Dragan Tomic
More informationXenData Archive Series Software Technical Overview
XenData White Paper XenData Archive Series Software Technical Overview Advanced and Video Editions, Version 4.0 December 2006 XenData Archive Series software manages digital assets on data tape and magnetic
More informationThe Expanding Digital Universe: Can we Contain it?
The Expanding Digital Universe: Can we Contain it? Dave Reinsel Vice President, Storage and Semiconductor Research July 16, 2007 Copyright 2007 IDC. Reproduction is forbidden unless authorized. All rights
More informationHow NAS Can Increase Reliability, Uptime & Data Loss Protection: An IT Executive s Story
How NAS Can Increase Reliability, Uptime & Data Loss Protection: An IT Executive s Story How NAS Can Increase Reliability, Uptime & Data Loss Protection: An IT Executive s Story 1 This is Connor, The Small-to-Midsize
More informationMulti-Terabyte Archives for Medical Imaging Applications
Multi-Terabyte Archives for Medical Imaging Applications This paper describes how Windows servers running XenData Archive Series software provide an attractive solution for storing and retrieving multiple
More informationConstructing the Subaru Advanced Data and Analysis Service on VO
Constructing the Subaru Advanced Data and Analysis Service on VO Yuji Shirasaki on behalf of ADC National Astronomical Observatory of Japan Astronomy Data Center Contents Subaru Telescope and Instruments
More informationAGILE Data Center at ASDC: Overview, Catalogs and Science Tools
AGILE Data Center at ASDC: Overview, Catalogs and Science Tools Carlotta Pittori, ASDC on behalf of the AGILE Data Center 12 th AGILE Science Workshop, May 8-9 2014 April 23, 2007: Launch! Equatorial orbit:
More informationIndiana University Science with the WIYN One Degree Imager
Indiana University Science with the WIYN One Degree Imager Katherine Rhode (Indiana University, WIYN SAC member) Indiana University Department of Astronomy Nine faculty members, plus active emeritus faculty
More informationMedia Cloud Building Practicalities
Media Cloud Building Practicalities MEDIA-TECH Hamburg May 2012 Damien Bommart Director - Product Marketing www.fpdigital.com Agenda Front Porch Digital DIVASolutions Family What is a Media Content Storage
More informationHow To Process Data From A Casu.Com Computer System
CASU Processing: Overview and Updates for the VVV Survey Nicholas Walton Eduardo Gonalez-Solares, Simon Hodgkin, Mike Irwin (Institute of Astronomy) Pipeline Processing Summary Data organization (check
More informationA User s Guide to ALMA Scheduling Blocks (Cycle 3)
Doc 3.19, ver. 1 September, 2015 A User s Guide to ALMA Scheduling Blocks (Cycle 3) ALMA, an international astronomy facility, is a partnership of ESO (representing its member states), NSF (USA) and NINS
More informationEMC DATA DOMAIN EXTENDED RETENTION SOFTWARE: MEETING NEEDS FOR LONG-TERM RETENTION OF BACKUP DATA ON EMC DATA DOMAIN SYSTEMS
SOLUTION PROFILE EMC DATA DOMAIN EXTENDED RETENTION SOFTWARE: MEETING NEEDS FOR LONG-TERM RETENTION OF BACKUP DATA ON EMC DATA DOMAIN SYSTEMS MAY 2012 Backups are essential for short-term data recovery
More informationRedefining Microsoft Exchange Data Management
Redefining Microsoft Exchange Data Management FEBBRUARY, 2013 Actifio PAS Specification Table of Contents Introduction.... 3 Background.... 3 Virtualizing Microsoft Exchange Data Management.... 3 Virtualizing
More informationA remote diagnostics system for the MERLIN array. D. Kettle, N.Roddis Jodrell Bank Observatory University of Manchester Macclesfield SK11 9DL
A remote diagnostics system for the MERLIN array D. Kettle, N.Roddis Jodrell Bank Observatory University of Manchester Macclesfield SK11 9DL 1. Introduction Jodrell Bank Observatory operates an array of
More informationSKSPI33 Undertake image asset management
Overview This Standard is about undertaking image asset management in your business or organisation for internal and/or external clients and understanding the resources required and available. The Standard
More informationA View on the Future of Tape
R. Fontana, G. Decad IBM Systems September 10, 2015 A View on the Future of Tape 2015 IBM Corporation 1 A View on the Future of Tape TAPE, HDD, NAND Flash are alive and evolving Post Consumer Nature of
More informationLSST Data Management. Tim Axelrod Project Scientist - LSST Data Management. Thursday, 28 Oct 2010
LSST Data Management Tim Axelrod Project Scientist - LSST Data Management Thursday, 28 Oct 2010 Outline of the Presentation LSST telescope and survey Functions and architecture of the LSST data management
More information