Virtual Observatories A New Era for Astronomy. Reinaldo R. de Carvalho DAS-INPE/MCT 2010

Size: px
Start display at page:

Download "Virtual Observatories A New Era for Astronomy. Reinaldo R. de Carvalho DAS-INPE/MCT 2010"

Transcription

1 Virtual Observatories

2 Virtual Observatories 1%%&'&$#-&6!&9:#,*3),!#,6!6#$C!&,&$D2 *:#%&+-3;& D&);&-$2!!"! "!" &,&$D2 %),-&,-!"#$%&'&#()*! $#%&!(!!! $ '!%&$ $! (% %)'6!6#$C!;#--&$G $! '!!! $#63#-3),G $! %&$!!! *%#'& E#%-)$ "# #!!#$$ % %& &F0#-3), )E!*-#-& +#&,&'-.'/0&#,&!!!!!"#$%&''&!()#$&*+(#,-)*!!.!!/'0*-&$!1,#'2*3*!3,!-4&!(5((!/)#66!!.!!(71/!89:&$3;&,-#'!(&;3,#$!!.!!<&=!>?!@ABA!!.!!@

3 Dark Energy & Cluster Abundance Sensitivity to Dark Energy equation of state Volume element Comoving distance Huterer & Turner 2001

4 C#'#92!%'0*-&$*!#,6!6#$D!&,&$E2!&F0#-3),!)G!*-#-& Virtual Observatories A New *./01)++ Era for Astronomy J3%4,&**+"#**!%#'3=$#-3), P*3,E!Q)4,*-),!&-!#'M!R@AAKS!!"# %&# $!""' ()* +, -)#!"# $!""!"!# $ " /'0*-&$!#=0,6#,%&!3*!H&$2! *&,*3-3H&!-)!%)*;)')E2 (&,*3-3H3-2!3,%$&#*&*!I3-4! $&6*43G-!#,6!;#**! -4$&*4)'6 5&&:!6#-#!I3-4!'#$E&!#$&#! %)H&$#E&!3*!$&F03$&6!"#$%& ''(#%(')!*'+%,- ''.!"#$%& ''(#%(')!*'+%,- ''. (5((!5JK LAAA AM> (5((!/)#66 (5((!5JK NAA LAAA AML AM> 58(!O2$ (5((!/)#66 OAAA NAA BM> AML 58(!O2$ OAAA BM> /(0('!%0!!!!!!"#$%&''&!()#$&*+(#,-)*!!.!!/'0*-&$!1,#'2*3*!3,!-4&!(5((!/)#66!!.!!(71/!89:&$3;&,-#'!(&;3,#$!!.!!<&=!>?!@ABA!!.!!>

5 Virtual Observatories

6 A roadmap for finding clusters in the Universe Virtual Observatories de Carvalho et al., JCIS Mass Function Star/Galaxy Separation Comparison to Mock Catalogs DARK ENERGY Translate an image into objects 2DPHOT What is a cluster? Photometric Redshift From Overdensity to Mass Measuring Completeness & Purity VO?

7 Virtual Observatories

8 Virtual Observatories 1 - The Virtual Observatory (VO) concept is the astronomical community s response to the scientific and technological challenges posed by massive and complex data sets. 2 - An underlying concept of the VO is that by providing improved and homogenized data access combined with the tools to manipulate and explore the data, the need for new observations will be reduced even as the scientific output is increased. 3 - VO is not an enterprise driven by a single institute or even one country. It is rather a community endeavor aimed at the democratization of information.

9 Virtual Observatories

10 Significant Increase of Data Virtual Observatories in Astronomy

11 Significant Increase of Data Virtual Observatories in Astronomy What are the Implications? 1 - Investment in Data Grid Network Infrastructure 2 - Development of Fast Data Processing 3 - Synergy with other areas for using specific Data Mining methods VO

12 Virtual Observatories

13 Virtual Observatories

14 Virtual Observatories

15 Star/Galaxy Separation Virtual Observatories Parametric Methods

16 Star/Galaxy Separation Virtual Observatories Machine Learning Methods Parametric Methods Training Set = Attributes + True Classification

17 Star/Galaxy Separation Virtual Observatories Machine Learning Methods Parametric Methods Training Set = Attributes + True Classification Observed Set = Only Attributes Classification based on the rules

18 Star/Galaxy Separation E.C. Vasconcelloset al., PASP Virtual Observatories Machine Learning Methods Parametric Methods Training Set = Attributes + True Classification Observed Set = Only Attributes Classification based on the rules

19 Translate an image into objects 2DPHOT BRAVO Processing of the entire SDSS-DR6 in four bands

20 Virtual Observatories

21 Photometric Redshift Virtual Observatories

22 Virtual Observatories

23 Photometric Redshift Virtual Observatories Coadd galaxy catalog v09.05 and v09.08 High redshift training set: DEEP2

24 Photometric Redshift Virtual Observatories Coadd galaxy catalog v09.05 and v09.08 High redshift training set: DEEP2 What is the context in which these tools are needed? 1 - Access to different types of data Interoperability 2 - Fast Data Processing VO

25 Virtual Observatories

26 What is a Cluster? Virtual Observatories How to Identify a Cluster? 1 - Clusters are not isolated structures time 2 - They evolve dynamically and content 3 - Cluster Galaxies evolve and this might depend on their mass (which mass?) 4 - Error budget when we go from ideal Reinaldo R. de to Carvalho real 5 - How do we handle galaxy evolution?

27 Virtual Observatories

28 How do we Find Clusters? Virtual Observatories

29 How do we Find Clusters? Virtual Observatories SPIDER Spheroidʼs Panchromatic Investigation in Different Environmental Regions

30 Virtual Observatories

31 From Overdensity to Mass Virtual Observatories Lopes et al 2009 Mass = f (Ngal, Lopt, Lx) Relations may change with z

32 Virtual Observatories

33 Comparison to Mock Catalogs Virtual Observatories Selection function Halo Mock true dark matter object with galaxies bounded to its potential well. Cluster (RA, DEC, z) coordinates + list of member galaxies matching Membership? Mtrue-Mobs scatter Raw Completeness Fraction of halos with a cluster counterpart Raw Purity Fraction of clusters corresponding to real halos Completeness C(M,z) Purity P(M,z)

34 Comparison to Mock Catalogs Virtual Observatories Selection function Halo Mock true dark matter object with galaxies bounded to its potential well. Cluster (RA, DEC, z) coordinates + list of member galaxies matching Membership? Mtrue-Mobs scatter Raw Completeness Fraction of halos with a cluster counterpart Raw Purity Fraction of clusters corresponding to real halos Completeness C(M,z) Purity P(M,z) VO needs to bring simulations to the same context as data

35 Virtual Observatories

36 Mass Function and Dark Energy Virtual Observatories

37 Mass Function and Dark Energy Constraints on w-ω m space: Virtual Observatories forecasts Optimistic forecasts for fiducial model: ΛCDM Fixed parameters: σ 8 = 0.8 h = 0.72 Ω x = 1 - Ω m Redshift range: z 1.2 Area: 1000 square degrees Mass threshold: solar masses Contours: Lima&Hu (2004,2005) Fisher Matrix Gray regions: MCMC

38 Virtual Observatories

39 A little Help From VO Virtual Observatories

40 A little Help From VO Virtual Observatories What do we need here? 1 - Tools to interconnect images and catalogs 2 - Development of Fast Data Processing VO

41 A little Help From VO Virtual Observatories Translate an image into objects 2DPHOT What do we need here? Star/Galaxy Separation Photometric Redshift 1 - Tools to interconnect images and catalogs 2 - Development of Fast Data Processing VO What is a cluster? From Overdensity to Mass

42

43 Astronomical Strategies

44 Astronomical Strategies PROBLEM SOLUTION

45 Astronomical Strategies PROBLEM Slow CPU growth SOLUTION Distributed Computing

46 Astronomical Strategies PROBLEM Slow CPU growth Limited storage SOLUTION Distributed Computing Distributed Data

47 Astronomical Strategies PROBLEM Slow CPU growth Limited storage Limited bandwidth SOLUTION Distributed Computing Distributed Data Information Hierarchies

48 Astronomical Strategies PROBLEM Slow CPU growth Limited storage Limited bandwidth SOLUTION Distributed Computing Distributed Data Information Hierarchies - Move only what you need

49 Astronomical Strategies PROBLEM Slow CPU growth Limited storage Limited bandwidth SOLUTION Distributed Computing Distributed Data Information Hierarchies - Move only what you need Data diversity Interoperability

50 Astronomical Strategies PROBLEM Slow CPU growth Limited storage Limited bandwidth SOLUTION Distributed Computing Distributed Data Information Hierarchies - Move only what you need Data diversity Interoperability

51 Astronomical Strategies PROBLEM Slow CPU growth Limited storage Limited bandwidth SOLUTION Distributed Computing Distributed Data Information Hierarchies - Move only what you need Data diversity VO Interoperability

52 Virtual Observatories

53 Strategy for the Future Virtual Observatories Concluding Remarks

54 Strategy for the Future Virtual Observatories Concluding Remarks Translate an image into objects 2DPHOT Star/Galaxy Separation Infrastructure + Data Mining Photometric Redshift From Overdensity to Mass Measuring Completeness & Purity What is a cluster? Comparison to Mock Catalogs IVOA (

Galaxy Survey data analysis using SDSS-III as an example

Galaxy Survey data analysis using SDSS-III as an example Galaxy Survey data analysis using SDSS-III as an example Will Percival (University of Portsmouth) showing work by the BOSS galaxy clustering working group" Cosmology from Spectroscopic Galaxy Surveys"

More information

Software challenges in the implementation of large surveys: the case of J-PAS

Software challenges in the implementation of large surveys: the case of J-PAS Software challenges in the implementation of large surveys: the case of J-PAS 1/21 Paulo Penteado - IAG/USP pp.penteado@gmail.com http://www.ppenteado.net/ast/pp_lsst_201204.pdf (K. Taylor) (A. Fernández-Soto)

More information

Australian Virtual Observatory

Australian Virtual Observatory Australian Virtual Observatory International Astronomical Union GA 2003 Joint Discussion 08 17th-18th July 2003 Sydney David Barnes The University of Melbourne Our take on virtual observatories bring legacy

More information

Commentary on Techniques for Massive- Data Machine Learning in Astronomy

Commentary on Techniques for Massive- Data Machine Learning in Astronomy 1 of 24 Commentary on Techniques for Massive- Data Machine Learning in Astronomy Nick Ball Herzberg Institute of Astrophysics Victoria, Canada The Problem 2 of 24 Astronomy faces enormous datasets Their

More information

How To Understand And Understand The Science Of Astronomy

How To Understand And Understand The Science Of Astronomy Introduction to the VO Christophe.Arviset@esa.int ESAVO ESA/ESAC Madrid, Spain The way Astronomy works Telescopes (ground- and space-based, covering the full electromagnetic spectrum) Observatories Instruments

More information

Physical Self-Calibration of X-ray and SZ Surveys

Physical Self-Calibration of X-ray and SZ Surveys Physical Self-Calibration of X-ray and SZ Surveys Greg L. Bryan, Zoltan Haiman (Columbia University) and Joshua D. Younger (CfA) 1. Cluster Surveys and Self-Calibration Clusters of galaxies form at the

More information

Visualization of Large Multi-Dimensional Datasets

Visualization of Large Multi-Dimensional Datasets ***TITLE*** ASP Conference Series, Vol. ***VOLUME***, ***PUBLICATION YEAR*** ***EDITORS*** Visualization of Large Multi-Dimensional Datasets Joel Welling Department of Statistics, Carnegie Mellon University,

More information

Astrophysics with Terabyte Datasets. Alex Szalay, JHU and Jim Gray, Microsoft Research

Astrophysics with Terabyte Datasets. Alex Szalay, JHU and Jim Gray, Microsoft Research Astrophysics with Terabyte Datasets Alex Szalay, JHU and Jim Gray, Microsoft Research Living in an Exponential World Astronomers have a few hundred TB now 1 pixel (byte) / sq arc second ~ 4TB Multi-spectral,

More information

Data centres in the. Virtual Observatory. F. Genova, IVOA Small Project meeting, September 2006 1

Data centres in the. Virtual Observatory. F. Genova, IVOA Small Project meeting, September 2006 1 Data centres in the Virtual Observatory F. Genova, IVOA Small Project meeting, September 2006 1 VO status (1) Many national projects Very different contexts/financing agencies A really world-wide, global

More information

Data analysis of L2-L3 products

Data analysis of L2-L3 products Data analysis of L2-L3 products Emmanuel Gangler UBP Clermont-Ferrand (France) Emmanuel Gangler BIDS 14 1/13 Data management is a pillar of the project : L3 Telescope Caméra Data Management Outreach L1

More information

DS6 Phase 4 Napoli group Astroneural 1,0 is available and includes tools for supervised and unsupervised data mining:

DS6 Phase 4 Napoli group Astroneural 1,0 is available and includes tools for supervised and unsupervised data mining: DS6 Phase 4 Napoli group Astroneural 1,0 is available and includes tools for supervised and unsupervised data mining: Preprocessing & visualization Supervised (MLP, RBF) Unsupervised (PPS, NEC+dendrogram,

More information

The Virtual Observatory: What is it and how can it help me? Enrique Solano LAEFF / INTA Spanish Virtual Observatory

The Virtual Observatory: What is it and how can it help me? Enrique Solano LAEFF / INTA Spanish Virtual Observatory The Virtual Observatory: What is it and how can it help me? Enrique Solano LAEFF / INTA Spanish Virtual Observatory Astronomy in the XXI century The Internet revolution (the dot com boom ) has transformed

More information

Astro 102 Test 5 Review Spring 2016. See Old Test 4 #16-23, Test 5 #1-3, Old Final #1-14

Astro 102 Test 5 Review Spring 2016. See Old Test 4 #16-23, Test 5 #1-3, Old Final #1-14 Astro 102 Test 5 Review Spring 2016 See Old Test 4 #16-23, Test 5 #1-3, Old Final #1-14 Sec 14.5 Expanding Universe Know: Doppler shift, redshift, Hubble s Law, cosmic distance ladder, standard candles,

More information

arxiv:1207.3646v1 [cs.ce] 16 Jul 2012

arxiv:1207.3646v1 [cs.ce] 16 Jul 2012 jcis@epacis.org OGCOSMO: An auxiliary tool for the study of the Universe within hierarchical scenario of structure formation arxiv:1207.3646v1 [cs.ce] 16 Jul 2012 Eduardo S. Pereira 1, Oswaldo D. Miranda

More information

Conquering the Astronomical Data Flood through Machine

Conquering the Astronomical Data Flood through Machine Conquering the Astronomical Data Flood through Machine Learning and Citizen Science Kirk Borne George Mason University School of Physics, Astronomy, & Computational Sciences http://spacs.gmu.edu/ The Problem:

More information

Self Calibration of Cluster Counts: Observable-Mass Distribution. Wayne Hu Kona, March 2005

Self Calibration of Cluster Counts: Observable-Mass Distribution. Wayne Hu Kona, March 2005 Self Calibration of Cluster Counts: Observable-Mass Distribution Wayne Hu Kona, March 2005 Self Calibration of Cluster Counts: Observable-Mass Distribution Wayne Hu Kona, August 2004 Scattered Forecasts

More information

Making the Most of Missing Values: Object Clustering with Partial Data in Astronomy

Making the Most of Missing Values: Object Clustering with Partial Data in Astronomy Astronomical Data Analysis Software and Systems XIV ASP Conference Series, Vol. XXX, 2005 P. L. Shopbell, M. C. Britton, and R. Ebert, eds. P2.1.25 Making the Most of Missing Values: Object Clustering

More information

Data Mining Challenges and Opportunities in Astronomy

Data Mining Challenges and Opportunities in Astronomy Data Mining Challenges and Opportunities in Astronomy S. G. Djorgovski (Caltech) With special thanks to R. Brunner, A. Szalay, A. Mahabal, et al. The Punchline: Astronomy has become an immensely datarich

More information

Cosmological Analysis of South Pole Telescope-detected Galaxy Clusters

Cosmological Analysis of South Pole Telescope-detected Galaxy Clusters Cosmological Analysis of South Pole Telescope-detected Galaxy Clusters March 24th Tijmen de Haan (McGill) - Moriond 2014 Photo credit: Keith Vanderlinde Outline The galaxy cluster sample Calibrating the

More information

How To Use Game To Learn From Data

How To Use Game To Learn From Data Astronomical data Mining DAMEWARE and beyond Giuseppe Longo Università Federico II Napoli (Italy) M. Brescia INAF OAC G.S. Djorgovski Caltech S. Cavuoti INAF UFII & the DAMEWARE people Astroinformatics

More information

Chapter 15.3 Galaxy Evolution

Chapter 15.3 Galaxy Evolution Chapter 15.3 Galaxy Evolution Elliptical Galaxies Spiral Galaxies Irregular Galaxies Are there any connections between the three types of galaxies? How do galaxies form? How do galaxies evolve? P.S. You

More information

Learning from Big Data in

Learning from Big Data in Learning from Big Data in Astronomy an overview Kirk Borne George Mason University School of Physics, Astronomy, & Computational Sciences http://spacs.gmu.edu/ From traditional astronomy 2 to Big Data

More information

VisIVO, an open source, interoperable visualization tool for the Virtual Observatory

VisIVO, an open source, interoperable visualization tool for the Virtual Observatory Claudio Gheller (CINECA) 1, Ugo Becciani (OACt) 2, Marco Comparato (OACt) 3 Alessandro Costa (OACt) 4 VisIVO, an open source, interoperable visualization tool for the Virtual Observatory 1: c.gheller@cineca.it

More information

Virtual Observatory tools for the detection of T dwarfs. Enrique Solano, LAEFF / SVO Eduardo Martín, J.A. Caballero, IAC

Virtual Observatory tools for the detection of T dwarfs. Enrique Solano, LAEFF / SVO Eduardo Martín, J.A. Caballero, IAC Virtual Observatory tools for the detection of T dwarfs Enrique Solano, LAEFF / SVO Eduardo Martín, J.A. Caballero, IAC T dwarfs Low-mass (60-13 MJup), low-temperature (< 1300-1500 K), low-luminosity brown

More information

Bringing Big Data to the Solar System. Paulo Penteado Northern Arizona University, Flagstaff (visiting David Trilling)

Bringing Big Data to the Solar System. Paulo Penteado Northern Arizona University, Flagstaff (visiting David Trilling) Bringing Big Data to the Solar System Paulo Penteado Northern Arizona University, Flagstaff (visiting David Trilling) pp.penteado@gmail.com http://www.ppenteado.net What is Big Data and why do we care?

More information

MANAGING AND MINING THE LSST DATA SETS

MANAGING AND MINING THE LSST DATA SETS MANAGING AND MINING THE LSST DATA SETS Astronomy is undergoing an exciting revolution -- a revolution in the way we probe the universe and the way we answer fundamental questions. New technology enables

More information

and the VO-Science Francisco Jiménez Esteban Suffolk University

and the VO-Science Francisco Jiménez Esteban Suffolk University The Spanish-VO and the VO-Science Francisco Jiménez Esteban CAB / SVO (INTA-CSIC) Suffolk University The Spanish-VO (SVO) IVOA was created in June 2002 with the mission to facilitate the international

More information

Cross-Matching Very Large Datasets

Cross-Matching Very Large Datasets 1 Cross-Matching Very Large Datasets María A. Nieto-Santisteban, Aniruddha R. Thakar, and Alexander S. Szalay Johns Hopkins University Abstract The primary mission of the National Virtual Observatory (NVO)

More information

Probing Dark Energy with Baryon Acoustic Oscillations from Future Large Galaxy Redshift Surveys

Probing Dark Energy with Baryon Acoustic Oscillations from Future Large Galaxy Redshift Surveys Probing Dark Energy with Baryon Acoustic Oscillations from Future Large Galaxy Redshift Surveys Hee-Jong Seo (Steward Observatory) Daniel J. Eisenstein (Steward Observatory) Martin White, Edwin Sirko,

More information

Description of the Dark Energy Survey for Astronomers

Description of the Dark Energy Survey for Astronomers Description of the Dark Energy Survey for Astronomers May 1, 2012 Abstract The Dark Energy Survey (DES) will use 525 nights on the CTIO Blanco 4-meter telescope with the new Dark Energy Camera built by

More information

Software Development for Virtual Observatories

Software Development for Virtual Observatories Software Development for Virtual Observatories BRAVO Workshop February 2007 Rafael Santos 1 Warning! This presentation is biased. I'll talk about VO software development, including some under the hood

More information

The astronomical Virtual Observatory : lessons learnt, looking forward. Françoise Genova - Forum VO-PDC d après ADASS XXI, Paris, nov.

The astronomical Virtual Observatory : lessons learnt, looking forward. Françoise Genova - Forum VO-PDC d après ADASS XXI, Paris, nov. The astronomical Virtual Observatory : lessons learnt, looking forward Examples taken from the European view, but other projects have followed similar paths The VO aim Enable seamless access to the wealth

More information

Lecture 5b: Data Mining. Peter Wheatley

Lecture 5b: Data Mining. Peter Wheatley Lecture 5b: Data Mining Peter Wheatley Data archives Most astronomical data now available via archives Raw data and high-level products usually available Data reduction software often specific to individual

More information

Silviu Panica, Marian Neagul, Daniela Zaharie and Dana Petcu (Romania)

Silviu Panica, Marian Neagul, Daniela Zaharie and Dana Petcu (Romania) Silviu Panica, Marian Neagul, Daniela Zaharie and Dana Petcu (Romania) Outline Introduction EO challenges; EO and classical/cloud computing; EO Services The computing platform Cluster -> Grid -> Cloud

More information

A Critical Review of Scientific Visualization in Astronomy

A Critical Review of Scientific Visualization in Astronomy A Critical Review of Scientific Visualization in Astronomy Christopher Fluke & Amr Hassan Those not properly initiated into the mysteries of visualization research often seek to understand the images rather

More information

Scientific Computing Meets Big Data Technology: An Astronomy Use Case

Scientific Computing Meets Big Data Technology: An Astronomy Use Case Scientific Computing Meets Big Data Technology: An Astronomy Use Case Zhao Zhang AMPLab and BIDS UC Berkeley zhaozhang@cs.berkeley.edu In collaboration with Kyle Barbary, Frank Nothaft, Evan Sparks, Oliver

More information

The Solar Journey: Modeling Features of the Local Bubble and Galactic Environment of the Sun

The Solar Journey: Modeling Features of the Local Bubble and Galactic Environment of the Sun The Solar Journey: Modeling Features of the Local Bubble and Galactic Environment of the Sun P.C. Frisch and A.J. Hanson Department of Astronomy and Astrophysics University of Chicago and Computer Science

More information

Research IT Plan. UCD IT Services. Seirbhísí TF UCD

Research IT Plan. UCD IT Services. Seirbhísí TF UCD Research IT Plan UCD IT Services Research IT Plan The main goal of this plan is to provide a sustainable and evolving campus Cyberinfrastructure for the UCD research community. We will continue the development

More information

Einstein Rings: Nature s Gravitational Lenses

Einstein Rings: Nature s Gravitational Lenses National Aeronautics and Space Administration Einstein Rings: Nature s Gravitational Lenses Leonidas Moustakas and Adam Bolton Taken from: Hubble 2006 Science Year in Review The full contents of this book

More information

LSST and the Cloud: Astro Collaboration in 2016 Tim Axelrod LSST Data Management Scientist

LSST and the Cloud: Astro Collaboration in 2016 Tim Axelrod LSST Data Management Scientist LSST and the Cloud: Astro Collaboration in 2016 Tim Axelrod LSST Data Management Scientist DERCAP Sydney, Australia, 2009 Overview of Presentation LSST - a large-scale Southern hemisphere optical survey

More information

Scientific papers by György Paál (1934 1992)

Scientific papers by György Paál (1934 1992) Scientific papers by György Paál (1934 1992) Some guidance to the list of papers (compiled by G. Paál) A. Earliest arguments (from years 1964 1971) for large scale inhomogeneities reaching Abell's distance

More information

Photo-z Requirements for Self-Calibration of Cluster Dark Energy Studies

Photo-z Requirements for Self-Calibration of Cluster Dark Energy Studies Photo-z Requirements for Self-Calibration of Cluster Dark Energy Studies Marcos Lima Department of Physics Kavli Institute for Cosmological Physics University of Chicago DES Collaboration Meeting Barcelona

More information

What is the Sloan Digital Sky Survey?

What is the Sloan Digital Sky Survey? What is the Sloan Digital Sky Survey? Simply put, the Sloan Digital Sky Survey is the most ambitious astronomical survey ever undertaken. The survey will map one-quarter of the entire sky in detail, determining

More information

Intro to Sessions 3 & 4: Data Management & Data Analysis. Bob Mann Wide-Field Astronomy Unit University of Edinburgh

Intro to Sessions 3 & 4: Data Management & Data Analysis. Bob Mann Wide-Field Astronomy Unit University of Edinburgh Intro to Sessions 3 & 4: Data Management & Data Analysis Bob Mann Wide-Field Astronomy Unit University of Edinburgh 1 Outline Data Management Issues Alternatives to monolithic RDBMS model Intercontinental

More information

Transcript 22 - Universe

Transcript 22 - Universe Transcript 22 - Universe A few introductory words of explanation about this transcript: This transcript includes the words sent to the narrator for inclusion in the latest version of the associated video.

More information

The Scientific Data Mining Process

The Scientific Data Mining Process Chapter 4 The Scientific Data Mining Process When I use a word, Humpty Dumpty said, in rather a scornful tone, it means just what I choose it to mean neither more nor less. Lewis Carroll [87, p. 214] In

More information

Big Data Hope or Hype?

Big Data Hope or Hype? Big Data Hope or Hype? David J. Hand Imperial College, London and Winton Capital Management Big data science, September 2013 1 Google trends on big data Google search 1 Sept 2013: 1.6 billion hits on big

More information

Data Models For Interoperability. Rob Atkinson

Data Models For Interoperability. Rob Atkinson Data Models For Interoperability Rob Atkinson Contents Problem Statement Conceptual Architecture Role of Data Models Role of Registries Integration of GRID and SDI Problem Statement How do we derive useful

More information

Data Centre Alliance - Science

Data Centre Alliance - Science Data Centre Alliance - Science Mark ALLEN, Jonathan TEDDS & DCA IST Nov 2008 IST Internal Science Team Activities of the IST are being rounded off for end of project Coordinated by IST telecon IST: T2-DCA-IST

More information

Dealing with large datasets

Dealing with large datasets Dealing with large datasets (by throwing away most of the data) Alan Heavens Institute for Astronomy, University of Edinburgh with Ben Panter, Rob Tweedie, Mark Bastin, Will Hossack, Keith McKellar, Trevor

More information

TELESCOPE AS TIME MACHINE

TELESCOPE AS TIME MACHINE TELESCOPE AS TIME MACHINE Read this article about NASA s latest high-tech space telescope. Then, have fun doing one or both of the word puzzles that use the important words in the article. A TELESCOPE

More information

FIRST LIGHT IN THE UNIVERSE

FIRST LIGHT IN THE UNIVERSE FIRST LIGHT IN THE UNIVERSE Richard Ellis, Caltech 1. Role of Observations in Cosmology & Galaxy Formation 2. Galaxies & the Hubble Sequence 3. Cosmic Star Formation Histories 4. Stellar Mass Assembly

More information

Astronomy & Physics Resources for Middle & High School Teachers

Astronomy & Physics Resources for Middle & High School Teachers Astronomy & Physics Resources for Middle & High School Teachers Gillian Wilson http://www.faculty.ucr.edu/~gillianw/k12 A cosmologist is.... an astronomer who studies the formation and evolution of the

More information

BIG DATA IN THE CLOUD : CHALLENGES AND OPPORTUNITIES MARY- JANE SULE & PROF. MAOZHEN LI BRUNEL UNIVERSITY, LONDON

BIG DATA IN THE CLOUD : CHALLENGES AND OPPORTUNITIES MARY- JANE SULE & PROF. MAOZHEN LI BRUNEL UNIVERSITY, LONDON BIG DATA IN THE CLOUD : CHALLENGES AND OPPORTUNITIES MARY- JANE SULE & PROF. MAOZHEN LI BRUNEL UNIVERSITY, LONDON Overview * Introduction * Multiple faces of Big Data * Challenges of Big Data * Cloud Computing

More information

The Big Picture: Information 01100 Technology Revolution, and 1010011 Science in the 21st Century 00101000

The Big Picture: Information 01100 Technology Revolution, and 1010011 Science in the 21st Century 00101000 011 The Big Picture: Information 01100 Technology Revolution, and 1010011 Science in the 21st Century 00101000 Roy & George s Excellent Adventure 1110100011 001001110110110 100101010001011101 Lecture 4

More information

A Preliminary Summary of The VLA Sky Survey

A Preliminary Summary of The VLA Sky Survey A Preliminary Summary of The VLA Sky Survey Eric J. Murphy and Stefi Baum (On behalf of the entire Science Survey Group) 1 Executive Summary After months of critical deliberation, the Survey Science Group

More information

Analytics-as-a-Service: From Science to Marketing

Analytics-as-a-Service: From Science to Marketing Analytics-as-a-Service: From Science to Marketing Data Information Knowledge Insights (Discovery & Decisions) Kirk Borne George Mason University, Fairfax, VA www.kirkborne.net @KirkDBorne Big Data: What

More information

How To Use The Wynn Odi

How To Use The Wynn Odi WIYN ODI: Observing Process, Data Analysis and Archiving Pierre Martin Yale Survey Workshop, October 2009 ODI: Scientific Challenges ODI is designed to take advantage of the best seeing conditions at WIYN.

More information

165 points. Name Date Period. Column B a. Cepheid variables b. luminosity c. RR Lyrae variables d. Sagittarius e. variable stars

165 points. Name Date Period. Column B a. Cepheid variables b. luminosity c. RR Lyrae variables d. Sagittarius e. variable stars Name Date Period 30 GALAXIES AND THE UNIVERSE SECTION 30.1 The Milky Way Galaxy In your textbook, read about discovering the Milky Way. (20 points) For each item in Column A, write the letter of the matching

More information

Visualizing and Analyzing Massive Astronomical Datasets with Partiview

Visualizing and Analyzing Massive Astronomical Datasets with Partiview Visualizing and Analyzing Massive Astronomical Datasets with Partiview Brian P. Abbott 1, Carter B. Emmart 1, Stuart Levy 2, and Charles T. Liu 1 1 American Museum of Natural History & Hayden Planetarium,

More information

Data Management Plan Extended Baryon Oscillation Spectroscopic Survey

Data Management Plan Extended Baryon Oscillation Spectroscopic Survey Data Management Plan Extended Baryon Oscillation Spectroscopic Survey Experiment description: eboss is the cosmological component of the fourth generation of the Sloan Digital Sky Survey (SDSS-IV) located

More information

Feedback Autonomic Provisioning for guaranteeing performance (and reliability. - application to Big Data Systems

Feedback Autonomic Provisioning for guaranteeing performance (and reliability. - application to Big Data Systems Feedback Autonomic Provisioning for guaranteeing performance (and reliability) - application to Big Data Systems Bogdan Robu bogdan.robu@gipsa-lab.fr HIPEAC - HPES Workshop Amsterdam 19-21.01.2015 Context

More information

A GENERAL TAXONOMY FOR VISUALIZATION OF PREDICTIVE SOCIAL MEDIA ANALYTICS

A GENERAL TAXONOMY FOR VISUALIZATION OF PREDICTIVE SOCIAL MEDIA ANALYTICS A GENERAL TAXONOMY FOR VISUALIZATION OF PREDICTIVE SOCIAL MEDIA ANALYTICS Stacey Franklin Jones, D.Sc. ProTech Global Solutions Annapolis, MD Abstract The use of Social Media as a resource to characterize

More information

The Milky Way Galaxy is Heading for a Major Cosmic Collision

The Milky Way Galaxy is Heading for a Major Cosmic Collision The Milky Way Galaxy is Heading for a Major Cosmic Collision Roeland van der Marel (STScI) [based on work with a team of collaborators reported in the Astrophysical Journal July 2012] Hubble Science Briefing

More information

thermal history of the universe and big bang nucleosynthesis

thermal history of the universe and big bang nucleosynthesis thermal history of the universe and big bang nucleosynthesis Kosmologie für Nichtphysiker Markus Pössel (vertreten durch Björn Malte Schäfer) Fakultät für Physik und Astronomie, Universität Heidelberg

More information

Science and the Taiwan Airborne Telescope

Science and the Taiwan Airborne Telescope Cosmic Variability Study in Taiwan Wen-Ping Chen Institute of Astronomy National Central University, Taiwan 2010 November 16@Jena/YETI Advantages in Taiwan: - Many high mountains - Western Pacific longitude

More information

Dominique Fouchez. 12 Fevrier 2011

Dominique Fouchez. 12 Fevrier 2011 données données CPPM 12 Fevrier 2011 The Data données one 6.4-gigabyte image every 17 seconds 15 terabytes of raw scientific image data / night 60-petabyte final image data archive 20-petabyte final database

More information

Using Photometric Data to Derive an HR Diagram for a Star Cluster

Using Photometric Data to Derive an HR Diagram for a Star Cluster Using Photometric Data to Derive an HR Diagram for a Star Cluster In In this Activity, we will investigate: 1. How to use photometric data for an open cluster to derive an H-R Diagram for the stars and

More information

Understanding the Accelerating Universe using the MSE. Gong-Bo Zhao NAOC

Understanding the Accelerating Universe using the MSE. Gong-Bo Zhao NAOC Understanding the Accelerating Universe using the MSE Gong-Bo Zhao NAOC Nobel Prize 2011 a > 0 The expansion of the Universe can accelerate if In GR, to add new repulsive matter, which contributes 70%

More information

Summary of Data Management Principles Dark Energy Survey V2.1, 7/16/15

Summary of Data Management Principles Dark Energy Survey V2.1, 7/16/15 Summary of Data Management Principles Dark Energy Survey V2.1, 7/16/15 This Summary of Data Management Principles (DMP) has been prepared at the request of the DOE Office of High Energy Physics, in support

More information

Multidimensional Data in the Virtual Observatory

Multidimensional Data in the Virtual Observatory IX Reunión Científica de la SEA Madrid- 15/09/2010 Red Temática SVO Multidimensional Data in the Virtual Observatory José Enrique Ruiz Grupo AMIGA Instituto de Astrofísica de Andalucía CSIC Contextual

More information

The LSST Data management and French computing activities. Dominique Fouchez on behalf of the IN2P3 Computing Team. LSST France April 8th,2015

The LSST Data management and French computing activities. Dominique Fouchez on behalf of the IN2P3 Computing Team. LSST France April 8th,2015 The LSST Data management and French computing activities Dominique Fouchez on behalf of the IN2P3 Computing Team LSST France April 8th,2015 OSG All Hands SLAC April 7-9, 2014 1 The LSST Data management

More information

HPC Deployment of OpenFOAM in an Industrial Setting

HPC Deployment of OpenFOAM in an Industrial Setting HPC Deployment of OpenFOAM in an Industrial Setting Hrvoje Jasak h.jasak@wikki.co.uk Wikki Ltd, United Kingdom PRACE Seminar: Industrial Usage of HPC Stockholm, Sweden, 28-29 March 2011 HPC Deployment

More information

Next Generation Operating Systems

Next Generation Operating Systems Next Generation Operating Systems Zeljko Susnjar, Cisco CTG June 2015 The end of CPU scaling Future computing challenges Power efficiency Performance == parallelism Cisco Confidential 2 Paradox of the

More information

14.10.2014. Overview. Swarms in nature. Fish, birds, ants, termites, Introduction to swarm intelligence principles Particle Swarm Optimization (PSO)

14.10.2014. Overview. Swarms in nature. Fish, birds, ants, termites, Introduction to swarm intelligence principles Particle Swarm Optimization (PSO) Overview Kyrre Glette kyrrehg@ifi INF3490 Swarm Intelligence Particle Swarm Optimization Introduction to swarm intelligence principles Particle Swarm Optimization (PSO) 3 Swarms in nature Fish, birds,

More information

STFC s Science and Technology Strategy

STFC s Science and Technology Strategy STFC s Science and Technology Strategy Director, Science Programmes Nuclear Physics and Particle Physics Town Meetings April 2008 Addressing the challenges of the twenty-first century The twenty-first

More information

ASKAP Science Data Archive: Users and Requirements CSIRO ASTRONOMY AND SPACE SCIENCE (CASS)

ASKAP Science Data Archive: Users and Requirements CSIRO ASTRONOMY AND SPACE SCIENCE (CASS) ASKAP Science Data Archive: Users and Requirements CSIRO ASTRONOMY AND SPACE SCIENCE (CASS) Jessica Chapman, Data Workshop March 2013 ASKAP Science Data Archive Talk outline Data flow in brief Some radio

More information

Cosmic Structure Formation and Dynamics: Cosmological N-body Simulations of Galaxy Formation and Magnetohydrodynamic Simulations of Solar Atmosphere

Cosmic Structure Formation and Dynamics: Cosmological N-body Simulations of Galaxy Formation and Magnetohydrodynamic Simulations of Solar Atmosphere Chapter 3 Epoch Making Simulation Cosmic Structure Formation and Dynamics: Cosmological N-body Simulations of Galaxy Formation and Magnetohydrodynamic Simulations of Solar Atmosphere Project Representative

More information

CFHT STRIPE82 SURVEY (CS82)

CFHT STRIPE82 SURVEY (CS82) CFHT STRIPE82 SURVEY (CS82) 160 degrees (130 effective) i=24.1, AB mag, 5σ, 2 aperture Seeing ~ 0.6 over full survey Pixel size : 0.187 Alexie Leauthaud. Jean-Paul Kneib. Martin Makler. Ludovic van Waerbeke.

More information

Cluster, Grid, Cloud Concepts

Cluster, Grid, Cloud Concepts Cluster, Grid, Cloud Concepts Kalaiselvan.K Contents Section 1: Cluster Section 2: Grid Section 3: Cloud Cluster An Overview Need for a Cluster Cluster categorizations A computer cluster is a group of

More information

Data Centric Systems (DCS)

Data Centric Systems (DCS) Data Centric Systems (DCS) Architecture and Solutions for High Performance Computing, Big Data and High Performance Analytics High Performance Computing with Data Centric Systems 1 Data Centric Systems

More information

7 th TYNDP WS. The role of storage in a liberalized market. Georg Dorfleutner RAG Energy Storage GmbH www.rag-energy-storage.

7 th TYNDP WS. The role of storage in a liberalized market. Georg Dorfleutner RAG Energy Storage GmbH www.rag-energy-storage. 7 th TYNDP WS The role of storage in a liberalized market Georg Dorfleutner RAG Energy Storage GmbH www.rag-energy-storage.at Unbundled SSO since 1.1.2013 Operating a storage-pool in Austria - CEGH TOV:

More information

Cloud Workload Planning and Placement: A New Opportunity

Cloud Workload Planning and Placement: A New Opportunity Cloud Workload Planning and Placement: A New Opportunity Table of Contents 3 Challenges 4 Gravitant s Unique Approach 4 cloudmatrix Overview 5 How Does it Work? 5 Application Screener 6 Cloud Compare 6

More information

Indiana University Science with the WIYN One Degree Imager

Indiana University Science with the WIYN One Degree Imager Indiana University Science with the WIYN One Degree Imager Katherine Rhode (Indiana University, WIYN SAC member) Indiana University Department of Astronomy Nine faculty members, plus active emeritus faculty

More information

First Discoveries. Asteroids

First Discoveries. Asteroids First Discoveries The Sloan Digital Sky Survey began operating on June 8, 1998. Since that time, SDSS scientists have been hard at work analyzing data and drawing conclusions. This page describes seven

More information

THESIS PROJECT PROPOSAL FORM

THESIS PROJECT PROPOSAL FORM INSTITUTO DE ASTROFÍSICA DE CANARIAS GRADUATE STUDIES DIVISION THESIS PROJECT PROPOSAL FORM NAME OF THE PROPONENT: Jorge Sánchez Almeida Casiana Muñoz Tuñón Alfonso López Aguerri AFFILIATION: (Please indicate

More information

Creating A Galactic Plane Atlas With Amazon Web Services

Creating A Galactic Plane Atlas With Amazon Web Services Creating A Galactic Plane Atlas With Amazon Web Services G. Bruce Berriman 1*, Ewa Deelman 2, John Good 1, Gideon Juve 2, Jamie Kinney 3, Ann Merrihew 3, and Mats Rynge 2 1 Infrared Processing and Analysis

More information

Canadian Astronomy Data Centre. Séverin Gaudet David Schade Canadian Astronomy Data Centre

Canadian Astronomy Data Centre. Séverin Gaudet David Schade Canadian Astronomy Data Centre Canadian Astronomy Data Centre Séverin Gaudet David Schade Canadian Astronomy Data Centre Data Activities in Astronomy Features of the astronomy data landscape Multi-wavelength datasets are increasingly

More information

The Next Generation Science Standards (NGSS) Correlation to. EarthComm, Second Edition. Project-Based Space and Earth System Science

The Next Generation Science Standards (NGSS) Correlation to. EarthComm, Second Edition. Project-Based Space and Earth System Science The Next Generation Science Standards (NGSS) Achieve, Inc. on behalf of the twenty-six states and partners that collaborated on the NGSS Copyright 2013 Achieve, Inc. All rights reserved. Correlation to,

More information

STAR WARS AND THE ART OF DATA SCIENCE

STAR WARS AND THE ART OF DATA SCIENCE STAR WARS AND THE ART OF DATA SCIENCE MELODIE RUSH, SENIOR ANALYTICAL ENGINEER CUSTOMER LOYALTY Original Presentation Created And Presented By Mary Osborne, Business Visualization Manager At 2014 SAS Global

More information

Observing the Universe

Observing the Universe Observing the Universe Stars & Galaxies Telescopes Any questions for next Monday? Light Doppler effect Doppler shift Doppler shift Spectra Doppler effect Spectra Stars Star and planet formation Sun Low-mass

More information

XtreemOS and Cloud Computing Alvaro Arenas E-Science Centre Science and Technologies Facilities Council, UK XtreemOS in a Nutshell An open source Linux-based Grid Operating System with native VO support

More information

Taming the Internet of Things: The Lord of the Things

Taming the Internet of Things: The Lord of the Things Taming the Internet of Things: The Lord of the Things Kirk Borne @KirkDBorne School of Physics, Astronomy, & Computational Sciences College of Science, George Mason University, Fairfax, VA Taming the Internet

More information

DAME Astrophysical DAta Mining Mining & & Exploration Exploration GRID

DAME Astrophysical DAta Mining Mining & & Exploration Exploration GRID DAME Astrophysical DAta Mining & Exploration on GRID M. Brescia S. G. Djorgovski G. Longo & DAME Working Group Istituto Nazionale di Astrofisica Astronomical Observatory of Capodimonte, Napoli Department

More information

Personal info Birth Date: July 2, 1972 Birth Place: Palmanova (UD) - Italy Address: via IV Novembre, 14-36020 Albettone (VI) - Italy

Personal info Birth Date: July 2, 1972 Birth Place: Palmanova (UD) - Italy Address: via IV Novembre, 14-36020 Albettone (VI) - Italy Tiziano Valentinuzzi Via IV Novembre, 14 36020 Albettone (VI) ITALY Tel.: +39 0444.790752 Cell: +39 320.9569470 E-mail: tiziano.valentinuzzi@unipd.it Personal info Birth Date: July 2, 1972 Birth Place:

More information

The Sloan Digital Sky Survey. From Big Data to Big Database to Big Compute. Heidi Newberg Rensselaer Polytechnic Institute

The Sloan Digital Sky Survey. From Big Data to Big Database to Big Compute. Heidi Newberg Rensselaer Polytechnic Institute The Sloan Digital Sky Survey From Big Data to Big Database to Big Compute Heidi Newberg Rensselaer Polytechnic Institute Summary History of the data deluge from a personal perspective. The transformation

More information

Summary of Data Management Principles South Pole Telescope

Summary of Data Management Principles South Pole Telescope Summary of Data Management Principles South Pole Telescope Experiment description - Where it is: The South Pole Telescope (SPT) is a 10-meter-diameter telescope located at the Amundsen-Scott South Pole

More information

Temperature anisotropy in the solar wind

Temperature anisotropy in the solar wind Introduction Observations Simulations Summary in the solar wind Petr Hellinger Institute of Atmospheric Physics & Astronomical Institute AS CR, Prague, Czech Republic Kinetic Instabilities, Plasma Turbulence

More information

Calculation of Minimum Distances. Minimum Distance to Means. Σi i = 1

Calculation of Minimum Distances. Minimum Distance to Means. Σi i = 1 Minimum Distance to Means Similar to Parallelepiped classifier, but instead of bounding areas, the user supplies spectral class means in n-dimensional space and the algorithm calculates the distance between

More information

How Do Galeries Form?

How Do Galeries Form? 8-5-2015see http://www.strw.leidenuniv.nl/ franx/college/ mf-sts-2015-c9-1 8-5-2015see http://www.strw.leidenuniv.nl/ franx/college/ mf-sts-2015-c9-2 Galaxy Formation Leading questions for today How do

More information