Big Data in Astronomy The Large Synoptic Survey Telescope

Transcription

1 Big Data in Astronomy The Large Synoptic Survey Telescope Prof. Sarah Bridle, University of Manchester 1. The Large Synoptic Survey Telescope (LSST) 2. Big Data challenges in LSST Image simulations Rapid image processing High precision image processing Catalogue search 3. Ways to get involved

2 Big Data in Astronomy The Large Synoptic Survey Telescope Prof. Sarah Bridle, University of Manchester 1. The Large Synoptic Survey Telescope (LSST) 2. Big Data challenges in LSST Image simulations Rapid image processing High precision image processing Catalogue search 3. Ways to get involved

3 Big Data from 3.2G pixel camera 2000 exposures per night -> 20TB per night 10 year survey

4 Big Camera

5 Big Telescope

6 Big Data from Within its first month of operation LSST will survey more of the Universe than all previous telescopes built by mankind

7

8

9

10

11

12

13 Big Data from 800 images (movie) of the southern hemisphere in 6 colours ~ alerts/ night worldwide, within 60 seconds

14 14

15 15

16

17 LSST Basics 8.4m mirror 9.6 sq deg FOV 20,000 deg of sky 1000 visits per field filters: ugrizy nm r ~24.7 in single visit, ~27.7 stacked depth 3.2 Gpix camera ~0.01 mag precision photometry

18 Big Collaboration a subset in Tucson Arizon

19 Big Data in Astronomy The Large Synoptic Survey Telescope Prof. Sarah Bridle, University of Manchester 1. The Large Synoptic Survey Telescope (LSST) 2. Big Data challenges in LSST Image simulations Rapid image processing High precision image processing Catalogue search 3. Ways to get involved

20 Big Data in Astronomy The Large Synoptic Survey Telescope Prof. Sarah Bridle, University of Manchester 1. The Large Synoptic Survey Telescope (LSST) 2. Big Data challenges in LSST Image simulations Rapid image processing High precision image processing Catalogue search 3. Ways to get involved

21 LSST Data Products

22 LSST'science'and'engineering'tools'' All%soKware%is%version%controlled%and%provenance%informa) on%is%output%with%the%data. % Systems%are%validated%through%project%ini) ated%reviews%with%external%members%.% Slide by Andy Connolly, LSST Simulation Scientist 3

23 C/C++/Python% The'simula4on'framework' CatSim% PhoSim% Slide by Andy Connolly, LSST Simulation Scientist 15

24 0.2 % Op) cal%model %+Tracking %+Diffrac) on %+Det%Perturba) ons% % % % % % % % % +Lens%Perturba) ons %+Mirror%Perturba) ons %+Detector %+Dome%Seeing% % % % % % % % % +Low%Al) tude %+Mid%Al) tude %+High%Al) tude %+Pixeliza) on% %Atmosphere %Atmosphere %Atmosphere% 12 PhoSim Peterson%et%al%2013%

25 3%Gigapixels% % 10%sq.%degrees% % 20%million%sources% % %photons% % 11%Gbytes% % 1000%CPU%hours% % 14

26 Big Data in Astronomy The Large Synoptic Survey Telescope Prof. Sarah Bridle, University of Manchester 1. The Large Synoptic Survey Telescope (LSST) 2. Big Data challenges in LSST Image simulations Rapid image processing High precision image processing Catalogue search 3. Ways to get involved

27 Flux Flux Flux Flux Flux Flux g r Supernova 0 r Classification r SNPhotCC (Kessler et 0 al 2011) i i i z z T obs T obs T obs SN SDSS 2007og z=0.2 SN SDSS z=0.14 SN SDSS 2006kn z= u 0 u 0 u g g g r r r i i Leads in LSST: Alex Kim, Michael Wood-Vasey Leads in LSST:UK: Mark Sullivan (Southampton), Hiranya Peiris (U i

28 Table adapted from Rau et al Slide by Lucianne Walkowicz, Co-Chair of Transients and Variable Sta Expected Rate of Transients Class Mag t (days) Universal Rate LSST Rate Luminous SNe Mpc -3 yr Orphan Afterglows SHB Orphan Afterglows LSB On- axis GRB afterglows Tidal Disruption Flares Luminous Red Novae x Mpc -3 yr -1 ~ x Mpc -3 yr Mpc -3 yr -1 ~ Mpc -3 yr yr -1 Lsun Fallback SNe <5 x 10-6 Mpc -3 yr -1 < 800 SNe Ia x 10-5 Mpc -3 yr SNe II (3..8) x 10-5 Mpc -3 yr

29 TABLE 5 List of Par t icipant s in t he SNPhot CC. Classified SN Part icipant s Abbreviat ion a +Z b /noz c z d ph CPU e Descript ion (st rat egy class f ) P. Belov and S. Glazov Belov & Glazov yes/ no no 90 light curve χ 2 test against Nugent templates (2) S. Gonzalez Gonzalez yes/ yes no 120 cuts on SiFT O fit χ 2 and fit paramet ers (1) J. Richards, Homrighausen, InCA g no/ yes no 1 Spline fit & nonlinear dimensionality C. Schafer, P. Freeman reduct ion (4) J. Newling, M. Varuguese, JEDI-K DE yes/ yes no 10 K ernel Density Evaluat ion with 21 params (4) B. Basset t, R. Hlozek, JEDI Boost yes/ yes no 10 Boost ed decision t rees (4) D. Parkinson, M. Smit h, JEDI-Hubble yes/ no no 10 Hubble diagram K DE (3) H. Campbell, M. Hilt on, JEDI Combo yes/ no no 10 Boost ed decision t rees + Hubble K DE (3+ 4) H. Lampeit l, M. Kunz, P. Pat el (JEDI group h ) S. Philip, V. Bhat nagar, M GU+ DU-1 i no/ yes no < 1 light curve slopes & Neural Network (2) A. Singhal, A. Rai, M GU+ DU-2 no/ yes no < 1 light curve slopes & Random Forest s (2) A. M ahabal, K. Indulekha H. Campbell, B. Nichol, Port smout h χ 2 yes/ no no 1 SA LT 2 χ 2 r & False Discovery Rat e St at ist ic (1) H. L ampiet l, M.Smit h Port smout h-hubble yes/ no no 1 Deviat ion from paramet rized Hubble diagram (3) D. Poznanski Poz2007 RAW yes/ no yes 2 SN A ut omat ed Bayesian Classifier (SN A BC) (2) Poz2007 OPT yes/ no yes 2 SN A BC wit h cut s t o opt imize C FoM Ia (2). S. Rodney Rodney yes/ yes yes 230 SN Ont ology wit h Fuzzy Templat es (2) M. Sako Sako yes/ yes yes 120 χ 2 test against grid of Ia/ I I/ Ibc templates (2) S. K uhlmann, R. K essler SNA NA cuts yes/ yes yes 2 Cut on ml cs fit probability, S/ N & sampling (1) a Groups are list ed alphabet ically by abbreviat ion. b Classificat ions included for SNPhot CC/ HOSTZ. c Classificat ions included for SNPhot CC/ nohostz. d phot o-z est imat es included. e Average processi ng t ime per SN (seconds) usi ng si milar 2-3 GHz cores. f From 3, st rat egy classes are 1) select ion cut s, 2) Bayesian probabilit ies, 3) Hubble-diagram paramet rizat ion and 4) st at istical inference. g Int er nat ional Comput at ional A st rophysics Group: ht t p: / / www. i ncagr oup. org h Joint Exchange and Development Init iat ive: ht t p: / / j edi. saao. ac. za i MGU= Mahat ma Gandhi University, DU= Delhi University. best method in this first SNPhot CC, here we carefully examine the C FoM Ia for the unconfirmed sample in the SNPhot CC/ HOSTZ (Fig. 4). The entry with the highest SNPhotCC (Kessler et al 2011) subset was generally treated as a random subset, which it clearly is not ( 2.5). T he magnit ude-limit ed select ion of spectroscopic targets resulted in the selection of brighter

30 Big Data in Astronomy The Large Synoptic Survey Telescope Prof. Sarah Bridle, University of Manchester 1. The Large Synoptic Survey Telescope (LSST) 2. Big Data challenges in LSST Image simulations Rapid image processing High precision image processing Catalogue search 3. Ways to get involved

31 It s a Big Deal Discovery of Accelerating Universe Wins 2011 Nobel Prize

32 Why is the Universe Accelerating? Einstein s cosmological constant A new fluid called Dark Energy Equation of state w = p/ General Relativity is wrong

33 Using the bending of light to see the invisible

34

35

36 Cosmic Shear Galaxies seen through dark matter distribution analogous to Streetlamps seen through your bathroom window

37 Cosmic Shear g i ~0.2 Real data: g i ~0.03

38 Atmosphere and Telescope Convolution with kernel Real data: Kernel size ~ Galaxy size

39 Pixelisation Sum light in each square Real data: Pixel size ~ Kernel size /2

40 Noise Mostly Poisson. Some Gaussian and bad pixels. Uncertainty on total light ~ 5 per cent

41 Bridle et al 2010

42 A typical galaxy image for cosmic shear Intrinsic galaxy shape b/a ~ 0.5 Uncertainty due to no σb/a ~ 0.5 Modification due to le Δb/a ~ 0.05 Effect of changing w b δb/a ~

43 Annals of Applied Statistics March 2009

44 Slide by David Hogg Following NIPS Cosmology Workshop discussion with Iain Mu

45 Typical Running+Joseph s+code+ DES data multiple exposures Image+ Model+ Weight+ Residuals DESDM data, PSFs; im3shape fit (Zuntz, Hirsch, Kacprzak, Rowe, Ma

46 Successful fits How to deal with overlaps? interloper target model mask removes interloper lovely residuals DESDM data, PSFs; im3shape fit (Zuntz, Hirsch, Kacprzak, Rowe, Ma

47 Slide by David Kirkby Today DES-r 800s 13.7 electrons

48 Slide by David Kirkby In 10 years LSST-r 6900s 13.7 electrons

49

50 Big Data in Astronomy The Large Synoptic Survey Telescope Prof. Sarah Bridle, University of Manchester 1. The Large Synoptic Survey Telescope (LSST) 2. Big Data challenges in LSST Image simulations Rapid image processing High precision image processing Catalogue search 3. Ways to get involved

51 Catalogue search in phase space Find remnants of galaxies colliding with the Milky Way From positions and velocities of 10 billion stars The Sagittarius dwarf galaxy in o Leads in LSST: John Bochanski, Nitya Jacob Kallivayalil, Beth W Leads in LSST:UK: Vasily Belokurov (Cambridge), Nic Walton (Cam

52 Big Data in Astronomy The Large Synoptic Survey Telescope Prof. Sarah Bridle, University of Manchester 1. The Large Synoptic Survey Telescope (LSST) 2. Big Data challenges in LSST Image simulations Rapid image processing High precision image processing Catalogue search 3. Ways to get involved

53 LSST Scientific Possibilities LSST Science Book: science/scibook 598 pages 245 authors Preface 8. The Transient and Variable Universe 1. Introduction 9. Galaxies 2. LSST System Design 10. Active Galactic Nuclei 3. System Pergormance 11. Supernovae 4. Education and Public Outreach 12. Strong Lenses 5. The Solar System 13. Large-Scale Structure 6. Stellar Populations 14. Weak Lensing 7. Milky Way and Local Volume 15. Cosmological Physics

54 Science Collaborations Solar System Milky Way and Local Volume Structure Transients & Variable Stars Galaxies Active Galactic Nuclei Supernovae Stellar Populations Strong Lensing Weak Lensing Large Scale Structure & Baryon Oscillations Informatics & Statistics

55 Science Collaborations Solar System Milky Way and Local Volume Structure Transients & Variable Stars Galaxies Active Galactic Nuclei Supernovae Stellar Populations Strong Lensing Weak Lensing Large Scale Structure & Baryon Oscillations Informatics & Statistics LSST:UK I&S Leads: Hiranya Peiris (UCL), Jason McEwen (UCL)

56 Slide by Kirk Bourne, Dept of Computational & Data Sciences Ge University

57 Sign up to get involvedhttps://docs.google.com/spreadsheet/ccc?key=0aqx4pj9ojyrudfvjq U85SS02eEZxeEhTaUJKYmZjVmc&usp=sharing Current status: Submitted 40 page proposal to STFC. PPRP panel presentation on 27 th October 2014

58 Open Problems Related to LSST Shear measurement (GREAT08,, GREAT3) Cosmological parameter estimation (e.g. CosmoSIS) LSST simulations (CatSim, PhoSim, ImSim) Real-time transient classification Supernova Classification Challenge Catalogue search Dark Worlds Kaggle Challenge Strong Lens Time Delay Challenge Communication in large collaborations

59

60 Noise Bias Many identical images with different noise

61 Bias disappears at high S/N Above requirements at low S/N

62 What causes the bias? For model fitting methods Noise bias Refregier, SB et al; Kacprzak, SB et al 2012 Maximum likelihood methods are biased Calibration works well enough Model bias Voigt & Bridle 2009 e.g use wrong profile in fit e.g. use elliptical isophote model in fit

63 Galaxy Models But galaxies aren t simple Model galaxy Actual galaxy

64 Model Bias The effect of realistic galaxy shapes Measure with sims from HST data Bias for red and blue galaxies shown DES 5-year requires mean m < Plots from Tomasz Kacprzak

65 Impact on dark energy constraints Simulate for different redshifts Kacprzak, SB, et al 2013

66

67

68

69 69/19 Taken from Bridle et al GREAT08 Handb

70 Slide by Kirk Bourne, Dept of Computational & Data Sciences Ge University

71 71/19 Typical gala used for cos shear analy Typical star Used for finding Convolution kernel

72 Big Data in Astronomy The Large Synoptic Survey Telescope 1. The Large Synoptic Survey Telescope (LSST) 2. Big Data challenges in LSST 3. Weak Lensing in LSST 3.1 Big Data: Galaxy shape measurement 3.2 Big Models: Covariance matrix estimation 3.3 It s a Big Deal: Proving Einstein wrong 4. Ways to get involved

73

74

75 Big Data in Astronomy The Large Synoptic Survey Telescope 1. The Large Synoptic Survey Telescope (LSST) 2. Big Data challenges in LSST 3. Weak Lensing in LSST 3.1 Big Data: Galaxy shape measurement 3.2 Big Models: Covariance matrix estimation 3.3 It s a Big Deal: Proving Einstein wrong 4. Ways to get involved